diff --git a/.agents/docs/2026-06-10-cli-modularization.md b/.agents/docs/2026-06-10-cli-modularization.md new file mode 100644 index 00000000..71d52950 --- /dev/null +++ b/.agents/docs/2026-06-10-cli-modularization.md @@ -0,0 +1,180 @@ +# CLI Modularization — Architecture & Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Reduce `src/cli.cppm` (6192 lines) to a thin command-dispatch layer (≤ 500 lines) by moving every implementation concern into focused C++20 modules, with zero behavior change. + +**Architecture:** Continue the PR-R4/PR-R5 extraction series (`2026-05-08-pm-subsystem-architecture.md`). `cli.cppm` keeps only the canonical usage screen and the `cmdline::App` dispatcher; commands move into `src/cli/cmd_*.cppm` modules grouped by CLI surface; cross-cutting CLI plumbing moves into `mcpp.cli.common` / `mcpp.cli.install_ui`; the build-orchestration core (`prepare_build`) becomes `mcpp.cli.build`; toolchain payload post-install fixups move into the toolchain domain as `mcpp.toolchain.post_install`. Every move is a strict zero-behavior-change relocation (same statement order, same messages, same exit codes), exactly like the PR-R5 precedent. + +**Tech Stack:** C++23 named modules (GCC 16 self-host), mcpp convention-mode source globbing (`src/**/*.cppm` — no build-file edits needed), existing `mcpp test` unit suite + `tests/e2e/run_all.sh`. + +--- + +## 1. Current state (analysis) + +`src/cli.cppm` mixes five unrelated responsibilities in one 6192-line module: + +| Lines (pre-refactor) | Content | Problem | +|---|---|---| +| 75–181 | usage screen, project/workspace root discovery | shared helpers trapped in CLI module (pm.commands keeps a private copy of `find_manifest_root` for this reason) | +| 183–517 | toolchain version matching, xlings NDJSON install-progress UI | UI plumbing interleaved with command logic | +| 519–1038 | fingerprint flag canonicalization, gcc/clang payload post-install fixups (patchelf, specs, cfg rewrite) | toolchain-domain logic living in the CLI layer | +| 1051–3561 | `cmd_new` + templates, `BuildContext` + `prepare_build` (≈ 2 240 lines: workspace → toolchain → dependency resolution → features → modgraph → fingerprint → plan → lockfile) | the build pipeline is unreachable for unit testing and unreviewable as a diff target | +| 3563–5789 | build cache/fast-path + 25 `cmd_*` entry points | every command edit rebuilds/reviews the whole module | +| 5793–6192 | `run()` dispatcher | the only part that belongs here | + +Consequences: slowest incremental rebuild unit in the repo, high merge-conflict surface, no module-level ownership boundaries, and `mcpp.cli` exports nothing reusable (`pm.commands` duplicates helpers to avoid a circular import). + +## 2. Target architecture + +``` +src/main.cpp ──▶ mcpp.cli (dispatcher only: usage + cmdline::App + run()) + │ + ├─ mcpp.cli.cmd_build build / run / test / clean / dyndep + │ └─ mcpp.cli.build BuildContext, BuildOverrides, prepare_build + ├─ mcpp.cli.cmd_new new + package templates + ├─ mcpp.cli.cmd_registry search / index * + ├─ mcpp.cli.cmd_cache cache list|info|prune|clean + ├─ mcpp.cli.cmd_toolchain toolchain install|list|default|remove + │ └─ mcpp.toolchain.post_install patchelf/specs/cfg fixups + ├─ mcpp.cli.cmd_publish publish / pack / emit xpkg + ├─ mcpp.cli.cmd_self self * / doctor / why / env / explain + └─ mcpp.pm.commands add / remove / update (PR-R5, unchanged) + +shared CLI plumbing: + mcpp.cli.common project/workspace root discovery, target_dir, fs size helpers + mcpp.cli.install_ui xlings NDJSON → ui::DownloadProgress adapters, PathContext +``` + +Layering rules (enforced by module imports — cycles are compile errors): + +1. `mcpp.cli` imports only `cmd_*` modules (+ `pm.commands`, `ui`, `log`, `cmdline`, `toolchain.fingerprint` for the version string). +2. `cmd_*` modules never import each other; shared code lives in `common` / `install_ui` / `build`. +3. `mcpp.cli.build` is the single owner of `prepare_build`; consumers are `cmd_build`, `cmd_publish` (pack), `cmd_self` (doctor/why). +4. `mcpp.toolchain.post_install` is CLI-independent (imports only config/xlings/platform/log/ui) so future non-CLI callers (e.g. a daemonized installer) can reuse it. +5. Export surface is explicit per symbol (`export` on the declaration) — internals keep module linkage. + +### Module inventory + +| New file | Module | Exports | Body source (line ranges in pre-refactor `src/cli.cppm`) | +|---|---|---|---| +| `src/cli/common.cppm` | `mcpp.cli.common` | `find_manifest_root`, `find_workspace_root`, `merge_workspace_deps`, `target_dir`, `dir_size`, `human_bytes` | 116–181, 4630–4650 (drop `static`) | +| `src/cli/install_ui.cppm` | `mcpp.cli.install_ui` | `make_path_ctx`, `make_bootstrap_progress_callback`, `CliInstallProgress` | 300–517 | +| `src/toolchain/post_install.cppm` | `mcpp.toolchain.post_install` (ns `mcpp::toolchain`) | `patchelf_walk`, `fixup_clang_cfg`, `gcc_post_install_fixup` | 722–1038 | +| `src/cli/build.cppm` | `mcpp.cli.build` | `BuildContext`, `BuildOverrides`, `prepare_build` | 519–720, 1321–3561 | +| `src/cli/cmd_build.cppm` | `mcpp.cli.cmd_build` | `cmd_build`, `cmd_run`, `cmd_test`, `cmd_clean`, `cmd_dyndep` | 3563–3976, 4406–4570, 4608–4627, 5741–5789 | +| `src/cli/cmd_new.cppm` | `mcpp.cli.cmd_new` | `cmd_new` | 1051–1319 | +| `src/cli/cmd_registry.cppm` | `mcpp.cli.cmd_registry` | `cmd_search`, `cmd_index_{list,add,remove,update,pin,unpin}` | 3996–4402 | +| `src/cli/cmd_cache.cppm` | `mcpp.cli.cmd_cache` | `cmd_cache_{list,info,prune,clean}` | 4837–4976 | +| `src/cli/cmd_toolchain.cppm` | `mcpp.cli.cmd_toolchain` | `cmd_toolchain` | 183–298, 4978–5317 | +| `src/cli/cmd_publish.cppm` | `mcpp.cli.cmd_publish` | `cmd_publish`, `cmd_pack`, `cmd_emit_xpkg` | 4572–4606, 5319–5571 | +| `src/cli/cmd_self.cppm` | `mcpp.cli.cmd_self` | `cmd_env`, `cmd_doctor`, `cmd_why`, `cmd_explain`, `cmd_explain_action`, `cmd_self_{version,init,config}` | 3978–3994, 4652–4835, 5573–5739 | + +`src/cli.cppm` keeps: file header, `print_usage` (75–114), `run()` (5795–6190, minus the now-dead `using namespace mcpp::cli::detail;`). All moved code switches namespace `mcpp::cli::detail` → `mcpp::cli` (or `mcpp::toolchain` for post_install); unqualified cross-references keep working because callers share the namespace. The three post-install call sites outside the toolchain namespace gain explicit `mcpp::toolchain::` qualification. + +### Cross-platform notes + +- No platform-conditional code is touched; `if constexpr (mcpp::platform::is_macos/…)` blocks move verbatim, so Linux/macOS/Windows behavior is bit-identical. +- Source discovery is glob-based (`src/**/*`), so the new `src/cli/` directory needs no manifest change on any platform. +- Each new module keeps the `module; #include #include ` global-module-fragment prologue `cli.cppm` used (for `stderr`/`stdout` macros) — required on all three toolchains. + +## 3. Implementation plan + +The extraction is a single mechanical transformation of one immutable source revision, so it is scripted with `sed -n 'A,Bp'` range extraction (no hand-retyping; byte-identical bodies), then compiled and tested. Verification = full self-host build + unit suite + e2e suite. + +### Task 1: Architecture doc (this file) + +- [x] **Step 1:** Write this document. +- [x] **Step 2:** Commit on branch `refactor/cli-modularization`. + +### Task 2: Scripted extraction + +- [x] **Step 1:** `cp src/cli.cppm /tmp/cli_orig.cppm` (immutable line-range source). +- [x] **Step 2:** For each row of the module inventory: write the module header (GMF prologue, `export module`, imports per §2, `namespace … {`), then `sed -n 'A,Bp' /tmp/cli_orig.cppm >>` the body ranges in table order, then close the namespace. Import lists per module: + - common: `std, mcpp.manifest, mcpp.toolchain.detect, mcpp.toolchain.fingerprint` + - install_ui: `std, mcpp.ui, mcpp.log, mcpp.config, mcpp.fetcher` + - post_install: `std, mcpp.config, mcpp.xlings, mcpp.platform, mcpp.log, mcpp.ui` + - build: `std, mcpp.libs.json, mcpp.manifest, mcpp.modgraph.{graph,scanner,validate}, mcpp.toolchain.{clang,detect,fingerprint,registry,stdmod}, mcpp.toolchain.post_install, mcpp.build.plan, mcpp.lockfile, mcpp.config, mcpp.xlings, mcpp.platform, mcpp.fetcher, mcpp.pm.{resolver,index_spec,mangle,compat,dep_spec}, mcpp.ui, mcpp.log, mcpp.fallback.install_integrity, mcpp.bmi_cache, mcpp.cli.common, mcpp.cli.install_ui` + - cmd_build: `std, mcpplibs.cmdline, mcpp.cli.{build,common,install_ui}, mcpp.build.{plan,backend,ninja}, mcpp.bmi_cache, mcpp.dyndep, mcpp.manifest, mcpp.modgraph.scanner, mcpp.toolchain.stdmod, mcpp.xlings, mcpp.platform, mcpp.ui, mcpp.log` + - cmd_new: `std, mcpplibs.cmdline, mcpp.cli.install_ui, mcpp.scaffold, mcpp.config, mcpp.manifest, mcpp.fetcher, mcpp.pm.resolver, mcpp.ui` + - cmd_registry: `std, mcpplibs.cmdline, mcpp.cli.{common,install_ui}, mcpp.config, mcpp.xlings, mcpp.fetcher, mcpp.manifest, mcpp.lockfile, mcpp.ui` + - cmd_cache: `std, mcpplibs.cmdline, mcpp.cli.common, mcpp.toolchain.stdmod, mcpp.ui` + - cmd_toolchain: `std, mcpplibs.cmdline, mcpp.cli.{common,install_ui}, mcpp.toolchain.{detect,registry,post_install}, mcpp.config, mcpp.xlings, mcpp.fetcher, mcpp.manifest, mcpp.ui, mcpp.log` + - cmd_publish: `std, mcpplibs.cmdline, mcpp.cli.{common,build,install_ui}, mcpp.manifest, mcpp.modgraph.scanner, mcpp.publish.xpkg_emit, mcpp.pack, mcpp.build.{backend,ninja}, mcpp.config, mcpp.platform, mcpp.ui` + - cmd_self: `std, mcpplibs.cmdline, mcpp.cli.{common,build,install_ui}, mcpp.config, mcpp.xlings, mcpp.fallback.install_integrity, mcpp.toolchain.{detect,fingerprint,stdmod}, mcpp.build.plan, mcpp.ui` +- [x] **Step 3:** Rewrite `src/cli.cppm`: header + imports (`std, mcpplibs.cmdline, mcpp.ui, mcpp.log, mcpp.toolchain.fingerprint, mcpp.pm.commands, mcpp.cli.cmd_*×7`) + exported `run()` decl + `print_usage` + `run()` body (drop the `using namespace …::detail;` line). +- [x] **Step 4:** Add `export` keywords on the symbols listed in §2 (Edit per declaration); drop `static` from `dir_size`/`human_bytes`; qualify the three `mcpp::toolchain::` post-install call sites (one in build.cppm: `gcc_post_install_fixup`; two-plus-one in cmd_toolchain.cppm: `gcc_post_install_fixup`, `patchelf_walk`, `fixup_clang_cfg`). + +### Task 3: Build & fix loop + +- [x] **Step 1:** `mcpp build` — fix any missing-import / linkage errors (expected failure mode: a helper referenced across modules that wasn't exported; fix = add `export` or import, never duplicate code). +- [x] **Step 2:** `wc -l src/cli.cppm` — expected < 500. +- [x] **Step 3:** Commit `refactor(cli): split cli.cppm into focused modules`. + +### Task 4: Verification + +- [x] **Step 1:** `mcpp test` with the freshly built binary — expect all unit tests pass. +- [x] **Step 2:** `MCPP= tests/e2e/run_all.sh` — expect all e2e tests pass (covers help/version text, exit codes 0/1/2/127, all command surfaces). +- [x] **Step 3:** Sanity: `mcpp --help`, `mcpp version`, `mcpp self doctor`, unknown-command exit 127. + +### Task 5: PR + CI + +- [x] **Step 1:** Push branch, open PR describing motivation, module map, zero-behavior-change guarantee, verification evidence. +- [x] **Step 2:** Watch ci-linux / ci-macos / ci-windows / fresh-install lanes; fix-forward on any platform-specific module issue (most likely candidate: MSVC/clang module-linkage strictness on exported-vs-internal helpers). +- [x] **Step 3:** Update this doc's status section. + +## 4. Phase 2 — domain relocation (cli = parse + route only) + +Phase 1 made `cli.cppm` a dispatcher but left implementations in `mcpp.cli.*` +modules. Phase 2 finishes the architecture: every implementation lives in its +owning subsystem, and `cli/cmd_*` modules contain ONLY argument handling, +validation of CLI shapes, and routing. Relocation map (bodies verbatim again): + +| Phase-1 location | Phase-2 owner | Contents | +|---|---|---| +| `mcpp.cli.common` | `mcpp.project` (`src/project.cppm`) | `find_manifest_root`, `find_workspace_root`, `merge_workspace_deps` (also folds the private copy `pm.commands` kept) | +| `mcpp.cli.common` | `mcpp.bmi_cache.maintenance` | `dir_size`, `human_bytes` | +| `mcpp.cli.common` | `mcpp.build.prepare` (internal) | `target_dir` | +| `mcpp.cli.install_ui` | `mcpp.fetcher.progress` (`src/fetcher/progress.cppm`) | NDJSON→ui adapters (`CliInstallProgress` → `InstallProgressHandler`), `make_bootstrap_progress_callback`, `make_path_ctx` | +| `mcpp.cli.build` | `mcpp.build.prepare` (`src/build/prepare.cppm`) | `BuildContext`, `BuildOverrides`, `prepare_build` | +| `mcpp.cli.cmd_build` | `mcpp.build.execute` (`src/build/execute.cppm`) | build cache + fast path, `run_build_plan`, `try_fast_build`, `build_run_target`, `run_tests`, `clean_project` | +| `mcpp.cli.cmd_toolchain` | `mcpp.toolchain.lifecycle` (`src/toolchain/lifecycle.cppm`) | version matching + `toolchain_list/install/set_default/remove` | +| `mcpp.cli.cmd_registry` | `mcpp.pm.index_management` (`src/pm/index_management.cppm`) | `search_packages`, `index_list/add/remove/update/pin/unpin` | +| `mcpp.cli.cmd_cache` | `mcpp.bmi_cache.maintenance` (`src/bmi_cache/maintenance.cppm`) | `cache_list/info/prune/clean` | +| `mcpp.cli.cmd_new` | `mcpp.scaffold.create` (`src/scaffold/create.cppm`) | template fetch/instantiate + `create_builtin_project` | +| `mcpp.cli.cmd_publish` | `mcpp.publish.pipeline` (`src/publish/pipeline.cppm`) | `publish_package`, `emit_xpkg_to` | +| `mcpp.cli.cmd_publish` | `mcpp.pack.pipeline` (`src/pack/pipeline.cppm`) | `build_and_pack` | +| `mcpp.cli.cmd_self` | `mcpp.doctor` (`src/doctor.cppm`) | `env_report`, `doctor_report`, `why_report`, `explain_code`, `self_init`, `self_config` | + +Resulting cli layer: `cli.cppm` (dispatcher, 481) + seven `cmd_*` adapters +totalling ~450 lines, none containing domain logic. Domain ops take plain +typed parameters (never `ParsedArgs`); `mcpplibs.cmdline` is imported only by +the cli layer. The split rule for each command: CLI-shape validation and +usage errors stay in the adapter; everything after lives in the domain op +with identical statements, messages and exit codes. + +Naming rule: module names state the responsibility (`maintenance`, `create`, +`pipeline`, `lifecycle`, `index_management`) — grab-bag suffixes like `ops`, +`manager` or `utils` are not acceptable module names in this codebase. + +## 5. Follow-ups (out of scope here) + +- Decompose `prepare_build` internally (workspace / toolchain / dep-resolution / feature phases) now that it lives in `mcpp.build.prepare`. +- Tighten `mcpp.build.prepare`'s import list (it inherited the union of the old `cli.cppm` imports). +- `pm.commands` still takes `ParsedArgs` (it predates the parse/route rule); migrating add/remove/update bodies behind typed ops would complete the pattern. + +## 6. Status + +- 2026-06-10: extraction done. `cli.cppm` 6192 -> 481 lines; 11 new modules. + Verified: self-host build clean; unit suite 18/18 pass; e2e 67 pass / + 1 skip, with the only failures being the 6 `llvm_*` tests that fail + identically with the pre-refactor baseline binary on the same host + (local LLVM payload cannot exec — environment issue, not a regression). + PR opened; CI green on linux/windows/macos. +- 2026-06-10 (phase 2): domain relocation executed — implementations moved out + of `mcpp.cli.*` into `mcpp.project`, `mcpp.fetcher.progress`, + `mcpp.build.{prepare,execute}`, `mcpp.toolchain.lifecycle`, + `mcpp.pm.index_management`, `mcpp.bmi_cache.maintenance`, `mcpp.scaffold.create`, + `mcpp.publish.pipeline`, `mcpp.pack.pipeline`, `mcpp.doctor`; `cli/cmd_*` reduced to + parse + route adapters (~450 lines total). Self-host build + 18/18 unit + tests pass; e2e parity with baseline re-verified. diff --git a/CHANGELOG.md b/CHANGELOG.md index b69a4284..bbc4e4b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,24 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [0.0.54] — 2026-06-10 + +### 修复 + +- `mcpp new --template `:对声明了命名空间的模板包(如 + `mcpplibs.llmapi` 以裸名 `llmapi` 引用)现在能从描述符派生出 + (namespace, shortName) 坐标,正确完成 semver 解析与安装(#130)。 + +### 其他 + +- 架构重构(零行为变更):`cli.cppm` 从 6192 行精简为约 480 行的纯命令 + 分发层;`src/cli/cmd_*` 仅保留参数解析与路由,全部领域实现下沉到属主 + 子系统 —— `mcpp.build.{prepare,execute}`、`mcpp.toolchain.{post_install, + lifecycle}`、`mcpp.pm.index_management`、`mcpp.bmi_cache.maintenance`、 + `mcpp.scaffold.create`、`mcpp.publish.pipeline`、`mcpp.pack.pipeline`、 + `mcpp.doctor`、`mcpp.project`、`mcpp.fetcher.progress`。 + 设计与迁移记录见 `.agents/docs/2026-06-10-cli-modularization.md`。 + ## [0.0.53] — 2026-06-09 ### 新增 diff --git a/mcpp.toml b/mcpp.toml index 7d856e24..cadb82ec 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "0.0.53" +version = "0.0.54" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/bmi_cache/maintenance.cppm b/src/bmi_cache/maintenance.cppm new file mode 100644 index 00000000..3d781293 --- /dev/null +++ b/src/bmi_cache/maintenance.cppm @@ -0,0 +1,180 @@ +// mcpp.bmi_cache.maintenance — global BMI cache inspection + pruning, and the +// shared fs-size/byte-formatting helpers they are built on. +// Bodies moved verbatim from the CLI layer. Zero behavior change. + +module; +#include +#include + +export module mcpp.bmi_cache.maintenance; + +import std; +import mcpp.toolchain.stdmod; +import mcpp.ui; + +namespace mcpp::bmi_cache { + + +export std::uintmax_t dir_size(const std::filesystem::path& p) { + std::error_code ec; + if (!std::filesystem::exists(p, ec)) return 0; + std::uintmax_t total = 0; + for (auto& e : std::filesystem::recursive_directory_iterator(p, ec)) { + if (ec) break; + std::error_code ec2; + if (e.is_regular_file(ec2) && !ec2) { + total += e.file_size(ec2); + } + } + return total; +} + +export std::string human_bytes(std::uintmax_t n) { + constexpr const char* units[] = {"B", "KiB", "MiB", "GiB", "TiB"}; + double v = static_cast(n); + int u = 0; + while (v >= 1024.0 && u < 4) { v /= 1024.0; ++u; } + return std::format("{:.1f} {}", v, units[u]); +} + + +// ─── M4 #4: mcpp cache list / prune / clean / info ────────────────────── +struct CacheEntry { + std::filesystem::path dir; + std::string fingerprint; + std::string pkgAtVer; // "/@" + std::uintmax_t size = 0; + std::filesystem::file_time_type lastWrite{}; + std::size_t fileCount = 0; +}; + +static std::vector walk_cache_entries() { + std::vector entries; + auto bmi = mcpp::toolchain::default_cache_root(); + std::error_code ec; + if (!std::filesystem::exists(bmi, ec)) return entries; + + for (auto& fpEntry : std::filesystem::directory_iterator(bmi, ec)) { + auto fpDir = fpEntry.path(); + auto depsDir = fpDir / "deps"; + if (!std::filesystem::exists(depsDir, ec)) continue; + for (auto& idxEntry : std::filesystem::directory_iterator(depsDir, ec)) { + for (auto& pkgEntry : std::filesystem::directory_iterator(idxEntry.path(), ec)) { + CacheEntry e; + e.dir = pkgEntry.path(); + e.fingerprint = fpDir.filename().string(); + e.pkgAtVer = idxEntry.path().filename().string() + + "/" + pkgEntry.path().filename().string(); + e.size = dir_size(e.dir); + e.lastWrite = std::filesystem::last_write_time(e.dir, ec); + for (auto& _ : std::filesystem::recursive_directory_iterator(e.dir, ec)) { + if (!ec) ++e.fileCount; + } + entries.push_back(std::move(e)); + } + } + } + return entries; +} + +static std::string format_age(std::filesystem::file_time_type t) { + auto now = std::chrono::file_clock::now(); + auto diff = std::chrono::duration_cast(now - t).count(); + if (diff < 60) return std::format("{}s ago", diff); + if (diff < 3600) return std::format("{}m ago", diff / 60); + if (diff < 86400) return std::format("{}h ago", diff / 3600); + return std::format("{}d ago", diff / 86400); +} + +// `mcpp cache` is dispatched at the App level — list / info / prune / clean +// each get their own action lambda invoking one of these helpers. + +// `mcpp cache list`. +export int cache_list() { + auto entries = walk_cache_entries(); + if (entries.empty()) { + std::println("(BMI cache is empty)"); + return 0; + } + std::println("{:<18} {:>10} {:>14} {}", + "fingerprint", "size", "last accessed", "package"); + for (auto& e : entries) { + auto fp = e.fingerprint.size() > 16 + ? e.fingerprint.substr(0, 16) : e.fingerprint; + std::println("{:<18} {:>10} {:>14} {}", + fp, human_bytes(e.size), format_age(e.lastWrite), e.pkgAtVer); + } + return 0; +} + +// `mcpp cache info @`. +export int cache_info(const std::string& needle) { + auto entries = walk_cache_entries(); + for (auto& e : entries) { + if (e.pkgAtVer.ends_with(needle)) { + std::println("dir = {}", e.dir.string()); + std::println("fingerprint = {}", e.fingerprint); + std::println("package = {}", e.pkgAtVer); + std::println("size = {}", human_bytes(e.size)); + std::println("file count = {}", e.fileCount); + std::println("last write = {}", format_age(e.lastWrite)); + return 0; + } + } + std::println("no cache entry matching '{}'", needle); + return 1; +} + +// `mcpp cache prune --older-than {s,m,h,d}` (v = raw option value). +export int cache_prune(const std::string& v) { + if (v.empty()) { + mcpp::ui::error("`mcpp cache prune` requires --older-than {s,m,h,d}"); + return 2; + } + char unit = v.back(); + long long n = 0; + try { n = std::stoll(v.substr(0, v.size() - 1)); } + catch (...) { mcpp::ui::error(std::format("bad --older-than value '{}'", v)); return 2; } + std::chrono::seconds threshold{0}; + if (unit == 's') threshold = std::chrono::seconds(n); + else if (unit == 'm') threshold = std::chrono::seconds(n * 60); + else if (unit == 'h') threshold = std::chrono::seconds(n * 3600); + else if (unit == 'd') threshold = std::chrono::seconds(n * 86400); + else { mcpp::ui::error(std::format("bad time unit '{}': use s/m/h/d", unit)); return 2; } + auto cutoff = std::chrono::file_clock::now() - threshold; + auto entries = walk_cache_entries(); + int removed = 0; + std::uintmax_t freed = 0; + for (auto& e : entries) { + if (e.lastWrite < cutoff) { + std::error_code ec; + std::filesystem::remove_all(e.dir, ec); + if (!ec) { + ++removed; + freed += e.size; + mcpp::ui::status("Pruned", + std::format("{} ({})", e.pkgAtVer, human_bytes(e.size))); + } + } + } + std::println(""); + std::println("Pruned {} entries, freed {}", removed, human_bytes(freed)); + return 0; +} + +// `mcpp cache clean` — drop dep entries, preserve std BMIs. +export int cache_clean() { + auto bmi = mcpp::toolchain::default_cache_root(); + std::error_code ec; + std::filesystem::remove_all(bmi / "deps", ec); // deps only; preserve std.gcm + if (std::filesystem::exists(bmi)) { + for (auto& f : std::filesystem::directory_iterator(bmi, ec)) { + auto deps = f.path() / "deps"; + std::filesystem::remove_all(deps, ec); + } + } + std::println("Cleaned all dep BMI cache entries (std.gcm preserved)"); + return 0; +} + +} // namespace mcpp::bmi_cache diff --git a/src/build/execute.cppm b/src/build/execute.cppm new file mode 100644 index 00000000..fd01edc5 --- /dev/null +++ b/src/build/execute.cppm @@ -0,0 +1,588 @@ +// mcpp.build.execute — drives a prepared BuildContext: ninja execution, +// build cache + fast-path rebuilds, and the run/test/clean pipelines. +// Bodies moved verbatim from the CLI layer. Zero behavior change. + +module; +#include +#include + +export module mcpp.build.execute; + +import std; +import mcpp.build.prepare; +import mcpp.build.plan; +import mcpp.build.backend; +import mcpp.build.ninja; +import mcpp.bmi_cache; +import mcpp.manifest; +import mcpp.modgraph.scanner; +import mcpp.toolchain.stdmod; +import mcpp.xlings; +import mcpp.platform; +import mcpp.fetcher.progress; +import mcpp.project; +import mcpp.ui; + +namespace mcpp::build { + +// ─── P0: build cache for fast-path rebuilds ───────────────────────── + +constexpr std::string_view kBuildCacheFile = "target/.build_cache"; +constexpr int kBuildCacheMaxEntries = 4; // P3: LRU capacity + +// P3: one entry per (target, fingerprint) pair. +struct BuildCacheEntry { + std::string targetTriple; // "" for default target + std::string outputDir; + std::string ninjaProgram; + std::string fingerprint; // outputDir basename + std::string runtimeEnvKey; // "-" means intentionally empty; "" means old cache + std::string runtimeEnvValue; +}; + +std::vector read_build_cache(const std::filesystem::path& projectRoot) { + auto path = projectRoot / kBuildCacheFile; + std::ifstream f(path); + if (!f) return {}; + + std::string firstLine; + if (!std::getline(f, firstLine) || firstLine.empty()) return {}; + + // Detect legacy format (first line is an absolute path, not "[target=...]"). + if (firstLine[0] != '[') { + // Legacy 4-line format: outputDir, ninjaProgram, target, fingerprint. + BuildCacheEntry e; + e.outputDir = firstLine; + std::getline(f, e.ninjaProgram); + std::getline(f, e.targetTriple); + std::getline(f, e.fingerprint); + if (e.outputDir.empty() || e.ninjaProgram.empty()) return {}; + return {e}; + } + + // P3 multi-entry format: sections of [target=] + 3 mandatory + // lines, plus optional runtime-env lines added after toolenv moved out of + // build.ninja. Old cache entries omit them and are treated as stale. + std::vector entries; + std::string line = firstLine; + while (true) { + // Parse [target=] + if (line.size() < 9 || !line.starts_with("[target=") || line.back() != ']') + break; + BuildCacheEntry e; + e.targetTriple = line.substr(8, line.size() - 9); + if (!std::getline(f, e.outputDir) || e.outputDir.empty()) break; + if (!std::getline(f, e.ninjaProgram) || e.ninjaProgram.empty()) break; + std::getline(f, e.fingerprint); + bool haveNextLine = static_cast(std::getline(f, line)); + if (haveNextLine && !line.starts_with("[target=")) { + e.runtimeEnvKey = line; + std::getline(f, e.runtimeEnvValue); + haveNextLine = static_cast(std::getline(f, line)); + } + entries.push_back(std::move(e)); + if (!haveNextLine || line.empty()) break; + } + return entries; +} + +void write_build_cache(const std::filesystem::path& projectRoot, + const std::filesystem::path& outputDir, + const std::string& ninjaProgram, + const std::string& targetTriple, + const std::string& fingerprintHex = "", + const std::string& runtimeEnvKey = "-", + const std::string& runtimeEnvValue = "") { + auto path = projectRoot / kBuildCacheFile; + auto entries = read_build_cache(projectRoot); + + // Remove existing entry for this target (will be re-added at front). + std::erase_if(entries, [&](const BuildCacheEntry& e) { + return e.targetTriple == targetTriple; + }); + + // Insert at front (MRU). + BuildCacheEntry newEntry{targetTriple, outputDir.string(), ninjaProgram, fingerprintHex, + runtimeEnvKey, runtimeEnvValue}; + entries.insert(entries.begin(), std::move(newEntry)); + + // Trim to LRU capacity. + if ((int)entries.size() > kBuildCacheMaxEntries) + entries.resize(kBuildCacheMaxEntries); + + // Write P3 format. + std::error_code ec; + std::filesystem::create_directories(path.parent_path(), ec); + std::ofstream f(path, std::ios::trunc); + if (!f) return; + for (auto& e : entries) { + f << "[target=" << e.targetTriple << "]\n"; + f << e.outputDir << '\n'; + f << e.ninjaProgram << '\n'; + f << e.fingerprint << '\n'; + f << (e.runtimeEnvKey.empty() ? "-" : e.runtimeEnvKey) << '\n'; + f << e.runtimeEnvValue << '\n'; + } +} + +std::vector read_ninja_command_prefixes(const std::filesystem::path& ninjaPath) { + std::ifstream f(ninjaPath); + if (!f) return {}; + + std::vector prefixes; + std::string line; + while (std::getline(f, line)) { + auto eq = line.find('='); + if (eq == std::string::npos) continue; + auto key = line.substr(0, eq); + while (!key.empty() && std::isspace(static_cast(key.back()))) + key.pop_back(); + if (key != "cxx" && key != "cc" && key != "ar" && key != "scan_deps") + continue; + + std::string value = line.substr(eq + 1); + while (!value.empty() && std::isspace(static_cast(value.front()))) + value.erase(value.begin()); + while (!value.empty() && std::isspace(static_cast(value.back()))) + value.pop_back(); + if (!value.empty()) + prefixes.push_back(std::move(value)); + } + return prefixes; +} + +bool is_stale_ninja_failure(std::string_view output) { + return output.find("loading 'build.ninja'") != std::string_view::npos + || output.find("loading build.ninja") != std::string_view::npos + || output.find("unknown target") != std::string_view::npos + || output.find("manifest 'build.ninja' still dirty") != std::string_view::npos; +} + +// Compile a prepared BuildContext. Shared between `mcpp build` and `mcpp run` +// so the latter doesn't call prepare_build twice (and re-print the toolchain +// resolution banner). +export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, + std::string_view targetOverride = "") { + if (no_cache) { + std::error_code ec; + std::filesystem::remove_all(ctx.outputDir, ec); + } + + auto be = mcpp::build::make_ninja_backend(); + + // M5.0: print "Inferred" banner when defaults / target inference fired. + for (auto& note : ctx.manifest.inferredNotes) { + mcpp::ui::status("Inferred", note); + } + + // Announce the package being built (and any deps). + // Deps that hit the BMI cache get "Cached" instead of "Compiling". + std::set cachedNames; + for (auto& label : ctx.cachedDepLabels) { + auto sp = label.find(' '); + cachedNames.insert(sp == std::string::npos ? label : label.substr(0, sp)); + } + std::set announced; + announced.insert(ctx.manifest.package.name); + mcpp::ui::status("Compiling", + std::format("{} v{} (.)", + ctx.manifest.package.name, ctx.manifest.package.version)); + for (auto& [name, spec] : ctx.manifest.dependencies) { + if (announced.contains(name)) continue; + announced.insert(name); + std::string ver = spec.isPath() ? "(path)" : std::string("v") + spec.version; + const char* verb = cachedNames.contains(name) ? "Cached" : "Compiling"; + mcpp::ui::status(verb, std::format("{} {}", name, ver)); + } + + mcpp::build::BuildOptions opts; + opts.verbose = verbose; + auto r = be->build(ctx.plan, opts); + if (!r) { + std::fflush(stdout); + mcpp::ui::error(r.error().message); + if (!r.error().diagnosticOutput.empty()) { + std::fputs(r.error().diagnosticOutput.c_str(), stderr); + if (r.error().diagnosticOutput.back() != '\n') + std::fputc('\n', stderr); + } + return 1; + } + + // M3.2: populate BMI cache for deps that did NOT hit cache. + for (auto& task : ctx.depsToPopulate) { + auto pr = mcpp::bmi_cache::populate_from(task.key, ctx.outputDir, task.artifacts); + if (!pr) { + mcpp::ui::warning(std::format( + "bmi cache populate failed for {}@{}: {}", + task.key.packageName, task.key.version, pr.error())); + } + } + + // P1.5: warn if fingerprint changed from last build (explains full rebuild). + { + auto entries = read_build_cache(ctx.projectRoot); + for (auto& e : entries) { + if (e.targetTriple == targetOverride && !e.fingerprint.empty()) { + auto newFp = ctx.outputDir.filename().string(); + if (e.fingerprint != newFp) { + mcpp::ui::warning(std::format( + "fingerprint changed ({} → {}), full rebuild", + e.fingerprint, newFp)); + } + break; + } + } + } + + // P0: save build cache for fast-path on next invocation. + if (!no_cache && !r->ninjaProgram.empty()) { + auto fpHex = ctx.outputDir.filename().string(); + write_build_cache(ctx.projectRoot, ctx.outputDir, r->ninjaProgram, + std::string(targetOverride), fpHex, + r->runtimeEnvKey.empty() ? "-" : r->runtimeEnvKey, + r->runtimeEnvValue); + } + + mcpp::ui::finished("release", r->elapsed); + return 0; +} + +// ─── P0 fast-path: skip prepare_build when build.ninja is fresh ────── +// +// On a successful build, we write `target/.build_cache` containing the +// outputDir path. On the next invocation, if build.ninja in that dir +// is newer than all source files and mcpp.toml, we invoke ninja directly +// without re-running the scanner, make_plan, or emit phases. +// +// This reduces no-change builds from ~10s to <0.5s. + +// Try to fast-path: if build.ninja is newer than all inputs, just run ninja. +// Returns exit code on fast-path, or nullopt if full rebuild needed. +export std::optional try_fast_build(const std::filesystem::path& projectRoot, + bool verbose, bool no_cache, + std::string_view currentTarget = "") { + if (no_cache) return std::nullopt; + + // P3: read multi-entry cache and find entry matching currentTarget. + auto entries = read_build_cache(projectRoot); + const BuildCacheEntry* match = nullptr; + for (auto& e : entries) { + if (e.targetTriple == currentTarget) { match = &e; break; } + } + if (!match) return std::nullopt; + + auto outputDirStr = match->outputDir; + auto ninjaProgram = match->ninjaProgram; + auto cachedFingerprint = match->fingerprint; + auto runtimeEnvKey = match->runtimeEnvKey; + auto runtimeEnvValue = match->runtimeEnvValue; + if (runtimeEnvKey.empty()) + return std::nullopt; // old cache entry; regenerate build.ninja once + + // P1: verify fingerprint matches the outputDir basename. + if (!cachedFingerprint.empty()) { + auto dirBasename = std::filesystem::path(outputDirStr).filename().string(); + if (dirBasename != cachedFingerprint) { + return std::nullopt; + } + } + + std::error_code ec; + std::filesystem::path outputDir(outputDirStr); + + auto ninjaPath = outputDir / "build.ninja"; + if (!std::filesystem::exists(ninjaPath, ec)) return std::nullopt; + + auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); + if (ec) return std::nullopt; + + // Check mcpp.toml + auto tomlPath = projectRoot / "mcpp.toml"; + auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); + if (ec || tomlTime > ninjaTime) return std::nullopt; + + // Check all source files under src/ + auto srcDir = projectRoot / "src"; + if (std::filesystem::exists(srcDir, ec)) { + for (auto& entry : std::filesystem::recursive_directory_iterator(srcDir, ec)) { + if (!entry.is_regular_file()) continue; + auto ext = entry.path().extension().string(); + if (ext != ".cppm" && ext != ".cpp" && ext != ".cc" && + ext != ".cxx" && ext != ".c" && ext != ".h" && ext != ".hpp") + continue; + auto ft = std::filesystem::last_write_time(entry.path(), ec); + if (ec || ft > ninjaTime) return std::nullopt; + } + } + + // All inputs are older than build.ninja → fast-path: just run ninja. + std::string cmd = ninjaProgram; + if (!verbose) cmd += " --quiet"; + cmd += std::format(" -C {}", mcpp::platform::shell::quote(outputDir.string())); + if (verbose) cmd += " -v"; + cmd += " 2>&1"; + + auto t0 = std::chrono::steady_clock::now(); + std::string out; + std::optional scopedEnv; + if (runtimeEnvKey != "-" && !runtimeEnvValue.empty()) + scopedEnv.emplace(runtimeEnvKey, runtimeEnvValue); + auto r = mcpp::platform::process::capture(cmd); + out = r.output; + int status = r.exit_code; + bool ok = (status == 0); + if (!ok) { + if (is_stale_ninja_failure(out)) + return std::nullopt; + std::fflush(stdout); + mcpp::ui::error("build failed"); + auto prefixes = read_ninja_command_prefixes(ninjaPath); + auto diagnostics = verbose ? out : mcpp::build::filter_ninja_output(out, prefixes); + if (!diagnostics.empty()) { + std::fputs(diagnostics.c_str(), stderr); + if (diagnostics.back() != '\n') + std::fputc('\n', stderr); + } + return 1; + } + if (verbose && !out.empty()) + std::fputs(out.c_str(), stdout); + + auto elapsed = std::chrono::duration_cast( + std::chrono::steady_clock::now() - t0); + mcpp::ui::finished("release", elapsed); + return 0; +} + +// `mcpp run` driver: build, locate the binary target, exec it with the +// resolved runtime environment. +export int build_run_target(const std::optional& targetName, + std::span passthrough) { + // Build first. Single prepare_build → drive build → reuse ctx to locate + // the binary, so we don't re-resolve the toolchain or re-scan modgraph. + auto ctx = prepare_build(/*print_fp=*/false); + if (!ctx) { std::println(stderr, "error: {}", ctx.error()); return 2; } + if (auto rc = run_build_plan(*ctx, /*verbose=*/false, /*no_cache=*/false); rc != 0) + return rc; + + // Find binary target + const mcpp::build::LinkUnit* chosen = nullptr; + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::Binary) continue; + if (targetName && lu.targetName != *targetName) continue; + chosen = &lu; + if (targetName) break; + } + if (!chosen) { + std::println(stderr, "error: no binary target {}", + targetName ? std::format("'{}' found", *targetName) : "in this package"); + return 2; + } + + auto exe = ctx->outputDir / chosen->output; + auto pathCtx = mcpp::fetcher::make_path_ctx(/*cfg=*/nullptr, ctx->projectRoot); + mcpp::ui::status("Running", + std::format("`{}`", mcpp::ui::shorten_path(exe, pathCtx))); + std::println(""); + std::fflush(stdout); + std::string cmd = mcpp::platform::shell::quote(exe.string()); + for (auto& a : passthrough) cmd += " " + mcpp::platform::shell::quote(a); + + std::optional runtimeEnv; + auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); + auto runtimeEnvValue = mcpp::platform::env::prepend_path_list( + runtimeEnvKey, ctx->plan.runtimeLibraryDirs); + if (!runtimeEnvKey.empty() && !runtimeEnvValue.empty()) { + runtimeEnv.emplace(runtimeEnvKey, runtimeEnvValue); + } + + int rc = std::system(cmd.c_str()); + return mcpp::platform::process::extract_exit_code(rc) == 0 ? 0 : 1; +} + +// `mcpp test` driver: discover tests/**/*.cpp, synthesize targets, build +// with dev-deps, run each test binary, summarize. +export int run_tests(std::span passthrough) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) { + mcpp::ui::error("no mcpp.toml found in current directory or any parent"); + return 2; + } + + // 1. Discover test files. + auto testFiles = mcpp::modgraph::expand_glob(*root, "tests/**/*.cpp"); + if (testFiles.empty()) { + std::println("no tests found in tests/"); + return 0; + } + + // 2. Synthesize a Target for each test file. + // Name = file stem; collisions → error. + std::vector testTargets; + std::set seenNames; + for (auto& f : testFiles) { + auto name = f.stem().string(); + if (!seenNames.insert(name).second) { + mcpp::ui::error(std::format( + "duplicate test name '{}' (two .cpp files share the same stem)", name)); + return 2; + } + mcpp::manifest::Target t; + t.name = name; + t.kind = mcpp::manifest::Target::TestBinary; + // Store as path relative to project root for portability of error messages. + t.main = std::filesystem::relative(f, *root).string(); + testTargets.push_back(std::move(t)); + } + + // 3. prepare_build with dev-deps enabled + synthetic targets. + auto ctx = prepare_build(/*print_fp=*/false, + /*includeDevDeps=*/true, + std::move(testTargets)); + if (!ctx) { mcpp::ui::error(ctx.error()); return 2; } + + // 4. "Compiling test_X (test)" lines for the test binaries. + std::set cachedNames; + for (auto& label : ctx->cachedDepLabels) { + auto sp = label.find(' '); + cachedNames.insert(sp == std::string::npos ? label : label.substr(0, sp)); + } + std::set announced; + announced.insert(ctx->manifest.package.name); + mcpp::ui::status("Compiling", + std::format("{} v{} (.)", + ctx->manifest.package.name, ctx->manifest.package.version)); + for (auto& [name, spec] : ctx->manifest.dependencies) { + if (announced.contains(name)) continue; + announced.insert(name); + std::string ver = spec.isPath() ? "(path)" : std::string("v") + spec.version; + const char* verb = cachedNames.contains(name) ? "Cached" : "Compiling"; + mcpp::ui::status(verb, std::format("{} {}", name, ver)); + } + for (auto& [name, spec] : ctx->manifest.devDependencies) { + if (announced.contains(name)) continue; + announced.insert(name); + std::string ver = spec.isPath() ? "(path)" : std::string("v") + spec.version; + const char* verb = cachedNames.contains(name) ? "Cached" : "Compiling"; + mcpp::ui::status(verb, + std::format("{} {} (dev)", name, ver)); + } + // List test binaries. + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind == mcpp::build::LinkUnit::TestBinary) { + mcpp::ui::status("Compiling", + std::format("{} (test)", lu.targetName)); + } + } + + // 5. Build everything. + auto backend = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions opts; + auto buildResult = backend->build(ctx->plan, opts); + if (!buildResult) { + mcpp::ui::error(buildResult.error().message); + return 1; + } + + // M3.2: populate BMI cache for deps that did NOT hit cache. + for (auto& task : ctx->depsToPopulate) { + auto pr = mcpp::bmi_cache::populate_from(task.key, ctx->outputDir, task.artifacts); + if (!pr) { + mcpp::ui::warning(std::format( + "bmi cache populate failed for {}@{}: {}", + task.key.packageName, task.key.version, pr.error())); + } + } + + mcpp::ui::finished("test", buildResult->elapsed); + + // 6. Run each test binary in sequence; collect pass/fail. + auto t0 = std::chrono::steady_clock::now(); + int passed = 0; + int failed = 0; + std::vector failures; + + std::optional runtimeEnv; + auto runtimeEnvKey = mcpp::platform::env::runtime_library_path_key(); + auto runtimeEnvValue = mcpp::platform::env::prepend_path_list( + runtimeEnvKey, ctx->plan.runtimeLibraryDirs); + if (!runtimeEnvKey.empty() && !runtimeEnvValue.empty()) { + runtimeEnv.emplace(runtimeEnvKey, runtimeEnvValue); + } + + for (auto& lu : ctx->plan.linkUnits) { + if (lu.kind != mcpp::build::LinkUnit::TestBinary) continue; + auto exe = ctx->outputDir / lu.output; + mcpp::ui::status("Running", std::format("bin/{}", lu.targetName)); + + // Prepend the sandbox's subos/default/bin to PATH so tools + // bootstrapped during sandbox init (patchelf, ninja, etc.) are + // visible to test binaries that shell out to them. The + // toolchain binary's path encodes the registry root — derive it. + std::string pathPrefix; + if constexpr (!mcpp::platform::is_windows) { + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(ctx->tc.binaryPath)) { + // xpkgs is /data/xpkgs → registry = xpkgs/../.. + auto registryDir = xpkgs->parent_path().parent_path(); + auto sandboxBin = registryDir / "subos" / "default" / "bin"; + if (std::filesystem::exists(sandboxBin)) + pathPrefix = std::format("PATH={}:\"$PATH\" ", + mcpp::platform::shell::quote(sandboxBin.string())); + } + } + + std::string cmd = pathPrefix + mcpp::platform::shell::quote(exe.string()); + for (auto& a : passthrough) cmd += " " + mcpp::platform::shell::quote(a); + int exitCode = mcpp::platform::process::extract_exit_code(std::system(cmd.c_str())); + + if (exitCode == 0) { + std::println("{} ... ok", lu.targetName); + ++passed; + } else { + std::println("{} ... FAIL (exit {})", lu.targetName, exitCode); + ++failed; + failures.push_back(lu.targetName); + } + } + auto elapsed = std::chrono::duration_cast( + std::chrono::steady_clock::now() - t0); + + // 7. Summary. + std::println(""); + if (failed == 0) { + mcpp::ui::status("test result", + std::format("ok. {} passed; 0 failed; finished in {:.2f}s", + passed, static_cast(elapsed.count()) / 1000.0)); + return 0; + } + mcpp::ui::error(std::format( + "test result: FAILED. {} passed; {} failed; finished in {:.2f}s", + passed, failed, static_cast(elapsed.count()) / 1000.0)); + std::println(""); + std::println("failures:"); + for (auto& n : failures) std::println(" {}", n); + return 1; +} + +// `mcpp clean` driver. +export int clean_project(bool wipe_bmi) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) { std::println(stderr, "error: not in an mcpp package"); return 2; } + std::error_code ec; + std::filesystem::remove_all(*root / "target", ec); + if (ec) { + std::println(stderr, "error: cannot remove target/: {}", ec.message()); + return 1; + } + std::println("Cleaned: {}", (*root / "target").string()); + + if (wipe_bmi) { + auto bmi = mcpp::toolchain::default_cache_root(); + std::filesystem::remove_all(bmi, ec); + std::println("Cleaned BMI cache: {}", bmi.string()); + } + return 0; +} + +} // namespace mcpp::build diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm new file mode 100644 index 00000000..b595a758 --- /dev/null +++ b/src/build/prepare.cppm @@ -0,0 +1,2500 @@ +// mcpp.build.prepare — BuildContext + prepare_build: the build-orchestration +// core (workspace -> toolchain -> dependency resolution -> features -> +// modgraph -> fingerprint -> plan -> lockfile). +// Bodies moved verbatim from the CLI layer. Zero behavior change. + +module; +#include +#include + +export module mcpp.build.prepare; + +import std; +import mcpp.libs.json; +import mcpp.manifest; +import mcpp.modgraph.graph; +import mcpp.modgraph.scanner; +import mcpp.modgraph.validate; +import mcpp.toolchain.clang; +import mcpp.toolchain.detect; +import mcpp.toolchain.fingerprint; +import mcpp.toolchain.registry; +import mcpp.toolchain.stdmod; +import mcpp.toolchain.post_install; +import mcpp.build.plan; +import mcpp.lockfile; +import mcpp.config; +import mcpp.xlings; +import mcpp.platform; +import mcpp.fetcher; +import mcpp.fetcher.progress; +import mcpp.pm.resolver; +import mcpp.pm.index_spec; +import mcpp.pm.mangle; +import mcpp.pm.compat; +import mcpp.pm.dep_spec; +import mcpp.version_req; +import mcpp.ui; +import mcpp.log; +import mcpp.fallback.install_integrity; +import mcpp.bmi_cache; +import mcpp.project; + +namespace mcpp::build { + +export std::filesystem::path target_dir(const mcpp::toolchain::Toolchain& tc, + const mcpp::toolchain::Fingerprint& fp, + const std::filesystem::path& root) +{ + auto triple = tc.targetTriple.empty() ? std::string{"unknown"} : tc.targetTriple; + return root / "target" / triple / fp.hex; +} + + +// Compose a stable canonical compile-flags string for fingerprinting. +std::string canonical_compile_flags(const mcpp::manifest::Manifest& m) { + std::string s; + s += "-std="; s += m.package.standard; + s += " -fmodules"; + // macOS deployment target changes the effective compile triple + // (arm64-apple-macosxNN) — a std.pcm built for one target cannot be + // loaded by a TU compiled for another. Fold the resolved value + // (env override > [build] macos_deployment_target manifest default) + // into the fingerprint so switching targets rebuilds the BMI cache + // instead of dying with a module config mismatch. + // + // The built-in default floor (rustc-style) lives in the single + // resolver (platform::macos::deployment_target), so this rule, the + // flags and the std-module prebuild always agree — the 0.0.50-era + // attempt to inject a default here alone left the test build's + // std.pcm unstaged (import std failed wholesale on macos CI). + if constexpr (mcpp::platform::is_macos) { + auto dtv = mcpp::platform::macos::deployment_target( + m.buildConfig.macosDeploymentTarget); + if (!dtv.empty()) { + s += " macos_deployment_target="; + s += dtv; + } + } + if (!m.buildConfig.cStandard.empty()) { + s += " c_standard="; + s += m.buildConfig.cStandard; + } + for (auto const& flag : m.buildConfig.cflags) { + s += " cflag:"; + s += flag; + } + for (auto const& flag : m.buildConfig.cxxflags) { + s += " cxxflag:"; + s += flag; + } + for (auto const& flag : m.buildConfig.ldflags) { + s += " ldflag:"; + s += flag; + } + return s; +} + +std::string canonical_package_build_metadata( + const std::vector& packages) +{ + std::string s; + for (auto const& pkg : packages) { + s += "\npackage:"; + s += pkg.manifest.package.namespace_; + s += "/"; + s += pkg.manifest.package.name; + s += "@"; + s += pkg.manifest.package.version; + if (!pkg.manifest.buildConfig.cStandard.empty()) { + s += " c_standard="; + s += pkg.manifest.buildConfig.cStandard; + } + for (auto const& flag : pkg.manifest.buildConfig.cflags) { + s += " cflag:"; + s += flag; + } + for (auto const& flag : pkg.manifest.buildConfig.cxxflags) { + s += " cxxflag:"; + s += flag; + } + for (auto const& flag : pkg.manifest.buildConfig.ldflags) { + s += " ldflag:"; + s += flag; + } + if (pkg.usageResolved) { + for (auto const& dir : pkg.privateBuild.includeDirs) { + s += " private_include:"; + s += dir.generic_string(); + } + for (auto const& dir : pkg.publicUsage.includeDirs) { + s += " public_include:"; + s += dir.generic_string(); + } + } + for (auto const& [path, content] : pkg.manifest.buildConfig.generatedFiles) { + s += " genfile:"; + s += path.generic_string(); + s += "="; + s += content; + } + } + return s; +} + +std::expected +materialize_generated_files(const std::filesystem::path& root, + const mcpp::manifest::Manifest& manifest) +{ + for (auto const& [relPath, content] : manifest.buildConfig.generatedFiles) { + if (relPath.empty()) { + return std::unexpected("generated_files contains an empty path"); + } + if (relPath.is_absolute()) { + return std::unexpected(std::format( + "generated_files path '{}' must be relative", relPath.generic_string())); + } + auto const genericPath = relPath.generic_string(); + for (std::size_t begin = 0; begin <= genericPath.size();) { + auto const end = genericPath.find('/', begin); + auto const part = genericPath.substr(begin, end == std::string::npos + ? std::string::npos + : end - begin); + if (part == "..") { + return std::unexpected(std::format( + "generated_files path '{}' must not escape the package root", + relPath.generic_string())); + } + if (end == std::string::npos) { + break; + } + begin = end + 1; + } + + auto out = root / relPath.lexically_normal(); + std::error_code ec; + std::filesystem::create_directories(out.parent_path(), ec); + if (ec) { + return std::unexpected(std::format( + "cannot create directory for generated file '{}': {}", + out.string(), ec.message())); + } + + std::ofstream os(out, std::ios::binary); + if (!os) { + return std::unexpected(std::format( + "cannot write generated file '{}'", out.string())); + } + os << content; + if (!os) { + return std::unexpected(std::format( + "failed while writing generated file '{}'", out.string())); + } + } + return {}; +} + +bool is_std_module(std::string_view name) { + return name == "std" || name == "std.compat"; +} + +std::string trim_copy(std::string s) { + while (!s.empty() && std::isspace(static_cast(s.front()))) + s.erase(0, 1); + while (!s.empty() && std::isspace(static_cast(s.back()))) + s.pop_back(); + return s; +} + +bool source_file_imports_std(const std::filesystem::path& path) { + std::ifstream is(path); + if (!is) return false; + + std::string line; + while (std::getline(is, line)) { + line = trim_copy(std::move(line)); + std::size_t i = std::string::npos; + if (line.starts_with("import ")) { + i = 7; + } else if (line.starts_with("export import ")) { + i = 14; + } + if (i == std::string::npos) continue; + while (i < line.size() && std::isspace(static_cast(line[i]))) + ++i; + + std::string name; + while (i < line.size() + && (std::isalnum(static_cast(line[i])) + || line[i] == '_' || line[i] == '.' || line[i] == ':')) { + name.push_back(line[i]); + ++i; + } + if (is_std_module(name)) return true; + } + return false; +} + +bool graph_or_targets_import_std(const mcpp::modgraph::Graph& graph, + const mcpp::manifest::Manifest& manifest, + const std::filesystem::path& projectRoot) { + for (auto& u : graph.units) { + for (auto& req : u.requires_) { + if (is_std_module(req.logicalName)) + return true; + } + } + + // Some target entry files can be added to the plan after the package scan. + // Check them here so std BMI setup matches what make_plan will compile. + for (auto& t : manifest.targets) { + if (!t.main.empty() && source_file_imports_std(projectRoot / t.main)) + return true; + } + return false; +} + +export struct BuildContext { + mcpp::manifest::Manifest manifest; + mcpp::toolchain::Toolchain tc; + mcpp::toolchain::Fingerprint fp; + std::filesystem::path projectRoot; + std::filesystem::path outputDir; + std::filesystem::path stdBmi; + std::filesystem::path stdObject; + mcpp::build::BuildPlan plan; + + // M3.2 BMI cache: deps that did NOT hit cache and therefore need + // populate_from(...) AFTER backend.build succeeds. + struct CacheTask { + mcpp::bmi_cache::CacheKey key; + mcpp::bmi_cache::DepArtifacts artifacts; + }; + std::vector depsToPopulate; + + // Names of deps that DID hit cache (for ui status output). + std::vector cachedDepLabels; // "mcpplibs.cmdline v0.0.1" +}; + +// Command-level overrides (--target / --static). +// Empty defaults preserve pre-existing behaviour exactly. +export struct BuildOverrides { + std::string target_triple; // empty = host triple, fall through to [toolchain] + bool force_static = false; // --static (or implied by musl target) + std::string package_filter; // -p : only build this workspace member + std::string profile; // --profile (default "release") + std::string features; // --features a,b,c (root package activation) + bool strict = false; // --strict: schema warnings become errors +}; + +// `prepare_build` builds the BuildContext for any verb that compiles. +// includeDevDeps: when true, dev-dependencies are also fetched + scanned +// into the modgraph. mcpp test passes true; build/run pass false. +// extraTargets: additional Target entries (e.g. synthetic test targets) +// appended to the manifest before the modgraph runs. +// overrides: --target / --static. +export std::expected +prepare_build(bool print_fingerprint, + bool includeDevDeps = false, + std::vector extraTargets = {}, + BuildOverrides overrides = {}) { + auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); + if (!root) { + return std::unexpected("no mcpp.toml found in current directory or any parent"); + } + + auto m = mcpp::manifest::load(*root / "mcpp.toml"); + if (!m) return std::unexpected(m.error().format()); + + // ─── Workspace handling ──────────────────────────────────────────── + // If the manifest has [workspace] and is a virtual workspace (no [package]), + // or if -p filter is set, switch to the target member's manifest. + std::optional wsManifest; // keep workspace manifest alive + if (m->workspace.present) { + std::string targetMember; + + if (!overrides.package_filter.empty()) { + // -p : find matching member by directory basename or path + for (auto& mp : m->workspace.members) { + auto basename = std::filesystem::path(mp).filename().string(); + if (basename == overrides.package_filter || mp == overrides.package_filter) { + targetMember = mp; + break; + } + } + if (targetMember.empty()) { + return std::unexpected(std::format( + "workspace member '{}' not found in [workspace].members", + overrides.package_filter)); + } + } else if (m->package.name.empty()) { + // Virtual workspace: find a member with a binary target, or use last member. + for (auto& mp : m->workspace.members) { + auto memberDir = *root / mp; + auto mm = mcpp::manifest::load(memberDir / "mcpp.toml"); + if (!mm) continue; + for (auto& t : mm->targets) { + if (t.kind == mcpp::manifest::Target::Binary) { + targetMember = mp; + break; + } + } + if (!targetMember.empty()) break; + } + if (targetMember.empty() && !m->workspace.members.empty()) { + targetMember = m->workspace.members.back(); + } + } + // else: rooted workspace with [package] — build root normally. + + if (!targetMember.empty()) { + auto memberDir = *root / targetMember; + if (!std::filesystem::exists(memberDir / "mcpp.toml")) { + return std::unexpected(std::format( + "workspace member '{}' has no mcpp.toml", targetMember)); + } + wsManifest = std::move(*m); // preserve workspace manifest + m = mcpp::manifest::load(memberDir / "mcpp.toml"); + if (!m) return std::unexpected(std::format( + "workspace member '{}': {}", targetMember, m.error().format())); + + // Merge workspace dependency versions + mcpp::project::merge_workspace_deps(*m, *wsManifest); + + // Inherit workspace toolchain if member doesn't define one + if (m->toolchain.byPlatform.empty()) { + m->toolchain = wsManifest->toolchain; + } + // Inherit workspace target overrides + for (auto& [triple, entry] : wsManifest->targetOverrides) { + if (!m->targetOverrides.contains(triple)) { + m->targetOverrides[triple] = entry; + } + } + // Inherit workspace indices if member doesn't define any + if (m->indices.empty() && !wsManifest->indices.empty()) { + m->indices = wsManifest->indices; + } + + mcpp::ui::status("Workspace", std::format("building member '{}'", targetMember)); + root = memberDir; + } + } else { + // Not at workspace root — check if we're inside a workspace + auto wsRoot = mcpp::project::find_workspace_root(*root); + if (!wsRoot.empty()) { + auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); + if (wsm && wsm->workspace.present) { + mcpp::project::merge_workspace_deps(*m, *wsm); + if (m->toolchain.byPlatform.empty()) { + m->toolchain = wsm->toolchain; + } + for (auto& [triple, entry] : wsm->targetOverrides) { + if (!m->targetOverrides.contains(triple)) { + m->targetOverrides[triple] = entry; + } + } + // Inherit workspace indices if member doesn't define any + if (m->indices.empty() && !wsm->indices.empty()) { + m->indices = wsm->indices; + } + } + } + } + + // Inject synthetic targets (e.g. test binaries from `mcpp test`). + for (auto& t : extraTargets) m->targets.push_back(t); + + // ─── Toolchain resolution (docs/21) ──────────────────────────────── + // Priority chain: + // 1. mcpp.toml [toolchain]. → resolve_xpkg_path → abs path + // 2. $CXX env var + // 3. PATH g++ (with warning) + std::filesystem::path explicit_compiler; + std::optional cfg_opt; + bool bootstrap_checked = false; + auto get_cfg = [&](bool requireBootstrap = true) -> std::expected { + if (!cfg_opt) { + auto c = mcpp::config::load_or_init(/*quiet=*/false, + mcpp::fetcher::make_bootstrap_progress_callback()); + if (!c) return std::unexpected(c.error().message); + cfg_opt = std::move(*c); + } + // Commands that need bootstrap tools (build, run, toolchain install) + // pass requireBootstrap=true to get an early, clear error. + if (requireBootstrap && !bootstrap_checked) { + bootstrap_checked = true; + auto problem = mcpp::config::check_base_init(*cfg_opt); + if (!problem.empty()) { + return std::unexpected(std::format( + "{}\n hint: run `mcpp self init --force` to reset and re-initialize", + problem)); + } + } + return &*cfg_opt; + }; + + constexpr std::string_view kCurrentPlatform = mcpp::platform::name; + + // M5.5: toolchain resolution priority: + // 0. --target X / --static, looked up in [target.] + // 1. project mcpp.toml [toolchain]. or .default + // 2. global ~/.mcpp/config.toml [toolchain].default + // 3. hard error (no system fallback) + // Resolve the build profile: --profile (default "release") → built-in + // defaults, overlaid by any [profile.] from the manifest → buildConfig. + { + std::string pname = overrides.profile.empty() ? "release" : overrides.profile; + mcpp::manifest::Profile pr; + if (pname == "dev" || pname == "debug") { pr.optLevel = "0"; pr.debug = true; } + else if (pname == "dist") { pr.optLevel = "3"; pr.strip = true; } + // (built-in dist intentionally leaves lto off: several packaged gcc + // payloads ship without the LTO plugin; enable via [profile.dist].) + else { pr.optLevel = "2"; } // release + if (auto it = m->profiles.find(pname); it != m->profiles.end()) pr = it->second; + m->buildConfig.optLevel = pr.optLevel; + m->buildConfig.debug = pr.debug; + m->buildConfig.lto = pr.lto; + m->buildConfig.strip = pr.strip; + m->buildConfig.cflags.insert(m->buildConfig.cflags.end(), + pr.cflags.begin(), pr.cflags.end()); + m->buildConfig.cxxflags.insert(m->buildConfig.cxxflags.end(), + pr.cxxflags.begin(), pr.cxxflags.end()); + m->buildConfig.ldflags.insert(m->buildConfig.ldflags.end(), + pr.ldflags.begin(), pr.ldflags.end()); + } + + // [package] platforms — fixed vocabulary owned by mcpp (it owns the + // target/triple system). Unknown values: warning, or error under --strict. + for (auto& pf : m->package.platforms) { + if (pf != "linux" && pf != "macos" && pf != "windows") { + auto msg = std::format( + "[package] platforms contains unknown platform '{}' " + "(expected: linux | macos | windows)", pf); + if (overrides.strict) return std::unexpected(msg); + std::println(stderr, "warning: {}", msg); + } + } + + auto tcSpec = m->toolchain.for_platform(kCurrentPlatform); + if (!tcSpec.has_value()) { + auto cfg = get_cfg(); + if (cfg && !(*cfg)->defaultToolchain.empty()) { + tcSpec = (*cfg)->defaultToolchain; + } + } + + // ─── --target / --static overrides ────────────────────────────────── + // Look up [target.] from manifest; fall back to convention + // (anything ending with "-musl" → gcc@-musl + static). + auto endswith = [](std::string_view s, std::string_view suf) { + return s.size() >= suf.size() + && s.compare(s.size() - suf.size(), suf.size(), suf) == 0; + }; + if (!overrides.target_triple.empty()) { + auto it = m->targetOverrides.find(overrides.target_triple); + if (it != m->targetOverrides.end()) { + if (!it->second.toolchain.empty()) tcSpec = it->second.toolchain; + if (!it->second.linkage.empty()) m->buildConfig.linkage = it->second.linkage; + } + // Convention: "*-musl" target without an explicit `[target.X]` + // override gets the canonical musl-gcc spec the rest of mcpp + // uses internally. We can't just append "-musl" to the inherited + // toolchain version because xim doesn't have a `musl-gcc@` for every gcc release — gcc 16.1 has no musl + // variant yet, only 9.4 / 11.5 / 13.3 / 15.1 do. Picking 15.1.0 + // as the static default matches what mcpp itself uses for + // `mcpp build --target x86_64-linux-musl` (see mcpp.toml). + if (endswith(overrides.target_triple, "-musl") + && (it == m->targetOverrides.end() || it->second.toolchain.empty())) + { + tcSpec = "gcc@15.1.0-musl"; + } + if (endswith(overrides.target_triple, "-musl") + && m->buildConfig.linkage.empty()) { + m->buildConfig.linkage = "static"; + } + } + if (overrides.force_static) m->buildConfig.linkage = "static"; + + if (tcSpec.has_value() && *tcSpec != "system") { + auto spec = mcpp::toolchain::parse_toolchain_spec(*tcSpec); + if (!spec || spec->version.empty()) { + return std::unexpected(std::format( + "[toolchain].{} = '{}' is invalid; expected '@'", + kCurrentPlatform, *tcSpec)); + } + auto pkg = mcpp::toolchain::to_xim_package(*spec); + + auto cfg = get_cfg(); + if (!cfg) return std::unexpected(cfg.error()); + mcpp::fetcher::Fetcher fetcher(**cfg); + + mcpp::ui::info("Resolving", "toolchain"); + mcpp::fetcher::InstallProgressHandler progress; + auto payload = fetcher.resolve_xpkg_path(pkg.target(), /*autoInstall=*/true, &progress); + if (!payload) { + return std::unexpected(std::format( + "toolchain '{}': {}", *tcSpec, payload.error().message)); + } + + explicit_compiler = mcpp::toolchain::toolchain_frontend(payload->binDir, pkg); + if (!std::filesystem::exists(explicit_compiler)) { + return std::unexpected(std::format( + "toolchain payload '{}' has no known C++ frontend in {}", + pkg.target(), payload->binDir.string())); + } + mcpp::ui::info("Resolved", + std::format("{} → {}", *tcSpec, + mcpp::ui::shorten_path(explicit_compiler, + mcpp::fetcher::make_path_ctx(&**get_cfg(), *root)))); + } else if (tcSpec.has_value() && *tcSpec == "system") { + // Explicit user opt-in to system PATH compiler — kept as escape hatch. + } else if (auto* opt = std::getenv("MCPP_NO_AUTO_INSTALL"); opt && *opt && *opt != '0') { + // CI / offline / test opt-out: hard-error instead of silently + // pulling ~800 MB of toolchain. Preserves the original M5.5 + // contract for environments that need it. + if constexpr (mcpp::platform::is_macos || mcpp::platform::is_windows) { + return std::unexpected( + "no toolchain configured.\n" + " run one of:\n" + " mcpp toolchain install llvm 20.1.7\n" + " mcpp toolchain default llvm@20.1.7\n" + " or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install."); + } else { + return std::unexpected( + "no toolchain configured.\n" + " run one of:\n" + " mcpp toolchain install gcc 15.1.0-musl\n" + " mcpp toolchain default gcc@15.1.0-musl\n" + " or unset MCPP_NO_AUTO_INSTALL to let mcpp auto-install."); + } + } else { + // First-run UX: no project-level [toolchain], no global default, + // and the user just ran `mcpp build` (or similar). Auto-install + // the platform's canonical default so the user gets a working + // binary out of the box without any config. We pin it as the + // global default so the next invocation is silent. + // Users can switch any time via `mcpp toolchain default `. + // + // macOS: LLVM/Clang — Apple doesn't ship GCC; upstream LLVM with + // bundled libc++ is the self-contained choice. + // Linux: glibc gcc — the platform-native ABI. A musl-static default + // cannot link the glibc world (X11/GL/system libs), so it + // breaks GUI/native packages out of the box. musl-static stays + // opt-in via `mcpp build --target x86_64-linux-musl` for users + // who explicitly want portable static binaries. + std::string defaultSpec = (mcpp::platform::is_macos || mcpp::platform::is_windows) + ? "llvm@20.1.7" : "gcc@16.1.0"; + auto defaultParsed = mcpp::toolchain::parse_toolchain_spec(defaultSpec); + auto defaultPkg = mcpp::toolchain::to_xim_package(*defaultParsed); + + if constexpr (mcpp::platform::is_macos || mcpp::platform::is_windows) { + mcpp::ui::info("First run", + std::format("no toolchain configured — installing {} (LLVM/Clang) as default", + defaultSpec)); + } else { + mcpp::ui::info("First run", + std::format("no toolchain configured — installing {} (glibc, native ABI) as default", + defaultSpec)); + } + + auto cfg = get_cfg(); + if (!cfg) return std::unexpected(cfg.error()); + mcpp::fetcher::Fetcher fetcher(**cfg); + + mcpp::fetcher::InstallProgressHandler progress; + // The glibc default toolchain needs the sysroot payloads (C library + + // kernel headers), exactly like `mcpp toolchain install` provides. + // The old musl-static default was self-contained, which masked this. + if constexpr (!mcpp::platform::is_macos && !mcpp::platform::is_windows) { + for (auto dep : {"xim:glibc", "xim:linux-headers"}) { + (void)fetcher.resolve_xpkg_path(dep, /*autoInstall=*/true, &progress); + } + } + auto payload = fetcher.resolve_xpkg_path(defaultPkg.target(), + /*autoInstall=*/true, &progress); + if (!payload) { + return std::unexpected(std::format( + "auto-installing default toolchain {} failed: {}\n" + " you can install it manually with:\n" + " mcpp toolchain install {} {}", + defaultSpec, payload.error().message, + defaultParsed->compiler, defaultParsed->version)); + } + explicit_compiler = mcpp::toolchain::toolchain_frontend(payload->binDir, defaultPkg); + if (!std::filesystem::exists(explicit_compiler)) { + return std::unexpected(std::format( + "default toolchain payload {} has no known C++ frontend in {}", + defaultPkg.target(), payload->binDir.string())); + } + + // The freshly-installed glibc gcc needs the SAME post-install fixup + // (patchelf + specs wiring against the sandbox glibc) that + // `mcpp toolchain install` performs — without it a fresh sandbox + // cannot find the C library (stdlib.h: No such file or directory). + if (defaultPkg.needsGccPostInstallFixup) { + mcpp::toolchain::gcc_post_install_fixup(**cfg, payload->root); + } + + // Persist the default so we don't ask again next time. + if (auto wr = mcpp::config::write_default_toolchain(**cfg, defaultSpec); wr) { + (*cfg)->defaultToolchain = defaultSpec; + mcpp::ui::status("Default", std::format("set to {}", defaultSpec)); + } // best-effort: a failed config write only loses the persistence, + // not the running build. + tcSpec = defaultSpec; + } + + auto tc = mcpp::toolchain::detect(explicit_compiler); + if (!tc) return std::unexpected(tc.error().message); + + // For musl-gcc the toolchain is fully self-contained + // (`/x86_64-linux-musl/{include,lib}` is its own sysroot). + // musl-gcc's `-dumpmachine` reports `x86_64-linux-musl`. + bool isMuslTc = tc->targetTriple.find("-musl") != std::string::npos; + + // A musl toolchain only really makes sense with static linkage — + // dynamic-musl binaries depend on a system /lib/ld-musl-x86_64.so.1 + // that most distros don't ship. Default linkage to "static" when + // the resolved toolchain is musl, unless the user has already opted + // out via [build].linkage / [target.].linkage. + if (isMuslTc && m->buildConfig.linkage.empty()) { + m->buildConfig.linkage = "static"; + } + + // Sysroot comes from the toolchain payload itself (GCC -print-sysroot, + // Clang clang++.cfg). mcpp does not override it — the payload is + // self-describing. See docs: 2026-05-21-linux-sysroot-missing-kernel-headers.md + + // Resolve dependencies: walk the **transitive** graph from the main + // manifest, BFS-style. Each unique `(namespace, shortName)` is fetched + // once, its `[build].include_dirs` are propagated to the main + // manifest, and its own `[dependencies]` are queued for processing + // (its `[dev-dependencies]` are NOT — those are private to the dep's + // own test runs). + // + // Conflict policy: C++ modules require globally-unique module names + // and ODR-respecting symbols, so the same `(ns, name)` resolved to + // two different exact versions is an error — mcpp prints both + // requesting parents and asks the user to align them. + + // Auto-refresh the builtin package index only when a version dependency + // is actually routed there. Local/remote project indices are handled by + // the project-scoped setup below; refreshing the global index for those + // packages is both unnecessary and can make offline/local-index builds + // block on unrelated remote repositories. + if (!m->dependencies.empty()) { + auto usesBuiltinIndex = [&](const mcpp::manifest::DependencySpec& spec) { + if (spec.isPath() || spec.isGit()) return false; + + auto ns = spec.namespace_.empty() + ? std::string(mcpp::pm::kDefaultNamespace) + : spec.namespace_; + if (ns == mcpp::pm::kDefaultNamespace) return true; + + auto it = m->indices.find(ns); + if (it == m->indices.end()) return true; + return it->second.is_builtin(); + }; + + bool needsBuiltinIndexRefresh = false; + for (auto& [_, spec] : m->dependencies) { + if (usesBuiltinIndex(spec)) { + needsBuiltinIndexRefresh = true; + break; + } + } + if (needsBuiltinIndexRefresh) { + auto cfg2 = get_cfg(); + if (cfg2) { + auto xlEnv = mcpp::config::make_xlings_env(**cfg2); + if (!mcpp::xlings::is_index_fresh(xlEnv, (*cfg2)->searchTtlSeconds)) { + mcpp::ui::status("Updating", "package index (auto-refresh)"); + mcpp::xlings::ensure_index_fresh( + xlEnv, (*cfg2)->searchTtlSeconds, /*quiet=*/true); + } + } + } + } + + // Set up project-level .mcpp/ directory for custom indices. + // This creates .mcpp/.xlings.json with custom non-builtin index + // entries so xlings can clone them into the project-scoped data dir. + if (!m->indices.empty()) { + auto cfg2 = get_cfg(); + if (cfg2) { + mcpp::config::ensure_project_index_dir(**cfg2, *root, m->indices); + + // On first build, the project index data root may be empty because + // ensure_project_index_dir only writes .xlings.json but does not + // trigger clone/link creation. Local path indices are read directly; + // remote custom indices are synced quietly before dependency resolution. + bool hasCustomIndices = false; + for (auto& [idxName, spec] : m->indices) { + if (!spec.is_builtin()) { + hasCustomIndices = true; + break; + } + } + if (hasCustomIndices) { + bool needsClone = !mcpp::config::project_index_data_initialized(*root); + if (needsClone) { + bool needsRemoteUpdate = false; + for (auto& [idxName, spec] : m->indices) { + if (spec.is_builtin() || spec.is_local()) continue; + needsRemoteUpdate = true; + break; + } + if (needsRemoteUpdate) { + mcpp::ui::status("Fetching", "custom index repos (first use)"); + auto projEnv = mcpp::config::make_project_xlings_env(**cfg2, *root); + int rc = mcpp::xlings::update_index(projEnv, /*quiet=*/true); + if (rc != 0) { + return std::unexpected( + "project custom index update failed; run `mcpp index update` for details"); + } + } + } + } + } + } + + std::vector packages; + packages.push_back({*root, *m}); + + // dep_manifests is kept around purely so the build plan can move it + // out at the end (PackageRoot stores a `Manifest` by value, so the + // unique_ptr is not load-bearing for liveness — it's a leftover from + // an earlier design and harmless). + std::vector> dep_manifests; + auto cache_index_name = [](std::string_view ns) { + if (ns.empty()) return std::string(mcpp::pm::kDefaultNamespace); + return std::string(ns); + }; + struct DepCacheIdentity { + std::string indexName; + std::string packageName; + std::string version; + }; + std::vector dep_cache_identities; + struct GitLockIdentity { + std::string source; + std::string hash; + }; + std::map root_git_lock_identities; + + struct ResolvedKey { + std::string ns; + std::string shortName; + auto operator<=>(const ResolvedKey&) const = default; + }; + struct ResolvedRecord { + std::string version; // empty for path/git deps + std::string constraint; // AND-combined original constraints (version src only) + std::string requestedBy; // human-readable for error messages + std::string source; // "version" | "path" | "git" — for type-clash check + std::size_t depIndex = 0; // index into dep_manifests/packages-1 (for in-place re-fetch) + std::vector linkFlagsAdded; // entries appended to m->buildConfig.ldflags by this dep + }; + std::map resolved; + + // Sentinel for "the consumer is the main package" (no dep_manifests entry). + constexpr std::size_t kMainConsumer = static_cast(-1); + + struct WorkItem { + std::string name; // dep map key as written + mcpp::manifest::DependencySpec spec; // copy (we may mutate version) + std::string requestedBy; // who asked for it + std::string originalConstraint; // spec.version BEFORE pinning (for SemVer merge) + std::size_t consumerDepIndex; // dep_manifests slot of who pushed this child; kMainConsumer for main + std::filesystem::path resolveRoot; // base dir for relative path deps (empty = use project root) + }; + std::deque worklist; + + // SemVer constraint resolver, shared across the worklist so transitive + // deps with caret/range constraints (`^1.0`) also get pinned to a + // concrete version before fetch. + auto resolveSemver = [&](mcpp::manifest::DependencySpec& s, + const std::string& depName) + -> std::expected + { + if (s.isPath() || s.isGit()) return {}; + if (!mcpp::pm::is_version_constraint(s.version)) return {}; + auto cfg = get_cfg(); + if (!cfg) return std::unexpected(cfg.error()); + mcpp::fetcher::Fetcher fetcher(**cfg); + // 0.0.10+: use structured namespace from DependencySpec. + auto resolved = mcpp::pm::resolve_semver( + s.namespace_, s.shortName.empty() ? depName : s.shortName, + s.version, fetcher); + if (!resolved) return std::unexpected(resolved.error()); + mcpp::ui::info("Resolved", + std::format("{} {} → v{}", depName, s.version, *resolved)); + s.version = std::move(*resolved); + return {}; + }; + + // Acquire a version-source dep at a specific pinned version. Used both + // by the first-time walk and by the SemVer merger when a re-fetch at a + // different version is needed. Returns the dep's effective root (where + // mcpp.toml lives) and a fully loaded manifest. + using LoadedDep = std::pair; + // Helper: find the IndexSpec for a namespace from the manifest's [indices]. + // Returns nullptr if the namespace maps to the default/builtin index. + auto findIndexForNs = [&](const std::string& ns) + -> const mcpp::pm::IndexSpec* + { + if (ns.empty() || ns == std::string(mcpp::pm::kDefaultNamespace)) return nullptr; + if (auto it = m->indices.find(ns); it != m->indices.end()) { + return &it->second; + } + auto root = ns.substr(0, ns.find('.')); + for (auto& [idxName, spec] : m->indices) { + if (idxName == ns) return &spec; + if (idxName == root) return &spec; + } + return nullptr; + }; + + auto canonicalXpkgLuaFilename = + [](std::string_view ns, std::string_view shortName) { + if (ns.empty() || ns == mcpp::pm::kDefaultNamespace) { + return std::string(shortName) + ".lua"; + } + return std::format("{}.{}.lua", ns, shortName); + }; + + auto readStrictLuaFromPkgsDir = + [&](const std::filesystem::path& pkgsDir, + std::string_view ns, + std::string_view shortName) -> std::optional + { + auto fname = canonicalXpkgLuaFilename(ns, shortName); + if (fname.empty()) return std::nullopt; + char first = static_cast(std::tolower( + static_cast(fname.front()))); + auto candidate = pkgsDir / std::string(1, first) / fname; + if (!std::filesystem::exists(candidate)) return std::nullopt; + + std::ifstream is(candidate); + std::stringstream ss; + ss << is.rdbuf(); + return ss.str(); + }; + + auto readStrictLuaForCandidate = + [&](const mcpp::pm::DependencyCoordinate& coord) + -> std::optional + { + auto cfg = get_cfg(); + if (!cfg) return std::nullopt; + + auto* idxSpec = findIndexForNs(coord.namespace_); + if (idxSpec && idxSpec->is_local()) { + auto indexPath = mcpp::config::resolve_project_index_path(*root, *idxSpec); + return readStrictLuaFromPkgsDir(indexPath / "pkgs", + coord.namespace_, + coord.shortName); + } + if (idxSpec && !idxSpec->is_builtin()) { + std::error_code ec; + for (auto& data : mcpp::config::project_xlings_data_roots(*root)) { + if (!std::filesystem::exists(data)) continue; + for (auto& entry : std::filesystem::directory_iterator(data, ec)) { + if (!entry.is_directory()) continue; + auto pkgsDir = entry.path() / "pkgs"; + if (auto lua = readStrictLuaFromPkgsDir( + pkgsDir, coord.namespace_, coord.shortName)) { + return lua; + } + } + } + return std::nullopt; + } + + auto data = (*cfg)->xlingsHome() / "data"; + if (!std::filesystem::exists(data)) return std::nullopt; + std::error_code ec; + for (auto& entry : std::filesystem::directory_iterator(data, ec)) { + if (!entry.is_directory()) continue; + auto pkgsDir = entry.path() / "pkgs"; + if (auto lua = readStrictLuaFromPkgsDir( + pkgsDir, coord.namespace_, coord.shortName)) { + return lua; + } + } + return std::nullopt; + }; + + auto candidateQualifiedName = + [](std::string_view ns, std::string_view shortName) { + if (ns.empty()) return std::string(shortName); + return std::format("{}.{}", ns, shortName); + }; + + auto xpkgLuaMatchesCandidate = + [&](const mcpp::pm::DependencyCoordinate& coord, + std::string_view luaContent, + bool allowLegacyBareDefault) { + auto luaName = mcpp::manifest::extract_xpkg_name(luaContent); + if (luaName.empty()) return true; + + auto luaNs = mcpp::manifest::extract_xpkg_namespace(luaContent); + auto qname = candidateQualifiedName(coord.namespace_, coord.shortName); + + if (coord.namespace_.empty()) { + return luaNs.empty() && luaName == coord.shortName; + } + + if (coord.namespace_ == mcpp::pm::kDefaultNamespace) { + if (luaNs == coord.namespace_) { + return luaName == coord.shortName || luaName == qname; + } + if (luaNs.empty() && luaName == qname) return true; + return allowLegacyBareDefault + && luaNs.empty() + && luaName == coord.shortName; + } + + if (luaNs == coord.namespace_) { + return luaName == coord.shortName || luaName == qname; + } + return luaNs.empty() && luaName == qname; + }; + + auto dependencyCoordinates = + [](const mcpp::manifest::DependencySpec& spec, + const std::string& depName) { + if (!spec.candidates.empty()) return spec.candidates; + std::vector out; + out.push_back({ + .namespace_ = spec.namespace_.empty() + ? std::string(mcpp::pm::kDefaultNamespace) + : spec.namespace_, + .shortName = spec.shortName.empty() ? depName : spec.shortName, + }); + return out; + }; + + auto selectDependencyCandidate = + [&](mcpp::manifest::DependencySpec& spec, + const std::string& depName) -> std::expected + { + auto candidates = dependencyCoordinates(spec, depName); + if (candidates.empty()) { + return std::unexpected( + std::format("dependency '{}' has no lookup candidates", depName)); + } + + auto selected = candidates.front(); + if (spec.isVersion() && candidates.size() > 1) { + for (auto& candidate : candidates) { + auto lua = readStrictLuaForCandidate(candidate); + if (lua && xpkgLuaMatchesCandidate( + candidate, *lua, /*allowLegacyBareDefault=*/false)) { + selected = candidate; + break; + } + } + } + + spec.namespace_ = std::move(selected.namespace_); + spec.shortName = std::move(selected.shortName); + spec.candidates = std::move(candidates); + return {}; + }; + + // 0.0.10+: loadVersionDep accepts structured (ns, shortName) for + // namespace-aware lookup. depName is the map key (qualified or bare), + // kept for install() target formatting and error messages. + std::set preinstallStack; + std::set preinstallDone; + + std::function( + const std::string&, + const std::string&, + const std::string&, + const std::string&)> loadVersionDep; + + loadVersionDep = [&](const std::string& depName, + const std::string& ns, + const std::string& shortName, + const std::string& version) + -> std::expected + { + auto cfg = get_cfg(); + if (!cfg) return std::unexpected(cfg.error()); + mcpp::fetcher::Fetcher fetcher(**cfg); + + // ─── Routing: check if this dep's namespace maps to a custom index ── + auto* idxSpec = findIndexForNs(ns); + + const bool useProjectEnv = idxSpec && !idxSpec->is_builtin(); + + auto readLuaContent = [&]() -> std::optional { + if (idxSpec && idxSpec->is_local()) { + auto indexPath = mcpp::config::resolve_project_index_path(*root, *idxSpec); + return mcpp::fetcher::Fetcher::read_xpkg_lua_from_path( + indexPath, ns, shortName); + } + if (idxSpec && !idxSpec->is_builtin()) { + return mcpp::fetcher::Fetcher::read_xpkg_lua_from_project_data( + *root, ns, shortName); + } + return fetcher.read_xpkg_lua(ns, shortName); + }; + + auto luaContent = readLuaContent(); + if (idxSpec && idxSpec->is_local() && !luaContent) { + auto indexPath = mcpp::config::resolve_project_index_path(*root, *idxSpec); + return std::unexpected(std::format( + "dependency '{}': not found in local index at '{}'", + depName, indexPath.string())); + } + + auto findRawInstalled = [&]() -> std::optional { + if (useProjectEnv) { + if (auto p = mcpp::fetcher::Fetcher::install_path_from_project_data( + *root, ns, shortName, version)) { + return p; + } + } + return fetcher.install_path(ns, shortName, version); + }; + + auto installedLayoutMatchesIndex = [&](const std::filesystem::path& verRoot) -> bool { + if (!luaContent) return false; + + auto field = mcpp::manifest::extract_mcpp_field(*luaContent); + if (field.kind == mcpp::manifest::McppField::StringPath) { + return !mcpp::modgraph::expand_glob(verRoot, field.value).empty(); + } + if (field.kind == mcpp::manifest::McppField::TableBody) { + auto dm = mcpp::manifest::synthesize_from_xpkg_lua( + *luaContent, depName, version); + if (!dm) return false; + for (auto const& [generatedPath, _] : dm->buildConfig.generatedFiles) { + if (!generatedPath.empty()) return true; + } + for (auto const& glob : dm->modules.sources) { + if (!glob.empty() && glob.front() == '!') continue; + if (!mcpp::modgraph::expand_glob(verRoot, glob).empty()) { + return true; + } + } + return false; + } + + for (auto pat : { "mcpp.toml", "*/mcpp.toml" }) { + if (!mcpp::modgraph::expand_glob(verRoot, pat).empty()) { + return true; + } + } + return false; + }; + + auto findCompleteInstalled = [&]() -> std::optional { + auto p = findRawInstalled(); + if (!p) return std::nullopt; + if (mcpp::fallback::is_install_complete(*p)) return p; + if (installedLayoutMatchesIndex(*p)) { + mcpp::fallback::mark_install_complete(*p); + return p; + } + mcpp::fallback::clean_incomplete_install(*p); + return std::nullopt; + }; + + auto markInstalled = [&](const std::filesystem::path& p) { + mcpp::fallback::mark_install_complete(p); + }; + + // For custom indices, try project-level xlings data roots first. + // Existing directories without the mcpp completion marker are treated + // as stale/incomplete on this active resolve path and reinstalled. + std::optional installed = findCompleteInstalled(); + + if (!installed) { + if (luaContent) { + auto field = mcpp::manifest::extract_mcpp_field(*luaContent); + if (field.kind == mcpp::manifest::McppField::TableBody) { + auto depManifest = mcpp::manifest::synthesize_from_xpkg_lua( + *luaContent, depName, version); + if (!depManifest) { + return std::unexpected(std::format( + "dependency '{}': {}", depName, depManifest.error().format())); + } + + auto preinstallKey = std::format("{}:{}@{}", ns, shortName, version); + if (preinstallStack.contains(preinstallKey)) { + return std::unexpected(std::format( + "dependency '{}': cyclic mcpp.deps while preparing install hooks", + depName)); + } + + if (!preinstallDone.contains(preinstallKey)) { + preinstallStack.insert(preinstallKey); + for (auto [childName, childSpec] : depManifest->dependencies) { + mcpp::pm::compat::normalize_nested_namespace( + childSpec.namespace_, + childSpec.shortName, + childSpec.legacyDottedKey); + + if (auto r = selectDependencyCandidate( + childSpec, childName); !r) { + preinstallStack.erase(preinstallKey); + return std::unexpected(r.error()); + } + + if (auto r = resolveSemver(childSpec, childName); !r) { + preinstallStack.erase(preinstallKey); + return std::unexpected(r.error()); + } + + if (!childSpec.isVersion()) continue; + + ResolvedKey childKey{ + childSpec.namespace_, + childSpec.shortName.empty() ? childName : childSpec.shortName, + }; + if (auto child = loadVersionDep( + childName, + childKey.ns, + childKey.shortName, + childSpec.version); !child) { + preinstallStack.erase(preinstallKey); + return std::unexpected(child.error()); + } + } + preinstallStack.erase(preinstallKey); + preinstallDone.insert(preinstallKey); + } + } + } + + // xlings resolves packages by the full qualified name (ns.shortName) + // as it appears in the index's name field. Use fqname, not the + // map key (which may be a bare short name for default-ns deps). + auto fqname = ns.empty() ? shortName + : std::format("{}.{}", ns, shortName); + mcpp::ui::info("Downloading", std::format("{} v{}", fqname, version)); + + auto install_one = [&](std::string target) -> std::expected { + if (useProjectEnv) { + // Project/custom-index deps install into the project-local + // xlings data root (so a package's install hook can find + // sibling packages from the same index). The NDJSON + // interface honors this: in the pinned xlings the + // `install_packages` capability and the `install` CLI share + // `xim::cmd_install`, and the install destination is chosen + // by package *scope* (project vs global), not by transport. + // Using the interface (rather than the silenced direct CLI) + // restores the live `Downloading … [bar] X/Y Z/s` UI here, + // matching the toolchain and builtin-index paths. + auto projEnv = mcpp::config::make_project_xlings_env(**cfg, *root); + auto argsJson = std::format( + R"({{"targets":["{}"],"yes":true}})", target); + mcpp::fetcher::InstallProgressHandler progress; + auto r = mcpp::xlings::call( + projEnv, "install_packages", argsJson, &progress); + if (!r) return std::unexpected(mcpp::pm::CallError{r.error()}); + return *r; + } + std::vector targets{ std::move(target) }; + mcpp::fetcher::InstallProgressHandler progress; + return fetcher.install(targets, &progress); + }; + auto target = std::format("{}@{}", fqname, version); + // For custom indices, use indexName:fullPackageName@version so + // xlings resolves the package by the descriptor's name field while + // still selecting the project-added index. + if (useProjectEnv) { + target = std::format("{}:{}@{}", idxSpec->name, fqname, version); + } + auto r = install_one(target); + if (r && r->exitCode != 0 && + (ns.empty() || ns == mcpp::pm::kDefaultNamespace)) { + auto compatTarget = std::format("compat.{}@{}", shortName, version); + if (compatTarget != target) { + mcpp::ui::info("Downloading", std::format("{} v{}", + std::format("compat.{}", shortName), version)); + r = install_one(compatTarget); + } + } + if (!r) return std::unexpected(std::format( + "fetch '{}@{}': {}", depName, version, r.error().message)); + if (r->exitCode != 0) { + std::string err = std::format( + "fetch '{}@{}' failed (exit {})", depName, version, r->exitCode); + if (r->error) err += ": " + r->error->message; + return std::unexpected(err); + } + // After install, check project data first for custom index packages. + installed = findRawInstalled(); + if (!installed) return std::unexpected(std::format( + "package '{}@{}' install path missing after fetch", depName, version)); + markInstalled(*installed); + } + std::filesystem::path verRoot = *installed; + + // Route xpkg.lua reading through the appropriate index. + if (!luaContent) { + luaContent = readLuaContent(); + } + if (!luaContent) return std::unexpected(std::format( + "dependency '{}': index entry not found in local clone", depName)); + auto field = mcpp::manifest::extract_mcpp_field(*luaContent); + + // 0.0.6+: read explicit namespace from xpkg lua if present. + auto luaNs = mcpp::manifest::extract_xpkg_namespace(*luaContent); + + std::optional manifest; + std::filesystem::path effRoot = verRoot; + auto loadFrom = [&](const std::filesystem::path& mcppToml) + -> std::expected + { + auto dm = mcpp::manifest::load(mcppToml); + if (!dm) return std::unexpected(std::format( + "dependency '{}' (at '{}'): {}", + depName, mcppToml.string(), dm.error().format())); + manifest = std::move(*dm); + effRoot = mcppToml.parent_path(); + return {}; + }; + if (field.kind == mcpp::manifest::McppField::StringPath) { + auto matches = mcpp::modgraph::expand_glob(verRoot, field.value); + if (matches.empty()) return std::unexpected(std::format( + "dependency '{}': mcpp pointer '{}' did not match any " + "file under '{}'", depName, field.value, verRoot.string())); + if (matches.size() > 1) return std::unexpected(std::format( + "dependency '{}': mcpp pointer '{}' matched {} files " + "(expected exactly one)", depName, field.value, matches.size())); + if (auto r = loadFrom(matches.front()); !r) return std::unexpected(r.error()); + } else if (field.kind == mcpp::manifest::McppField::TableBody) { + auto dm = mcpp::manifest::synthesize_from_xpkg_lua(*luaContent, depName, version); + if (!dm) return std::unexpected(std::format( + "dependency '{}': {}", depName, dm.error().format())); + manifest = std::move(*dm); + // effRoot stays as verRoot + } else { + std::vector matches; + for (auto pat : { "mcpp.toml", "*/mcpp.toml" }) { + matches = mcpp::modgraph::expand_glob(verRoot, pat); + if (!matches.empty()) break; + } + if (matches.empty()) return std::unexpected(std::format( + "dependency '{}': index entry has no `mcpp = ...` field, " + "and no mcpp.toml was found at /mcpp.toml or " + "/*/mcpp.toml — add an explicit `mcpp = \"\"` " + "or `mcpp = {{ ... }}` block to the .lua descriptor.", + depName)); + if (matches.size() > 1) return std::unexpected(std::format( + "dependency '{}': default mcpp.toml lookup matched {} " + "files; pin one with explicit `mcpp = \"\"`.", + depName, matches.size())); + if (auto r = loadFrom(matches.front()); !r) return std::unexpected(r.error()); + } + // Propagate lua-level namespace into the loaded manifest when + // the manifest itself doesn't carry one (Form A descriptors + // whose upstream mcpp.toml predates the namespace field). + // Guard: if the manifest's name already starts with luaNs+"." + // (e.g. name="mcpplibs.tinyhttps" with luaNs="mcpplibs"), + // the namespace is already embedded in the name — don't inject + // it again or the scanner will produce a double-prefixed + // qualified name like "mcpplibs.mcpplibs.tinyhttps". + if (manifest->package.namespace_.empty() && !luaNs.empty()) { + auto prefix = luaNs + "."; + if (!manifest->package.name.starts_with(prefix)) { + manifest->package.namespace_ = luaNs; + } + } + + if (auto r = materialize_generated_files(effRoot, *manifest); !r) { + return std::unexpected(std::format( + "dependency '{}': {}", depName, r.error())); + } + + return std::pair{effRoot, std::move(*manifest)}; + }; + + struct DependencyEdge { + std::size_t consumerPackageIndex = 0; + std::size_t dependencyPackageIndex = 0; + mcpp::modgraph::DependencyVisibility visibility = + mcpp::modgraph::DependencyVisibility::Public; + }; + std::vector dependencyEdges; + + auto parseVisibility = [](std::string_view visibility) { + if (visibility == "private") + return mcpp::modgraph::DependencyVisibility::Private; + if (visibility == "interface") + return mcpp::modgraph::DependencyVisibility::Interface; + return mcpp::modgraph::DependencyVisibility::Public; + }; + + auto packageIndexForConsumer = [&](std::size_t consumerDepIndex) { + if (consumerDepIndex == kMainConsumer) return std::size_t{0}; + return consumerDepIndex + 1; + }; + + auto appendUniquePath = + [](std::vector& dirs, + const std::filesystem::path& dir) -> bool + { + if (std::find(dirs.begin(), dirs.end(), dir) != dirs.end()) return false; + dirs.push_back(dir); + return true; + }; + + auto appendUniquePaths = + [&](std::vector& dirs, + const std::vector& additions) -> bool + { + bool changed = false; + for (auto const& dir : additions) { + changed = appendUniquePath(dirs, dir) || changed; + } + return changed; + }; + + auto expandIncludeDirs = + [&](const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) + { + std::vector dirs; + for (auto const& inc : manifest.buildConfig.includeDirs) { + if (inc.is_absolute()) { + appendUniquePath(dirs, inc); + continue; + } + for (auto& dir : mcpp::modgraph::expand_dir_glob( + packageRoot, inc.generic_string())) { + appendUniquePath(dirs, dir); + } + } + return dirs; + }; + + auto makePackageRoot = + [&](const std::filesystem::path& packageRoot, + const mcpp::manifest::Manifest& manifest) + { + mcpp::modgraph::PackageRoot pkg; + pkg.root = packageRoot; + pkg.manifest = manifest; + pkg.usageResolved = true; + + pkg.privateBuild.includeDirs = expandIncludeDirs(packageRoot, manifest); + pkg.privateBuild.cflags = manifest.buildConfig.cflags; + pkg.privateBuild.cxxflags = manifest.buildConfig.cxxflags; + pkg.publicUsage.includeDirs = pkg.privateBuild.includeDirs; + pkg.linkUsage.ldflags = manifest.buildConfig.ldflags; + return pkg; + }; + + packages[0] = makePackageRoot(*root, *m); + + auto recordDependencyEdge = + [&](std::size_t consumerDepIndex, + std::size_t dependencyPackageIndex, + const mcpp::manifest::DependencySpec& spec) + { + const auto consumerPackageIndex = packageIndexForConsumer(consumerDepIndex); + if (consumerPackageIndex >= packages.size() + || dependencyPackageIndex >= packages.size()) { + return; + } + const auto visibility = parseVisibility(spec.visibility); + auto same = [&](const DependencyEdge& edge) { + return edge.consumerPackageIndex == consumerPackageIndex + && edge.dependencyPackageIndex == dependencyPackageIndex + && edge.visibility == visibility; + }; + if (std::find_if(dependencyEdges.begin(), dependencyEdges.end(), same) + != dependencyEdges.end()) { + return; + } + dependencyEdges.push_back(DependencyEdge{ + .consumerPackageIndex = consumerPackageIndex, + .dependencyPackageIndex = dependencyPackageIndex, + .visibility = visibility, + }); + }; + + auto computeUsageRequirements = [&] { + bool changed = true; + while (changed) { + changed = false; + for (auto const& edge : dependencyEdges) { + if (edge.consumerPackageIndex >= packages.size() + || edge.dependencyPackageIndex >= packages.size()) { + continue; + } + auto& consumer = packages[edge.consumerPackageIndex]; + auto const& dependency = packages[edge.dependencyPackageIndex]; + + if (edge.visibility == mcpp::modgraph::DependencyVisibility::Private + || edge.visibility == mcpp::modgraph::DependencyVisibility::Public) { + changed = appendUniquePaths(consumer.privateBuild.includeDirs, + dependency.publicUsage.includeDirs) + || changed; + } + if (edge.visibility == mcpp::modgraph::DependencyVisibility::Public + || edge.visibility == mcpp::modgraph::DependencyVisibility::Interface) { + changed = appendUniquePaths(consumer.publicUsage.includeDirs, + dependency.publicUsage.includeDirs) + || changed; + } + } + } + }; + + auto normalizeDepLdflag = [](const std::filesystem::path& depRoot, + const std::string& flag) { + auto absolute_path = [&](std::string_view raw) { + std::filesystem::path p{std::string(raw)}; + if (p.is_absolute() || raw.starts_with("$")) return p; + return depRoot / p; + }; + + if (flag.starts_with("-L") && flag.size() > 2) { + return "-L" + absolute_path(std::string_view(flag).substr(2)).string(); + } + + constexpr std::string_view rpathPrefix = "-Wl,-rpath,"; + if (flag.starts_with(rpathPrefix) && flag.size() > rpathPrefix.size()) { + return std::string(rpathPrefix) + + absolute_path(std::string_view(flag).substr(rpathPrefix.size())).string(); + } + + return flag; + }; + + auto propagateLinkFlags = [&](const std::filesystem::path& depRoot, + const mcpp::manifest::Manifest& depManifest) + -> std::vector + { + std::vector added; + for (auto const& flag : depManifest.buildConfig.ldflags) { + auto normalized = normalizeDepLdflag(depRoot, flag); + m->buildConfig.ldflags.push_back(normalized); + added.push_back(std::move(normalized)); + } + return added; + }; + + auto removeLinkFlags = [&](const std::vector& flags) { + auto& ldflags = m->buildConfig.ldflags; + for (auto const& flag : flags) { + auto pos = std::find(ldflags.begin(), ldflags.end(), flag); + if (pos != ldflags.end()) ldflags.erase(pos); + } + }; + + // Stage a dep's source files into a fresh directory, rewriting their + // module / import declarations against `rename`. Used by the multi- + // version mangling fallback (Level 1) so two cross-major copies of + // the same package can coexist with distinct module names. + // + // Headers (referenced via `[build].include_dirs`) are NOT staged — + // those keep pointing at the original install dir via absolutized + // include paths. + auto stage_with_rewrite = [](const std::filesystem::path& srcRoot, + const std::filesystem::path& dstRoot, + const mcpp::manifest::Manifest& depManifest, + const std::map& rename) + -> std::expected + { + std::error_code ec; + std::filesystem::create_directories(dstRoot, ec); + if (ec) return std::unexpected(std::format( + "stage: cannot create '{}': {}", dstRoot.string(), ec.message())); + + // Resolve the source globs against the original root, falling + // back to the convention default if the manifest didn't set any. + std::vector globs = depManifest.modules.sources; + if (globs.empty()) { + globs = { "src/**/*.cppm", "src/**/*.cpp", + "src/**/*.cc", "src/**/*.c" }; + } + // Glob exclusion (same as scan_one_into): `!` prefix removes. + std::set sourceFiles; + std::set excluded; + for (auto const& g : globs) { + if (!g.empty() && g[0] == '!') { + for (auto& p : mcpp::modgraph::expand_glob(srcRoot, g.substr(1))) + excluded.insert(p); + } else { + for (auto& p : mcpp::modgraph::expand_glob(srcRoot, g)) + sourceFiles.insert(p); + } + } + for (auto& p : excluded) sourceFiles.erase(p); + if (sourceFiles.empty()) { + return std::unexpected(std::format( + "stage: no source files found under '{}' (globs={})", + srcRoot.string(), globs.size())); + } + + for (auto const& f : sourceFiles) { + auto rel = std::filesystem::relative(f, srcRoot, ec); + if (ec) return std::unexpected(std::format( + "stage: cannot relativize '{}': {}", f.string(), ec.message())); + auto dst = dstRoot / rel; + std::filesystem::create_directories(dst.parent_path(), ec); + + std::ifstream is(f); + if (!is) return std::unexpected(std::format( + "stage: cannot read '{}'", f.string())); + std::stringstream buf; buf << is.rdbuf(); + std::string content = buf.str(); + + std::string out = mcpp::pm::rewrite_module_decls(content, rename); + std::ofstream os(dst); + if (!os) return std::unexpected(std::format( + "stage: cannot write '{}'", dst.string())); + os << out; + } + return {}; + }; + + // Seed the worklist from the main manifest. Dev-deps only when the + // caller wants them; they're never propagated transitively. + const std::string mainPkgLabel = m->package.name; + for (auto& [n, s] : m->dependencies) { + worklist.push_back({n, s, mainPkgLabel, s.version, kMainConsumer, {}}); + } + if (includeDevDeps) { + for (auto& [n, s] : m->devDependencies) { + worklist.push_back({n, s, mainPkgLabel + " (dev-dep)", + s.version, kMainConsumer, {}}); + } + } + + while (!worklist.empty()) { + auto item = std::move(worklist.front()); + worklist.pop_front(); + + const auto& name = item.name; + auto& spec = item.spec; + + mcpp::pm::compat::normalize_nested_namespace( + spec.namespace_, spec.shortName, spec.legacyDottedKey); + if (spec.legacyDottedKey) { + spec.candidates = {{ + .namespace_ = spec.namespace_, + .shortName = spec.shortName, + }}; + } + + if (auto r = selectDependencyCandidate(spec, name); !r) { + return std::unexpected(r.error()); + } + if (item.consumerDepIndex == kMainConsumer) { + if (auto it = m->dependencies.find(name); it != m->dependencies.end()) { + it->second.namespace_ = spec.namespace_; + it->second.shortName = spec.shortName; + it->second.candidates = spec.candidates; + } + } + + // Pin SemVer constraint before dedup/fetch. + if (auto r = resolveSemver(spec, name); !r) { + return std::unexpected(r.error()); + } + + ResolvedKey key{ + spec.namespace_, + spec.shortName.empty() ? name : spec.shortName, + }; + const std::string sourceKind = + spec.isPath() ? "path" + : spec.isGit() ? "git" + : "version"; + + if (auto it = resolved.find(key); it != resolved.end()) { + // Conflict detection. + if (it->second.source != sourceKind) { + return std::unexpected(std::format( + "dependency '{}{}{}' is requested as both a {} dep " + "(by '{}') and a {} dep (by '{}'). Pick one.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.source, it->second.requestedBy, + sourceKind, item.requestedBy)); + } + if (sourceKind == "version" && it->second.version != spec.version) { + // SemVer merge attempt: AND-combine the two original + // constraint strings and ask the index for a single version + // satisfying both. Same-major caret/tilde/exact pairs that + // overlap converge here; cross-major or otherwise + // unsatisfiable pairs fall through to a hard error (a future + // PR adds multi-version mangling as a Level-1 fallback). + auto cfg = get_cfg(); + if (!cfg) return std::unexpected(cfg.error()); + mcpp::fetcher::Fetcher fetcher(**cfg); + + auto merged = mcpp::pm::try_merge_semver( + key.ns, key.shortName, + it->second.constraint, + item.originalConstraint, + fetcher); + if (!merged) { + // Level 1 fallback: multi-version mangling. Two + // versions can't be reconciled by SemVer, but they + // can coexist in the same build if we mangle the + // secondary copy's module name and rewrite the one + // consumer that asked for it. The primary keeps its + // authored module name so consumers that don't care + // about the secondary see no churn. + // + // MVP scope (these limits surface as clear errors): + // * The conflicting consumer must be a dep, not + // the main package — main-package mangling + // would mean rewriting user-authored sources, + // which is too surprising for a fallback path. + // * The secondary version must be a leaf (no own + // transitive deps) — recursive mangling is + // deferred to a follow-up. + if (item.consumerDepIndex == kMainConsumer) { + return std::unexpected(std::format( + "dependency '{}{}{}' has irreconcilable versions:\n" + " '{}' (constraint '{}') requested by '{}'\n" + " '{}' (constraint '{}') requested by '{}'\n" + "SemVer merge: {}\n" + "Multi-version mangling can't help here — the conflict " + "involves the main package directly. Pin one version " + "explicitly in your mcpp.toml.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, it->second.constraint, it->second.requestedBy, + spec.version, item.originalConstraint, item.requestedBy, + merged.error())); + } + + auto loaded = loadVersionDep(name, key.ns, key.shortName, spec.version); + if (!loaded) return std::unexpected(loaded.error()); + auto& [secondaryRoot, secondaryManifest] = *loaded; + + if (!secondaryManifest.dependencies.empty()) { + return std::unexpected(std::format( + "dependency '{}{}{}' has irreconcilable versions:\n" + " '{}' requested by '{}'\n" + " '{}' requested by '{}'\n" + "Multi-version mangling fallback only handles leaf " + "secondaries in 0.0.3 — but the secondary v{} declares " + "its own dependencies, which would need recursive " + "mangling. Pin one version explicitly, or wait for " + "the recursive-mangling extension.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, it->second.requestedBy, + spec.version, item.requestedBy, + spec.version)); + } + + // Module names in the source files use the dep's full + // [package].name (e.g. "mcpplibs.cmdline"), not the + // namespaced-subtable shortName. Use that for the + // rename key so the rewriter actually matches what the + // .cppm sources declare. + const std::string moduleName = secondaryManifest.package.name; + std::string mangled = + mcpp::pm::mangle_name(moduleName, spec.version); + + // Stage layout: + // /target/.mangled//__/ ← rewritten secondary source + // /target/.mangled//__self__/ ← rewritten consumer source + auto& consumerManifest = *dep_manifests[item.consumerDepIndex]; + auto consumerRoot = packages[item.consumerDepIndex + 1].root; + auto stageBase = *root / "target" / ".mangled" + / consumerManifest.package.name; + auto secStage = stageBase + / std::format("{}__{}", moduleName, spec.version); + auto consumerStage = stageBase / "__self__"; + + std::map rename{ {moduleName, mangled} }; + if (auto r = stage_with_rewrite(secondaryRoot, secStage, + secondaryManifest, rename); !r) + return std::unexpected(r.error()); + if (auto r = stage_with_rewrite(consumerRoot, consumerStage, + consumerManifest, rename); !r) + return std::unexpected(r.error()); + + // Re-anchor the consumer's PackageRoot at its staged copy + // so the modgraph scanner picks up the rewritten imports. + packages[item.consumerDepIndex + 1].root = consumerStage; + + // Record the staged secondary as a brand-new dep entry + // under its mangled name, so future encounters of this + // exact (ns, mangled) pair dedup cleanly. The original + // primary entry (it->second) is untouched. + auto stagedManifest = secondaryManifest; + // Update [package].name to the mangled module name so + // the modgraph validator (which checks "exported module + // must be prefixed by package name") accepts the + // rewritten sources. + stagedManifest.package.name = mangled; + // Absolutize secondary's include_dirs against its original + // install root so the staged copy still finds headers. + for (auto& inc : stagedManifest.buildConfig.includeDirs) { + if (inc.is_relative()) inc = secondaryRoot / inc; + } + + dep_manifests.push_back( + std::make_unique(std::move(stagedManifest))); + dep_cache_identities.push_back({ + .indexName = cache_index_name(key.ns), + .packageName = mangled, + .version = spec.version, + }); + const auto depPackageIndex = packages.size(); + packages.push_back(makePackageRoot(secStage, *dep_manifests.back())); + recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec); + auto linkFlagsAdded = propagateLinkFlags(secStage, *dep_manifests.back()); + + ResolvedKey mangledKey{key.ns, mangled}; + resolved[mangledKey] = ResolvedRecord{ + .version = spec.version, + .constraint = item.originalConstraint, + .requestedBy = item.requestedBy, + .source = "version", + .depIndex = dep_manifests.size() - 1, + .linkFlagsAdded = std::move(linkFlagsAdded), + }; + + mcpp::ui::info("Mangled", + std::format("{} v{} ↔ v{} → {} (cross-major fallback)", + moduleName, it->second.version, spec.version, mangled)); + continue; + } + + // Combine the constraint strings so future merges AND with + // both. Empty originalConstraint means "any" — use "*". + const std::string& addCstr = + item.originalConstraint.empty() ? std::string("*") + : item.originalConstraint; + if (it->second.constraint.empty()) + it->second.constraint = addCstr; + else + it->second.constraint += "," + addCstr; + + if (*merged == it->second.version) { + // The existing pin already satisfies the new constraint — + // no re-fetch needed; just record this consumer edge. + recordDependencyEdge(item.consumerDepIndex, + it->second.depIndex + 1, + spec); + continue; + } + + // Merged version differs from the previously-pinned one. + // Re-fetch the dep at the merged version and replace the + // earlier slot in dep_manifests / packages so the build plan + // sees only one version. Old include_dir entries are evicted + // and the new manifest's entries are appended. + mcpp::ui::info("Merged", + std::format("{}{}{} {} ⨯ {} → v{}", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.version, spec.version, *merged)); + auto reloaded = loadVersionDep(name, key.ns, key.shortName, *merged); + if (!reloaded) return std::unexpected(reloaded.error()); + auto& [newRoot, newManifest] = *reloaded; + + // Name match against the re-loaded manifest. + { + const std::string& expectedShort = + spec.shortName.empty() ? name : spec.shortName; + // Also accept the fully-qualified form (ns.short) since + // synthesize_from_xpkg_lua may set package.name to the + // composite name for backward compat. + auto expectedComposite = spec.namespace_.empty() + ? std::string{} + : std::format("{}.{}", spec.namespace_, expectedShort); + const bool nameOk = + newManifest.package.name == expectedShort + || newManifest.package.name == name + || (!expectedComposite.empty() + && newManifest.package.name == expectedComposite); + if (!nameOk) { + return std::unexpected(std::format( + "dependency '{}' (merged to v{}) resolved to " + "package '{}' (mismatch with declared name '{}')", + name, *merged, newManifest.package.name, + expectedShort)); + } + } + + removeLinkFlags(it->second.linkFlagsAdded); + auto linkFlagsAdded = propagateLinkFlags(newRoot, newManifest); + + // Replace in dep_manifests + packages. depIndex is the slot + // in dep_manifests; packages = [main, dep_0, dep_1, …], so + // packages[depIndex+1] is the same dep. + *dep_manifests[it->second.depIndex] = std::move(newManifest); + packages[it->second.depIndex + 1] = + makePackageRoot(newRoot, *dep_manifests[it->second.depIndex]); + recordDependencyEdge(item.consumerDepIndex, + it->second.depIndex + 1, + spec); + + it->second.version = *merged; + it->second.linkFlagsAdded = std::move(linkFlagsAdded); + if (it->second.depIndex < dep_cache_identities.size()) + dep_cache_identities[it->second.depIndex].version = *merged; + + // Walk the *new* manifest's deps so their constraints feed + // future merges. Already-resolved children dedup via the + // resolved map. + const std::string newLabel = std::format("{}{}{}@{}", + key.ns, key.ns.empty() ? "" : ".", + key.shortName, *merged); + for (auto& [child_name, child_spec] : + dep_manifests[it->second.depIndex]->dependencies) { + worklist.push_back({child_name, child_spec, newLabel, + child_spec.version, + it->second.depIndex, {}}); + } + continue; + } + // Same key, same version (or compatible path/git) — already + // processed; still record the dependency edge before skipping. + // Usage propagation is per edge, not per unique package: two + // consumers can need the same dep's public surface even though + // the dep itself is fetched/scanned once. + if (it->second.depIndex + 1 < packages.size()) { + recordDependencyEdge(item.consumerDepIndex, + it->second.depIndex + 1, + spec); + } + continue; + } + + std::filesystem::path dep_root; + + if (spec.isPath()) { + // Path-based: resolve relative to the consumer's root dir. + // For top-level deps this is the project root; for transitive + // deps it's the parent dep's directory (stored in resolveRoot). + dep_root = spec.path; + auto base = item.resolveRoot.empty() ? *root : item.resolveRoot; + if (dep_root.is_relative()) dep_root = base / dep_root; + dep_root = std::filesystem::weakly_canonical(dep_root); + } else if (spec.isGit()) { + // Git-based (M4 #5): clone into ~/.mcpp/git// and treat + // as a path dep from there. Branch refs are floating, so resolve + // them to a commit before forming the cache key; this lets + // `mcpp update ` pick up a moved branch without deleting + // unrelated git caches. + auto mcppHome = [] { + if (auto* e = std::getenv("MCPP_HOME"); e && *e) + return std::filesystem::path(e); + if (auto* e = std::getenv("HOME"); e && *e) + return std::filesystem::path(e) / ".mcpp"; + return std::filesystem::current_path() / ".mcpp"; + }(); + std::string resolvedGitRev = spec.gitRev; + if (spec.gitRefKind == "branch") { + auto ref = std::format("refs/heads/{}", spec.gitRev); + auto cmd = std::format( + "git ls-remote {} {} 2>&1", + mcpp::platform::shell::quote(spec.git), + mcpp::platform::shell::quote(ref)); + auto r = mcpp::platform::process::capture(cmd); + if (r.exit_code != 0) { + return std::unexpected(std::format( + "git ls-remote of '{}' failed:\n{}", spec.git, r.output)); + } + std::istringstream is(r.output); + is >> resolvedGitRev; + if (resolvedGitRev.empty()) { + return std::unexpected(std::format( + "git branch '{}' not found in '{}'", spec.gitRev, spec.git)); + } + } + + // Cache key: hash(url + refkind + declared ref + resolved commit). + // For fixed rev/tag deps the declared ref is also the resolved ref. + std::hash H; + auto urlHash = std::format("{:016x}", + H(spec.git + "|" + spec.gitRefKind + "|" + spec.gitRev + + "|" + resolvedGitRev)); + auto gitRoot = mcppHome / "git" / urlHash; + std::error_code ec; + std::filesystem::create_directories(gitRoot.parent_path(), ec); + if (!std::filesystem::exists(gitRoot / ".git")) { + mcpp::ui::info("Cloning", + std::format("{} ({} = {})", spec.git, spec.gitRefKind, spec.gitRev)); + std::string cloneCmd; + if (spec.gitRefKind == "branch") { + cloneCmd = std::format( + "git clone --depth 1 --branch {} {} {} && cd {} && git checkout --quiet {} 2>&1", + mcpp::platform::shell::quote(spec.gitRev), + mcpp::platform::shell::quote(spec.git), + mcpp::platform::shell::quote(gitRoot.string()), + mcpp::platform::shell::quote(gitRoot.string()), + mcpp::platform::shell::quote(resolvedGitRev)); + } else { + // For tag/rev: full clone, then checkout (depth-1 may miss the rev). + cloneCmd = std::format( + "git clone {} {} && cd {} && git checkout --quiet {} 2>&1", + mcpp::platform::shell::quote(spec.git), + mcpp::platform::shell::quote(gitRoot.string()), + mcpp::platform::shell::quote(gitRoot.string()), + mcpp::platform::shell::quote(spec.gitRev)); + } + std::string out; + { + auto r = mcpp::platform::process::capture(cloneCmd); + out = r.output; + int rc = r.exit_code; + if (rc != 0) { + std::filesystem::remove_all(gitRoot, ec); + return std::unexpected(std::format( + "git clone of '{}' failed:\n{}", spec.git, out)); + } + } + } + if (item.consumerDepIndex == kMainConsumer) { + auto source = std::format("git+{}#{}={}", + spec.git, spec.gitRefKind, spec.gitRev); + if (spec.gitRefKind == "branch") source += "@" + resolvedGitRev; + root_git_lock_identities[name] = GitLockIdentity{ + .source = std::move(source), + .hash = std::format("fnv1a:{:016x}", H(spec.git + "|" + + spec.gitRefKind + "|" + spec.gitRev + "|" + + resolvedGitRev)), + }; + } + dep_root = gitRoot; + } + // (version-source: dep_root + manifest are loaded together via + // loadVersionDep below since the index entry drives both.) + + // Manifest acquisition. + // - Path/git dep: dep_root is the source tree, mcpp.toml at root. + // - Version dep: delegate to loadVersionDep — the index entry's + // `mcpp` field decides where mcpp.toml lives (StringPath / + // TableBody / default lookup). + std::optional dep_manifest; + if (spec.isPath() || spec.isGit()) { + if (!std::filesystem::exists(dep_root / "mcpp.toml")) { + return std::unexpected(std::format( + "{} dependency '{}' (at '{}') has no mcpp.toml", + spec.isGit() ? "git" : "path", name, dep_root.string())); + } + auto dm = mcpp::manifest::load(dep_root / "mcpp.toml"); + if (!dm) { + return std::unexpected(std::format( + "dependency '{}' (at '{}'): {}", + name, dep_root.string(), dm.error().format())); + } + dep_manifest = std::move(*dm); + } else { + auto loaded = loadVersionDep(name, key.ns, key.shortName, spec.version); + if (!loaded) return std::unexpected(loaded.error()); + dep_root = std::move(loaded->first); + dep_manifest = std::move(loaded->second); + } + + // Name match via compat::resolve_package_name — handles both + // canonical (explicit namespace field) and legacy (dotted name) + // forms transparently. + { + auto resolved = mcpp::pm::compat::resolve_package_name( + dep_manifest->package.name, dep_manifest->package.namespace_); + const std::string& expectedShort = + spec.shortName.empty() ? name : spec.shortName; + const bool nameOk = + resolved.shortName == expectedShort + || dep_manifest->package.name == expectedShort + || dep_manifest->package.name == + mcpp::pm::compat::qualified_name(spec.namespace_, expectedShort); + if (!nameOk) { + return std::unexpected(std::format( + "dependency '{}' resolved to package '{}' (mismatch with declared name '{}')", + name, dep_manifest->package.name, expectedShort)); + } + } + + auto linkFlagsAdded = propagateLinkFlags(dep_root, *dep_manifest); + + // Move the manifest into stable storage so we can later look it up + // by depIndex (the SemVer merger needs to overwrite the slot). + dep_manifests.push_back( + std::make_unique(std::move(*dep_manifest))); + dep_cache_identities.push_back({ + .indexName = cache_index_name(key.ns), + .packageName = name, + .version = sourceKind == "version" + ? spec.version + : dep_manifests.back()->package.version, + }); + const auto depPackageIndex = packages.size(); + packages.push_back(makePackageRoot(dep_root, *dep_manifests.back())); + recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec); + + // Record this dep as resolved so future encounters of the same + // (ns, name) hit the fast path (skip / merge / conflict). + resolved[key] = ResolvedRecord{ + .version = sourceKind == "version" ? spec.version : "", + .constraint = sourceKind == "version" ? item.originalConstraint : "", + .requestedBy = item.requestedBy, + .source = sourceKind, + .depIndex = dep_manifests.size() - 1, + .linkFlagsAdded = std::move(linkFlagsAdded), + }; + + // Recurse: the dep's own [dependencies] become new worklist items. + // dev-dependencies are intentionally NOT walked — those are + // private to the dep's test runs, not part of its public ABI. + const std::string thisDepLabel = std::format( + "{}{}{}@{}", + key.ns, + key.ns.empty() ? "" : ".", + key.shortName, + sourceKind == "version" ? spec.version : sourceKind); + const std::size_t selfIdx = dep_manifests.size() - 1; + for (auto& [child_name, child_spec] : dep_manifests.back()->dependencies) { + worklist.push_back({child_name, child_spec, thisDepLabel, + child_spec.version, selfIdx, dep_root}); + } + } + + computeUsageRequirements(); + + // ─── Feature activation (Cargo-style, additive) ──────────────────── + // activated(pkg) = pkg.[features].default ∪ features requested for it + // (root: --features; deps: the root dep spec's `features = [...]`). + // Implied features expand transitively. Each active feature becomes + // -DMCPP_FEATURE_ on that package's compile flags. + // (Transitive dep→dep feature requests are not yet propagated.) + { + auto sanitize = [](std::string f) { + for (auto& c : f) + c = std::isalnum(static_cast(c)) + ? static_cast(std::toupper(static_cast(c))) : '_'; + return f; + }; + auto activate = [](const mcpp::manifest::Manifest& pm, + const std::vector& requested) { + std::vector act, q; + if (auto it = pm.featuresMap.find("default"); it != pm.featuresMap.end()) + q.insert(q.end(), it->second.begin(), it->second.end()); + q.insert(q.end(), requested.begin(), requested.end()); + std::set seen; + while (!q.empty()) { + auto f = q.back(); q.pop_back(); + if (f == "default" || !seen.insert(f).second) continue; + act.push_back(f); + if (auto it = pm.featuresMap.find(f); it != pm.featuresMap.end()) + q.insert(q.end(), it->second.begin(), it->second.end()); + } + return act; + }; + auto apply = [&](mcpp::modgraph::PackageRoot& pkg, + const std::vector& requested) { + for (auto& f : activate(pkg.manifest, requested)) { + auto def = "-DMCPP_FEATURE_" + sanitize(f); + pkg.manifest.buildConfig.cflags.push_back(def); + pkg.manifest.buildConfig.cxxflags.push_back(def); + pkg.privateBuild.cflags.push_back(def); + pkg.privateBuild.cxxflags.push_back(def); + } + }; + if (!packages.empty()) { + std::vector rootReq; + for (std::size_t p = 0; p < overrides.features.size();) { + auto c = overrides.features.find_first_of(", ", p); + auto tok = overrides.features.substr( + p, c == std::string::npos ? std::string::npos : c - p); + if (!tok.empty()) rootReq.push_back(tok); + if (c == std::string::npos) break; + p = c + 1; + } + // Strict schema check: a requested feature must exist in the + // target package's [features] table when one is declared (a + // package with no [features] accepts any request — pure-define + // usage). Covers backend= sugar (feature backend-) too. + auto unknown_requested = [](const mcpp::manifest::Manifest& pm, + const std::vector& requested) + -> std::optional { + if (pm.featuresMap.empty()) return std::nullopt; + for (auto& f : requested) + if (!pm.featuresMap.contains(f)) return f; + return std::nullopt; + }; + if (auto bad = unknown_requested(packages[0].manifest, rootReq)) { + auto msg = std::format( + "--features requests '{}' which [features] does not declare", *bad); + if (overrides.strict) return std::unexpected(msg); + std::println(stderr, "warning: {}", msg); + } + apply(packages[0], rootReq); + } + for (std::size_t i = 1; i < packages.size(); ++i) { + auto& pname = packages[i].manifest.package.name; + std::vector req; + for (auto& [dname, dspec] : m->dependencies) { + if (dname == pname || dspec.shortName == pname) { req = dspec.features; break; } + } + if (!req.empty() && !packages[i].manifest.featuresMap.empty()) { + for (auto& f : req) { + if (packages[i].manifest.featuresMap.contains(f)) continue; + auto msg = std::format( + "dependency '{}' does not declare requested feature '{}' " + "in its [features] table", pname, f); + if (overrides.strict) return std::unexpected(msg); + std::println(stderr, "warning: {}", msg); + } + } + if (!req.empty() || packages[i].manifest.featuresMap.contains("default")) + apply(packages[i], req); + } + } + + // Modgraph: regex scanner by default; opt-in to compiler-driven P1689 + // scanner via env var MCPP_SCANNER=p1689 (see docs/27). + auto scan = [&] { + const char* sel = std::getenv("MCPP_SCANNER"); + if (sel && std::string_view(sel) == "p1689") { + auto tmp = std::filesystem::temp_directory_path() + / std::format("mcpp_p1689_{}", std::random_device{}()); + std::filesystem::create_directories(tmp); + return mcpp::modgraph::scan_packages_p1689(packages, *tc, tmp, m->cppStandard.flag); + } + return mcpp::modgraph::scan_packages(packages); + }(); + if (!scan.errors.empty()) { + std::string msg = "scanner errors:\n"; + for (auto& e : scan.errors) msg += " " + e.format() + "\n"; + return std::unexpected(msg); + } + for (auto& w : scan.warnings) { + std::println(stderr, "warning: {}", w.format()); + } + + auto report = mcpp::modgraph::validate(scan.graph, *m, *root); + for (auto& w : report.warnings) { + if (w.path.empty()) std::println(stderr, "warning: {}", w.message); + else std::println(stderr, "warning: {}: {}", w.path.string(), w.message); + } + if (!report.ok()) { + std::string msg = "validation errors:\n"; + for (auto& e : report.errors) { + if (e.path.empty()) msg += " " + e.message + "\n"; + else msg += " " + e.path.string() + ": " + e.message + "\n"; + } + return std::unexpected(msg); + } + + bool needsStdModule = graph_or_targets_import_std(scan.graph, *m, *root); + if (needsStdModule && !tc->hasImportStd) { + return std::unexpected(std::format( + "source imports std but toolchain '{}' provides no std module source", + tc->label())); + } + + // Compute fingerprint (no lockfile in M1 → empty hash) + mcpp::toolchain::FingerprintInputs fpi; + fpi.toolchain = *tc; + fpi.cppStandard = m->package.standard; + fpi.compileFlags = canonical_compile_flags(*m) + + canonical_package_build_metadata(packages); + fpi.dependencyLockHash = ""; // M2 + fpi.stdBmiHash = ""; // updated after stdmod build (chicken/egg ok for M1) + auto fp = mcpp::toolchain::compute_fingerprint(fpi); + + // Pre-build std module only when the source graph actually imports it. + std::filesystem::path stdBmiPath; + std::filesystem::path stdObjectPath; + std::filesystem::path stdCompatBmiPath; + std::filesystem::path stdCompatObjectPath; + if (needsStdModule) { + auto sm = mcpp::toolchain::ensure_built( + *tc, fp.hex, m->package.standard, m->cppStandard.flag, + mcpp::platform::macos::deployment_target( + m->buildConfig.macosDeploymentTarget)); + if (!sm) return std::unexpected(sm.error().message); + stdBmiPath = sm->bmiPath; + stdObjectPath = sm->objectPath; + stdCompatBmiPath = sm->compatBmiPath; + stdCompatObjectPath = sm->compatObjectPath; + } + + if (print_fingerprint) { + std::println("Toolchain: {}", tc->label()); + std::println("Fingerprint: {}", fp.hex); + for (std::size_t i = 0; i < fp.parts.size(); ++i) { + std::println(" [{}] {}", i + 1, fp.parts[i]); + } + } + + BuildContext ctx; + ctx.manifest = *m; + ctx.tc = *tc; + ctx.fp = fp; + ctx.projectRoot= *root; + ctx.outputDir = target_dir(*tc, fp, *root); + ctx.stdBmi = stdBmiPath; + ctx.stdObject = stdObjectPath; + ctx.plan = mcpp::build::make_plan(*m, *tc, fp, scan.graph, report.topoOrder, + packages, *root, ctx.outputDir, + stdBmiPath, stdObjectPath); + ctx.plan.stdCompatBmiPath = stdCompatBmiPath; + ctx.plan.stdCompatObjectPath = stdCompatObjectPath; + + // Clang: discover clang-scan-deps for P1689 dyndep scanning. + if (mcpp::toolchain::is_clang(*tc)) { + if (auto sd = mcpp::toolchain::clang::find_scan_deps(*tc)) { + ctx.plan.scanDepsPath = *sd; + } + } + + // ─── M3.2: BMI cache stage / populate-task collection ───────────── + // For each version-based dep package (i.e. fetched from a registry, + // not a path dep), check the global BMI cache. If cached → stage into + // the project's target dir so ninja sees those outputs as up-to-date + // and skips them. If not → record a populate task for AFTER build. + // + // Path deps don't go through the cache: their sources can change at + // any time outside fingerprint awareness. + auto cfg2 = get_cfg(); + if (cfg2) { + std::error_code mkEc; + std::filesystem::create_directories(ctx.outputDir, mkEc); + auto usable_object_rel = [](const std::filesystem::path& rel) + -> std::optional + { + auto s = rel.generic_string(); + if (s.empty() || s == "." || s == ".." || s.starts_with("../")) { + return std::nullopt; + } + return s; + }; + auto object_cache_path = [&](const std::filesystem::path& objectPath) { + if (objectPath.is_absolute()) { + if (auto s = usable_object_rel( + objectPath.lexically_relative(ctx.outputDir / "obj"))) { + return *s; + } + } + if (auto s = usable_object_rel(objectPath.lexically_relative("obj"))) { + return *s; + } + return objectPath.filename().generic_string(); + }; + for (std::size_t i = 1; i < packages.size(); ++i) { // skip [0] = main + const auto& pkgRoot = packages[i]; + const auto* depIdent = i - 1 < dep_cache_identities.size() + ? &dep_cache_identities[i - 1] + : nullptr; + const auto fallbackName = pkgRoot.manifest.package.namespace_.empty() + ? pkgRoot.manifest.package.name + : std::format("{}.{}", pkgRoot.manifest.package.namespace_, + pkgRoot.manifest.package.name); + const auto& depName = depIdent ? depIdent->packageName : fallbackName; + const auto& depVer = depIdent && !depIdent->version.empty() + ? depIdent->version + : pkgRoot.manifest.package.version; + + // Find this dep's spec from the consumer manifest to know + // if it's path-based or version-based. + auto specIt = m->dependencies.find(depName); + // Path AND git deps bypass the BMI cache: their sources can + // change outside the fingerprint's awareness. + bool skipCache = (specIt != m->dependencies.end() && + (specIt->second.isPath() || specIt->second.isGit())); + if (specIt == m->dependencies.end()) { + auto devIt = m->devDependencies.find(depName); + if (devIt != m->devDependencies.end()) { + skipCache = devIt->second.isPath() || devIt->second.isGit(); + } + } + if (skipCache) continue; + + auto bmiT = mcpp::toolchain::bmi_traits(*tc); + mcpp::bmi_cache::CacheKey key { + .mcppHome = (*cfg2)->mcppHome, + .fingerprint = fp.hex, + .indexName = depIdent + ? depIdent->indexName + : (*cfg2)->defaultIndex, + .packageName = depName, + .version = depVer, + .bmiDirName = std::string(bmiT.bmiDir), + .manifestTag = std::string(bmiT.manifestPrefix), + }; + + // Compute the artifacts list from the build plan: every + // CompileUnit whose source lies under this dep's root contributes. + mcpp::bmi_cache::DepArtifacts arts; + for (auto& cu : ctx.plan.compileUnits) { + std::error_code ec; + auto rel = std::filesystem::relative(cu.source, pkgRoot.root, ec); + if (ec || rel.empty()) continue; + auto rels = rel.string(); + if (rels.starts_with("..")) continue; // not under depRoot + + if (cu.providesModule) { + std::string bmi; + for (char c : *cu.providesModule) + bmi.push_back(c == ':' ? '-' : c); + bmi += std::string(bmiT.bmiExt); + arts.bmiFiles.push_back(std::move(bmi)); + } + arts.objFiles.push_back(object_cache_path(cu.object)); + } + + if (mcpp::bmi_cache::is_cached(key)) { + auto staged = mcpp::bmi_cache::stage_into(key, ctx.outputDir); + if (staged) { + ctx.cachedDepLabels.push_back( + std::format("{} v{}", depName, depVer)); + continue; // skip populate task; it's already cached + } + // stage failed — fall through to recompile + repopulate + } + ctx.depsToPopulate.push_back({ std::move(key), std::move(arts) }); + } + } + // ────────────────────────────────────────────────────────────────── + + // Write/update mcpp.lock for any version-based deps that succeeded. + // Path deps are intentionally NOT locked — their source is local filesystem. + { + mcpp::lockfile::Lockfile lock; + lock.schemaVersion = 2; + + // Lock custom index shas from manifest [indices] section. + for (auto const& [idxName, spec] : m->indices) { + if (spec.is_local() || spec.is_builtin()) continue; + mcpp::lockfile::LockedIndex li; + li.name = idxName; + li.url = spec.url; + li.rev = spec.rev; // may be empty if not yet resolved + lock.indices.push_back(std::move(li)); + } + + for (auto const& [name, spec] : m->dependencies) { + if (spec.isPath()) continue; + mcpp::lockfile::LockedPackage lp; + lp.name = name; + if (spec.isGit()) { + auto gitIt = root_git_lock_identities.find(name); + lp.version = spec.gitRev; + if (gitIt == root_git_lock_identities.end()) { + lp.source = std::format("git+{}#{}={}", + spec.git, spec.gitRefKind, spec.gitRev); + std::hash hasher; + lp.hash = std::format("fnv1a:{:016x}", hasher(lp.source)); + } else { + lp.source = gitIt->second.source; + lp.hash = gitIt->second.hash; + } + } else { + lp.namespace_ = spec.namespace_.empty() + ? std::string{} + : spec.namespace_; + lp.version = spec.version; + // Use the namespace and resolved version as the source identifier. + // For custom indices, include the index name for traceability. + auto sourceIndex = lp.namespace_.empty() + ? std::string(mcpp::pm::kDefaultNamespace) + : lp.namespace_; + lp.source = std::format("index+{}@{}", sourceIndex, lp.version); + // Use a deterministic hash based on namespace + name + version. + // A future PR can replace this with a real content hash from the + // xpkg.lua's declared sha256 or from the install plan. + std::hash hasher; + auto hashInput = std::format("{}:{}@{}", sourceIndex, name, lp.version); + lp.hash = std::format("fnv1a:{:016x}", hasher(hashInput)); + } + lock.packages.push_back(std::move(lp)); + } + if (!lock.packages.empty() || !lock.indices.empty()) { + auto lockPath = *root / "mcpp.lock"; + (void)mcpp::lockfile::write(lock, lockPath); + } + } + + // Apply [runtime.] provider = "" overrides: prefer the + // named provider for matching capabilities (capability name prefix match). + // Warn if the named provider isn't in the dependency graph. + for (auto& [capKey, prov] : ctx.manifest.runtimeConfig.providerOverrides) { + bool found = false; + std::stable_partition(ctx.plan.runtimeProviders.begin(), + ctx.plan.runtimeProviders.end(), + [&](const auto& pr) { + bool match = pr.capability.rfind(capKey, 0) == 0 && pr.provider == prov; + found = found || match; + return match; + }); + if (!found) { + std::println(stderr, + "warning: [runtime.{}] provider = \"{}\" — no such provider in the " + "dependency graph for that capability", capKey, prov); + } + } + + // Capability-driven ABI enforcement: if any dependency declares an + // `abi:` capability, the resolved toolchain must satisfy it. (Toolchain + // is resolved before the dep graph, so this enforces/diagnoses rather than + // reselects — abi-driven reselection is a resolution-ordering follow-up.) + { + const std::string tcAbi = + ctx.tc.targetTriple.find("musl") != std::string::npos ? "musl" + : ctx.tc.stdlibId == "libc++" ? "libc++" + : ctx.tc.compiler == mcpp::toolchain::CompilerId::MSVC ? "msvc" + : "glibc"; + for (auto& cap : ctx.plan.runtimeCapabilities) { + if (cap.rfind("abi:", 0) != 0) continue; + std::string need = cap.substr(4); + if (need == tcAbi) continue; + std::string provider; + for (auto& [c, p] : ctx.plan.runtimeProviders) + if (c == cap) { provider = p; break; } + return std::unexpected(std::format( + "ABI mismatch: dependency '{}' requires abi={} but the resolved " + "toolchain '{}' is abi={}.\n" + " fix: `mcpp toolchain default <{}-compatible>` " + "(e.g. gcc@16.1.0 for glibc), or set [toolchain] in mcpp.toml.", + provider.empty() ? "?" : provider, need, ctx.tc.label(), tcAbi, need)); + } + } + + // Per-build resolution manifest artifact: a machine-readable record of the + // resolved plan (toolchain/abi, runtime closure, capabilities+providers, + // deps) written next to the build outputs. Same data as `mcpp why`; usable + // by CI/tooling. (capability -> plan, serialized.) + { + const std::string tcAbi = + ctx.tc.targetTriple.find("musl") != std::string::npos ? "musl" + : ctx.tc.stdlibId == "libc++" ? "libc++" + : ctx.tc.compiler == mcpp::toolchain::CompilerId::MSVC ? "msvc" + : "glibc"; + nlohmann::json j; + j["toolchain"] = { + {"spec", ctx.tc.label()}, {"abi", tcAbi}, + {"triple", ctx.tc.targetTriple}, {"stdlib", ctx.tc.stdlibId}, + }; + nlohmann::json dirs = nlohmann::json::array(); + for (auto& d : ctx.plan.runtimeLibraryDirs) dirs.push_back(d.string()); + nlohmann::json caps = nlohmann::json::array(); + for (auto& [cap, prov] : ctx.plan.runtimeProviders) + caps.push_back({{"capability", cap}, {"provider", prov}}); + j["runtime"] = { + {"library_dirs", dirs}, + {"dlopen_libs", ctx.plan.runtimeDlopenLibs}, + {"capabilities", caps}, + }; + std::error_code ec; + std::filesystem::create_directories(ctx.plan.outputDir, ec); + if (std::ofstream js(ctx.plan.outputDir / "resolution.json"); js) + js << j.dump(2) << "\n"; + } + + return ctx; +} + + +} // namespace mcpp::build diff --git a/src/cli.cppm b/src/cli.cppm index cc1aff9e..cdbc907a 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -1,12 +1,14 @@ -// mcpp.cli — top-level command dispatch. +// mcpp.cli — top-level command dispatch (and nothing else). // -// MVP commands: -// mcpp new -// mcpp build [--verbose] [--print-fingerprint] [--no-cache] -// mcpp run [target] [-- args...] -// mcpp clean [--bmi-cache] -// mcpp emit xpkg [--version V] [--output FILE] (M2) -// mcpp --help / mcpp --version +// The cli layer only parses arguments and routes: +// mcpp.cli.cmd_build / cmd_new / cmd_registry / cmd_cache / +// mcpp.cli.cmd_toolchain / cmd_publish / cmd_self (parse + route) +// mcpp.pm.commands (add / remove / update) +// Domain logic lives in its owning subsystem: mcpp.build.{prepare,execute}, +// mcpp.pm.index_management, mcpp.toolchain.lifecycle, mcpp.scaffold.create, +// mcpp.publish.pipeline, mcpp.pack.pipeline, mcpp.bmi_cache.maintenance, mcpp.doctor, +// mcpp.project, mcpp.fetcher.progress. +// See .agents/docs/2026-06-10-cli-modularization.md for the architecture. module; #include @@ -15,40 +17,18 @@ module; export module mcpp.cli; import std; -import mcpp.libs.json; -import mcpp.manifest; -import mcpp.modgraph.graph; -import mcpp.modgraph.scanner; -import mcpp.modgraph.validate; -import mcpp.toolchain.clang; -import mcpp.toolchain.detect; -import mcpp.toolchain.fingerprint; -import mcpp.toolchain.registry; -import mcpp.toolchain.stdmod; -import mcpp.build.plan; -import mcpp.build.backend; -import mcpp.build.ninja; -import mcpp.lockfile; -import mcpp.publish.xpkg_emit; -import mcpp.pack; -import mcpp.config; -import mcpp.xlings; -import mcpp.platform; -import mcpp.fetcher; -import mcpp.pm.resolver; // PR-R4: extracted from cli.cppm -import mcpp.pm.commands; // PR-R5: cmd_add / cmd_remove / cmd_update live here now -import mcpp.pm.index_spec; // IndexSpec for [indices] support -import mcpp.scaffold; // package-based project templates -import mcpp.pm.mangle; // Level 1 multi-version fallback (cross-major coexistence) -import mcpp.pm.compat; // 0.0.6: namespace field + dotted-name compat shims -import mcpp.pm.dep_spec; +import mcpplibs.cmdline; +import mcpp.cli.cmd_build; +import mcpp.cli.cmd_cache; +import mcpp.cli.cmd_new; +import mcpp.cli.cmd_publish; +import mcpp.cli.cmd_registry; +import mcpp.cli.cmd_self; +import mcpp.cli.cmd_toolchain; +import mcpp.pm.commands; +import mcpp.toolchain.fingerprint; // MCPP_VERSION import mcpp.ui; import mcpp.log; -import mcpp.fallback.install_integrity; -import mcpp.bmi_cache; -import mcpp.dyndep; -import mcpp.version_req; // SemVer constraint resolution -import mcpplibs.cmdline; // M6.1: dogfooded CLI parser export namespace mcpp::cli { @@ -56,15 +36,7 @@ int run(int argc, char** argv); } // namespace mcpp::cli -namespace mcpp::cli::detail { - -// ----- helpers ----- -// -// As of M6.1 phase 3, all CLI commands dispatch through a single -// `cmdline::App` declared in `run()` below. The previous per-command -// `cl::App` build + `parse_cmd_args(...)` double-parse is gone; each -// `cmd_*` now takes the already-parsed `ParsedArgs` and reads from it. -// `cmdline` handles `--help` / `--version` / unknown-option errors itself. +namespace mcpp::cli { // Custom top-level help. cmdline's auto-generated `print_help` is a fine // default but its layout (`USAGE:`, no command-specific blurbs) doesn't @@ -113,5697 +85,7 @@ void print_usage() { std::println("Docs: https://github.com/mcpp-community/mcpp/tree/main/docs"); } -// Locate mcpp.toml by walking upward from cwd. -std::optional find_manifest_root(std::filesystem::path start) { - auto p = std::filesystem::absolute(start); - while (true) { - if (std::filesystem::exists(p / "mcpp.toml")) return p; - auto parent = p.parent_path(); - if (parent == p) return std::nullopt; - p = parent; - } -} - -// Find the workspace root by walking upward from a member directory. -// Returns empty if no workspace root found. -std::filesystem::path find_workspace_root(const std::filesystem::path& memberRoot) { - auto p = memberRoot.parent_path(); - while (true) { - if (std::filesystem::exists(p / "mcpp.toml")) { - auto m = mcpp::manifest::load(p / "mcpp.toml"); - if (m && m->workspace.present) { - // Verify memberRoot is in members list - auto rel = std::filesystem::relative(memberRoot, p); - for (auto& member : m->workspace.members) { - if (rel == std::filesystem::path(member)) return p; - } - } - } - auto parent = p.parent_path(); - if (parent == p) break; - p = parent; - } - return {}; -} - -// Merge workspace.dependencies versions into a member's deps. -void merge_workspace_deps(mcpp::manifest::Manifest& member, - const mcpp::manifest::Manifest& workspace) { - auto merge_map = [&](std::map& deps) { - for (auto& [name, spec] : deps) { - if (!spec.inheritWorkspace) continue; - // Try exact key match first - auto it = workspace.workspace.dependencies.find(name); - if (it != workspace.workspace.dependencies.end()) { - spec.version = it->second.version; - spec.inheritWorkspace = false; - continue; - } - // Try short name for default-ns deps - auto shortIt = workspace.workspace.dependencies.find(spec.shortName); - if (shortIt != workspace.workspace.dependencies.end()) { - spec.version = shortIt->second.version; - spec.inheritWorkspace = false; - } - } - }; - merge_map(member.dependencies); - merge_map(member.devDependencies); - merge_map(member.buildDependencies); -} - -std::filesystem::path target_dir(const mcpp::toolchain::Toolchain& tc, - const mcpp::toolchain::Fingerprint& fp, - const std::filesystem::path& root) -{ - auto triple = tc.targetTriple.empty() ? std::string{"unknown"} : tc.targetTriple; - return root / "target" / triple / fp.hex; -} - -// ─── Toolchain version-spec helpers ────────────────────────────────── -// -// Partial versions: `mcpp toolchain install gcc 15` must match -// the latest installed/available 15.x.y, `gcc 15.1` matches the latest -// 15.1.y, etc. Accept either ` ` (two positionals) or `@` -// (one positional with `@`) — both forms are normalised here. - -// Split "X.Y.Z…" into integer components. A trailing "-musl" (or any other -// non-numeric tail) is dropped — the caller has already handled the libc -// flavour and we only care about the numeric prefix for matching. -std::vector parse_version_components(std::string_view s) { - std::vector out; - int cur = 0; - bool any = false; - for (char c : s) { - if (c >= '0' && c <= '9') { cur = cur * 10 + (c - '0'); any = true; } - else if (c == '.') { - if (any) { out.push_back(cur); cur = 0; any = false; } - else { out.clear(); break; } - } else { - break; // non-numeric tail (e.g. "-musl") - } - } - if (any) out.push_back(cur); - return out; -} - -// Pick the version from `available` that best matches `partial`: -// "" → highest version overall -// "15" → highest 15.X.Y -// "15.1" → highest 15.1.Y -// "15.1.0" → exact match (or empty if not present) -// Empty result = no match. -std::optional -resolve_version_match(std::string_view partial, - std::vector available) -{ - if (available.empty()) return std::nullopt; - auto want = parse_version_components(partial); - auto matches = [&](const std::vector& cand) { - if (want.size() > cand.size()) return false; - for (std::size_t i = 0; i < want.size(); ++i) - if (cand[i] != want[i]) return false; - return true; - }; - std::optional best; - std::vector bestVec; - for (auto& v : available) { - auto comps = parse_version_components(v); - if (comps.empty()) continue; - if (!matches(comps)) continue; - if (!best || std::lexicographical_compare( - bestVec.begin(), bestVec.end(), comps.begin(), comps.end())) - { - best = v; - bestVec = std::move(comps); - } - } - return best; -} - -// Enumerate installed `/xim-x-//` subdirs. -std::vector -list_installed_versions(const std::filesystem::path& pkgsDir, - std::string_view ximName) -{ - std::vector out; - auto root = pkgsDir / std::format("xim-x-{}", ximName); - std::error_code ec; - if (!std::filesystem::exists(root, ec)) return out; - for (auto& v : std::filesystem::directory_iterator(root, ec)) { - if (v.is_directory(ec)) out.push_back(v.path().filename().string()); - } - return out; -} - -// Look up available versions for `xim:` from the locally synced index. -// Falls back to an empty list silently — the caller will then either error -// out with a clear message or just keep the partial as-is. -// -// Index layout in mcpp's sandbox is two-tier: -// /data/xim-pkgindex/pkgs//.lua — primary -// /data/xim-index-repos//pkgs//.lua -// We scan both so a package living in either tier resolves. -std::vector -list_available_xpkg_versions(const mcpp::config::GlobalConfig& cfg, - std::string_view ximName) -{ - if (ximName.empty()) return {}; - std::string subdir(1, ximName[0]); - std::string fname = std::string(ximName) + ".lua"; - - auto try_load = [&](const std::filesystem::path& p) - -> std::optional> - { - std::error_code ec; - if (!std::filesystem::exists(p, ec)) return std::nullopt; - std::ifstream is(p); - std::string body((std::istreambuf_iterator(is)), {}); - return mcpp::manifest::list_xpkg_versions(body, "linux"); - }; - - auto data = cfg.xlingsHome() / "data"; - if (auto v = try_load(data / "xim-pkgindex" / "pkgs" / subdir / fname); v) - return std::move(*v); - - std::error_code ec; - auto repos = data / "xim-index-repos"; - if (std::filesystem::exists(repos, ec)) { - for (auto& repo : std::filesystem::directory_iterator(repos, ec)) { - auto cand = repo.path() / "pkgs" / subdir / fname; - if (auto v = try_load(cand); v) return std::move(*v); - } - } - return {}; -} - -// ─── Install-time progress display ─────────────────────────────────── -// -// xlings emits NDJSON events on stdout via `xlings interface install_packages -// --args ...` (see fetcher.cppm). The events we care about for UX are: -// -// {"kind":"data","dataKind":"download_progress","payload":{ -// "elapsedSec": 2.0, -// "files": [{"name":"...", "downloadedBytes":..., "totalBytes":..., "finished":bool, ...}], -// ... -// }} -// -// We parse the first file in the `files` array (xlings serializes the -// currently-active download first) and feed (current, total) to a -// ui::ProgressBar so the user sees a "Downloading [==== ] -// 45 MB / 110 MB" line. - -struct InstallProgressFile { - std::string name; - double downloaded = 0; - double total = 0; - bool started = false; - bool finished = false; -}; - -namespace { - -// Extract one `{ ... }` object starting at payload[*pos], moving *pos past -// the closing `}`. Returns the slice or empty when no object is here. -std::string_view scan_one_object(std::string_view payload, std::size_t* pos) { - auto p = *pos; - while (p < payload.size() && (payload[p] == ' ' || payload[p] == '\n')) ++p; - if (p >= payload.size() || payload[p] != '{') { *pos = p; return {}; } - auto start = p; - int depth = 0; - bool in_string = false; - for (; p < payload.size(); ++p) { - char c = payload[p]; - if (in_string) { - if (c == '\\' && p + 1 < payload.size()) { ++p; continue; } - if (c == '"') in_string = false; - continue; - } - if (c == '"') in_string = true; - else if (c == '{') ++depth; - else if (c == '}') { if (--depth == 0) { ++p; break; } } - } - *pos = p; - return payload.substr(start, (p == payload.size() ? p : p) - start); -} - -InstallProgressFile parse_one_install_file(std::string_view obj) { - auto get_str = [&](std::string_view key) -> std::string { - std::string n = std::format("\"{}\":\"", key); - auto q = obj.find(n); - if (q == std::string_view::npos) return ""; - q += n.size(); - std::string out; - while (q < obj.size() && obj[q] != '"') { - if (obj[q] == '\\' && q + 1 < obj.size()) { out.push_back(obj[q+1]); q += 2; continue; } - out.push_back(obj[q++]); - } - return out; - }; - auto get_num = [&](std::string_view key) -> double { - std::string n = std::format("\"{}\":", key); - auto q = obj.find(n); - if (q == std::string_view::npos) return 0; - q += n.size(); - auto e = q; - while (e < obj.size() - && (std::isdigit(static_cast(obj[e])) - || obj[e] == '.' || obj[e] == '-' || obj[e] == '+' - || obj[e] == 'e' || obj[e] == 'E')) ++e; - try { return std::stod(std::string(obj.substr(q, e - q))); } - catch (...) { return 0; } - }; - auto get_bool = [&](std::string_view key) -> bool { - std::string n = std::format("\"{}\":", key); - auto q = obj.find(n); - if (q == std::string_view::npos) return false; - q += n.size(); - return obj.size() - q >= 4 && obj.substr(q, 4) == "true"; - }; - - InstallProgressFile f; - f.name = get_str("name"); - f.downloaded = get_num("downloadedBytes"); - f.total = get_num("totalBytes"); - f.started = get_bool("started"); - f.finished = get_bool("finished"); - return f; -} - -} // namespace - -// Parse every entry in the payload's `files` array. xlings emits an -// array-of-files for download_progress events even when only one is -// active, and during multi-package installs (gcc → glibc / binutils / -// linux-headers / gcc-runtime / gcc) the order of entries shifts as -// each file starts and finishes. Reading just the first one would -// flicker between names and re-emit the static "Downloading " -// line every time the first slot rotates. -std::vector -parse_all_install_files(std::string_view payload) -{ - std::vector out; - constexpr std::string_view kKey{"\"files\":["}; - auto p = payload.find(kKey); - if (p == std::string_view::npos) return out; - p += kKey.size(); - while (p < payload.size()) { - while (p < payload.size() && (payload[p] == ' ' || payload[p] == '\n' - || payload[p] == ',')) ++p; - if (p >= payload.size() || payload[p] == ']') break; - if (payload[p] != '{') break; - auto obj = scan_one_object(payload, &p); - if (obj.empty()) break; - auto f = parse_one_install_file(obj); - if (!f.name.empty()) out.push_back(std::move(f)); - } - return out; -} - -// Pull a top-level numeric field out of a payload JSON string. Cheap; -// only used for `elapsedSec` which we trust to be a plain number. -double extract_payload_number(std::string_view payload, std::string_view key) { - std::string n = std::format("\"{}\":", key); - auto q = payload.find(n); - if (q == std::string_view::npos) return 0; - q += n.size(); - auto e = q; - while (e < payload.size() - && (std::isdigit(static_cast(payload[e])) - || payload[e] == '.' || payload[e] == '-' || payload[e] == '+' - || payload[e] == 'e' || payload[e] == 'E')) ++e; - try { return std::stod(std::string(payload.substr(q, e - q))); } - catch (...) { return 0; } -} - -// Build the PathContext used to shorten user-visible paths in status -// output. project_root may be empty (for verbs that don't need it). -mcpp::ui::PathContext make_path_ctx(const mcpp::config::GlobalConfig* cfg, - std::filesystem::path project_root = {}) -{ - mcpp::ui::PathContext ctx; - ctx.project_root = std::move(project_root); - if (cfg) ctx.mcpp_home = cfg->mcppHome; - if (auto* h = std::getenv("HOME"); h && *h) ctx.home = h; - return ctx; -} - -// Map a decoded NDJSON `download_progress` files[] snapshot onto the neutral -// `mcpp::ui::DownloadFile` the centralized renderer consumes. -template -std::vector to_ui_download_files(const std::vector& files) { - std::vector out; - out.reserve(files.size()); - for (auto& f : files) { - if constexpr (requires { f.downloadedBytes; }) { - out.push_back({ f.name, - static_cast(f.downloadedBytes), - static_cast(f.totalBytes), - f.started, f.finished }); - } else { - out.push_back({ f.name, - static_cast(f.downloaded), - static_cast(f.total), - f.started, f.finished }); - } - } - return out; -} - -// Adapter from `mcpp::config::BootstrapProgress` (xlings download_progress -// event) to the centralized download renderer. Used by load_or_init() during -// the one-time sandbox bootstrap (xim:patchelf, xim:ninja + transitive deps). -mcpp::config::BootstrapProgressCallback make_bootstrap_progress_callback() { - auto progress = std::make_shared(); - return [progress](const mcpp::config::BootstrapProgress& ev) { - auto files = to_ui_download_files(ev.files); - progress->update(files, ev.elapsedSec); - }; -} - -// EventHandler that forwards xlings `download_progress` events to the same -// centralized renderer. Used for toolchain, builtin-index and custom-index -// installs alike, so all three show identical UI. -struct CliInstallProgress : mcpp::fetcher::EventHandler { - mcpp::ui::DownloadProgress progress_; - - void on_data(const mcpp::fetcher::DataEvent& d) override { - if (d.dataKind != "download_progress") return; - auto files = parse_all_install_files(d.payloadJson); - if (files.empty()) return; - double elapsed = extract_payload_number(d.payloadJson, "elapsedSec"); - auto ui_files = to_ui_download_files(files); - progress_.update(ui_files, elapsed); - } - - void on_log(const mcpp::fetcher::LogEvent& e) override { - if (e.level == "error") - mcpp::log::error("xlings", e.message); - else if (e.level == "warn") - mcpp::log::warn("xlings", e.message); - else - mcpp::log::info("xlings", e.message); - mcpp::log::verbose("xlings", std::format("[{}] {}", e.level, e.message)); - } - - void on_error(const mcpp::fetcher::ErrorEvent& e) override { - mcpp::log::error("xlings", std::format("{}: {}", e.code, e.message)); - if (!e.hint.empty()) - mcpp::log::info("xlings", std::format("hint: {}", e.hint)); - } - - // progress_'s own destructor finishes the active bar. - ~CliInstallProgress() override = default; -}; - -// Compose a stable canonical compile-flags string for fingerprinting. -std::string canonical_compile_flags(const mcpp::manifest::Manifest& m) { - std::string s; - s += "-std="; s += m.package.standard; - s += " -fmodules"; - // macOS deployment target changes the effective compile triple - // (arm64-apple-macosxNN) — a std.pcm built for one target cannot be - // loaded by a TU compiled for another. Fold the resolved value - // (env override > [build] macos_deployment_target manifest default) - // into the fingerprint so switching targets rebuilds the BMI cache - // instead of dying with a module config mismatch. - // - // The built-in default floor (rustc-style) lives in the single - // resolver (platform::macos::deployment_target), so this rule, the - // flags and the std-module prebuild always agree — the 0.0.50-era - // attempt to inject a default here alone left the test build's - // std.pcm unstaged (import std failed wholesale on macos CI). - if constexpr (mcpp::platform::is_macos) { - auto dtv = mcpp::platform::macos::deployment_target( - m.buildConfig.macosDeploymentTarget); - if (!dtv.empty()) { - s += " macos_deployment_target="; - s += dtv; - } - } - if (!m.buildConfig.cStandard.empty()) { - s += " c_standard="; - s += m.buildConfig.cStandard; - } - for (auto const& flag : m.buildConfig.cflags) { - s += " cflag:"; - s += flag; - } - for (auto const& flag : m.buildConfig.cxxflags) { - s += " cxxflag:"; - s += flag; - } - for (auto const& flag : m.buildConfig.ldflags) { - s += " ldflag:"; - s += flag; - } - return s; -} - -std::string canonical_package_build_metadata( - const std::vector& packages) -{ - std::string s; - for (auto const& pkg : packages) { - s += "\npackage:"; - s += pkg.manifest.package.namespace_; - s += "/"; - s += pkg.manifest.package.name; - s += "@"; - s += pkg.manifest.package.version; - if (!pkg.manifest.buildConfig.cStandard.empty()) { - s += " c_standard="; - s += pkg.manifest.buildConfig.cStandard; - } - for (auto const& flag : pkg.manifest.buildConfig.cflags) { - s += " cflag:"; - s += flag; - } - for (auto const& flag : pkg.manifest.buildConfig.cxxflags) { - s += " cxxflag:"; - s += flag; - } - for (auto const& flag : pkg.manifest.buildConfig.ldflags) { - s += " ldflag:"; - s += flag; - } - if (pkg.usageResolved) { - for (auto const& dir : pkg.privateBuild.includeDirs) { - s += " private_include:"; - s += dir.generic_string(); - } - for (auto const& dir : pkg.publicUsage.includeDirs) { - s += " public_include:"; - s += dir.generic_string(); - } - } - for (auto const& [path, content] : pkg.manifest.buildConfig.generatedFiles) { - s += " genfile:"; - s += path.generic_string(); - s += "="; - s += content; - } - } - return s; -} - -std::expected -materialize_generated_files(const std::filesystem::path& root, - const mcpp::manifest::Manifest& manifest) -{ - for (auto const& [relPath, content] : manifest.buildConfig.generatedFiles) { - if (relPath.empty()) { - return std::unexpected("generated_files contains an empty path"); - } - if (relPath.is_absolute()) { - return std::unexpected(std::format( - "generated_files path '{}' must be relative", relPath.generic_string())); - } - auto const genericPath = relPath.generic_string(); - for (std::size_t begin = 0; begin <= genericPath.size();) { - auto const end = genericPath.find('/', begin); - auto const part = genericPath.substr(begin, end == std::string::npos - ? std::string::npos - : end - begin); - if (part == "..") { - return std::unexpected(std::format( - "generated_files path '{}' must not escape the package root", - relPath.generic_string())); - } - if (end == std::string::npos) { - break; - } - begin = end + 1; - } - - auto out = root / relPath.lexically_normal(); - std::error_code ec; - std::filesystem::create_directories(out.parent_path(), ec); - if (ec) { - return std::unexpected(std::format( - "cannot create directory for generated file '{}': {}", - out.string(), ec.message())); - } - - std::ofstream os(out, std::ios::binary); - if (!os) { - return std::unexpected(std::format( - "cannot write generated file '{}'", out.string())); - } - os << content; - if (!os) { - return std::unexpected(std::format( - "failed while writing generated file '{}'", out.string())); - } - } - return {}; -} - -bool is_std_module(std::string_view name) { - return name == "std" || name == "std.compat"; -} - -std::string trim_copy(std::string s) { - while (!s.empty() && std::isspace(static_cast(s.front()))) - s.erase(0, 1); - while (!s.empty() && std::isspace(static_cast(s.back()))) - s.pop_back(); - return s; -} - -bool source_file_imports_std(const std::filesystem::path& path) { - std::ifstream is(path); - if (!is) return false; - - std::string line; - while (std::getline(is, line)) { - line = trim_copy(std::move(line)); - std::size_t i = std::string::npos; - if (line.starts_with("import ")) { - i = 7; - } else if (line.starts_with("export import ")) { - i = 14; - } - if (i == std::string::npos) continue; - while (i < line.size() && std::isspace(static_cast(line[i]))) - ++i; - - std::string name; - while (i < line.size() - && (std::isalnum(static_cast(line[i])) - || line[i] == '_' || line[i] == '.' || line[i] == ':')) { - name.push_back(line[i]); - ++i; - } - if (is_std_module(name)) return true; - } - return false; -} - -bool graph_or_targets_import_std(const mcpp::modgraph::Graph& graph, - const mcpp::manifest::Manifest& manifest, - const std::filesystem::path& projectRoot) { - for (auto& u : graph.units) { - for (auto& req : u.requires_) { - if (is_std_module(req.logicalName)) - return true; - } - } - - // Some target entry files can be added to the plan after the package scan. - // Check them here so std BMI setup matches what make_plan will compile. - for (auto& t : manifest.targets) { - if (!t.main.empty() && source_file_imports_std(projectRoot / t.main)) - return true; - } - return false; -} - -// Run patchelf on every dynamic ELF in `dir` (recursively): -// - Set PT_INTERP to `loader` (the sandbox-local glibc loader). -// - Set RUNPATH to `rpath` (colon-separated list of sandbox lib dirs). -// Idempotent; skips static binaries and shared libs without PT_INTERP. -// -// TODO(xlings/libxpkg-upstream): xim 0.4.10's `elfpatch.auto({interpreter=...})` -// is supposed to do this in install hooks but currently scans 0 files for -// some packages (verified empirically: `binutils: elfpatch auto: 0 0 0`). -// Once the upstream legacy elfpatch path is fixed, this mcpp-side walker -// can be deleted. -void patchelf_walk(const std::filesystem::path& dir, - const std::filesystem::path& loader, - const std::string& rpath, - const std::filesystem::path& patchelfBin) -{ - if (!std::filesystem::exists(dir) || !std::filesystem::exists(patchelfBin)) - return; - std::error_code ec; - for (auto it = std::filesystem::recursive_directory_iterator(dir, ec); - it != std::filesystem::recursive_directory_iterator{}; it.increment(ec)) - { - if (ec) { ec.clear(); continue; } - if (!it->is_regular_file(ec)) continue; - auto path = it->path(); - // Skip non-ELF (cheap magic check) - std::ifstream is(path, std::ios::binary); - char m[4]{}; - is.read(m, 4); - if (!is || m[0] != 0x7f || m[1] != 'E' || m[2] != 'L' || m[3] != 'F') - continue; - is.close(); - // Probe PT_INTERP — skip static binaries (no interp). - auto probe = std::format("{} --print-interpreter {} 2>/dev/null", - mcpp::platform::shell::quote(patchelfBin.string()), - mcpp::platform::shell::quote(path.string())); - auto probeResult = mcpp::platform::process::capture(probe); - bool hasInterp = (probeResult.exit_code == 0 && !probeResult.output.empty()); - if (hasInterp) { - (void)mcpp::platform::process::run_silent(std::format( - "{} --set-interpreter {} {} 2>/dev/null", - mcpp::platform::shell::quote(patchelfBin.string()), - mcpp::platform::shell::quote(loader.string()), - mcpp::platform::shell::quote(path.string()))); - } - // Always set RUNPATH (works on .so too — they need to find deps). - if (!rpath.empty()) { - (void)mcpp::platform::process::run_silent(std::format( - "{} --set-rpath {} {} 2>/dev/null", - mcpp::platform::shell::quote(patchelfBin.string()), - mcpp::platform::shell::quote(rpath), - mcpp::platform::shell::quote(path.string()))); - } - } -} - -// xim bakes the installing user's XLINGS_HOME into gcc specs at install -// time (as `--dynamic-linker` and `-rpath`). When mcpp uses its own -// isolated sandbox (MCPP_HOME/registry/), the baked-in paths point to -// xlings' home, not mcpp's sandbox glibc — binaries would fail to exec. -// -// Mcpp does a post-install spec rewrite: -// - Dynamically detects the baked-in lib dir from the specs file -// - Replaces the dynamic-linker path with /ld-linux-x86-64.so.2 -// - Replaces the rpath with : -// Idempotent — skips if already pointing at the correct glibc. -// Extract the baked-in lib directory from a gcc specs file by finding -// the dynamic-linker path that ends with `/ld-linux-x86-64.so.2`. -// xim bakes the installing user's XLINGS_HOME into specs at install -// time, so the path varies per machine — we cannot hardcode it. -std::string detect_baked_lib_dir(const std::string& specsContent) { - constexpr std::string_view kLoader = "/ld-linux-x86-64.so.2"; - auto pos = specsContent.find(kLoader); - if (pos == std::string::npos) return ""; - // Walk backwards to find start of the absolute path - auto start = pos; - while (start > 0 && specsContent[start - 1] != ' ' - && specsContent[start - 1] != ':' - && specsContent[start - 1] != ';' - && specsContent[start - 1] != '\n') { - --start; - } - auto dir = specsContent.substr(start, pos - start); - // Sanity: must be absolute - if (dir.empty() || dir[0] != '/') return ""; - // Skip if it already points to the target glibc (no fixup needed) - return dir; -} - -void fixup_gcc_specs(const std::filesystem::path& gccPkgRoot, - const std::filesystem::path& glibcLibDir, - const std::filesystem::path& gccLibDir) -{ - auto specsParent = gccPkgRoot / "lib" / "gcc" / "x86_64-linux-gnu"; - if (!std::filesystem::exists(specsParent)) return; - - auto loaderReplacement = (glibcLibDir / "ld-linux-x86-64.so.2").string(); - auto rpathReplacement = std::format("{}:{}", - glibcLibDir.string(), - gccLibDir.string()); - - auto replace_all = [](std::string& s, std::string_view needle, - std::string_view rep) - { - for (std::size_t pos = 0; - (pos = s.find(needle, pos)) != std::string::npos;) { - s.replace(pos, needle.size(), rep); - pos += rep.size(); - } - }; - - for (auto& sub : std::filesystem::directory_iterator(specsParent)) { - auto specs = sub.path() / "specs"; - if (!std::filesystem::exists(specs)) continue; - - std::ifstream is(specs); - std::stringstream ss; ss << is.rdbuf(); - std::string content = ss.str(); - - auto bakedDir = detect_baked_lib_dir(content); - if (bakedDir.empty()) continue; - // Already pointing at the right place — no fixup needed. - if (bakedDir == glibcLibDir.string()) continue; - - auto bakedLoader = bakedDir + "/ld-linux-x86-64.so.2"; - - // Order matters: replace the full loader file path first so the - // shorter dir pattern doesn't eat its prefix. - replace_all(content, bakedLoader, loaderReplacement); - replace_all(content, bakedDir, rpathReplacement); - - std::ofstream os(specs); - os << content; - } -} - -// Rewrite clang++.cfg paths after the LLVM payload has been copied to the -// mcpp sandbox. The cfg was authored by xlings at install time and contains -// absolute paths pointing to ~/.xlings/. We rewrite them to point to the -// actual payload location + sibling xpkgs (glibc, linux-headers). -void fixup_clang_cfg(const std::filesystem::path& payloadRoot, - const std::filesystem::path& glibcLibDir) { - for (auto cfgName : {"clang++.cfg", "clang.cfg"}) { - auto cfgPath = payloadRoot / "bin" / cfgName; - if (!std::filesystem::exists(cfgPath)) continue; - - std::ifstream is(cfgPath); - std::stringstream ss; ss << is.rdbuf(); - std::string content = ss.str(); - is.close(); - - auto llvmRoot = payloadRoot; - auto replace_line_prefix = [&](std::string& s, std::string_view prefix, - const std::string& newValue) { - std::istringstream lines(s); - std::string result, line; - while (std::getline(lines, line)) { - if (line.starts_with(prefix)) { - result += std::string(prefix) + newValue + '\n'; - } else { - result += line + '\n'; - } - } - s = result; - }; - - // Rewrite --sysroot to remove (mcpp provides this explicitly). - // Rewrite -isystem to point to payload's libc++ headers. - // Rewrite -L and -rpath to point to payload's lib dir. - // Rewrite dynamic-linker to use glibc payload's ld-linux. - std::istringstream lines(content); - std::string result, line; - while (std::getline(lines, line)) { - if (line.starts_with("--sysroot=")) { - // Remove — mcpp provides sysroot via payload paths. - continue; - } - if (line.starts_with("-isystem ")) { - auto oldPath = line.substr(9); - if (oldPath.find("include/c++/v1") != std::string::npos) { - auto relative = oldPath.substr(oldPath.find("include/c++/v1")); - result += "-isystem " + (llvmRoot / relative).string() + '\n'; - continue; - } - if (oldPath.find("include/x86_64") != std::string::npos || - oldPath.find("include/aarch64") != std::string::npos) { - // Target-specific libc++ include. - auto includePos = oldPath.find("include/"); - auto relative = oldPath.substr(includePos); - result += "-isystem " + (llvmRoot / relative).string() + '\n'; - continue; - } - } - if (line.starts_with("-L")) { - auto oldPath = line.substr(2); - if (oldPath.find("lib/x86_64") != std::string::npos || - oldPath.find("lib/aarch64") != std::string::npos) { - auto libPos = oldPath.find("lib/"); - auto relative = oldPath.substr(libPos); - result += "-L" + (llvmRoot / relative).string() + '\n'; - continue; - } - } - if (line.starts_with("-Wl,-rpath,")) { - auto oldPath = line.substr(11); - // Rpath for LLVM lib dir - if (oldPath.find("lib/x86_64") != std::string::npos || - oldPath.find("lib/aarch64") != std::string::npos) { - auto libPos = oldPath.find("lib/"); - auto relative = oldPath.substr(libPos); - result += "-Wl,-rpath," + (llvmRoot / relative).string() + '\n'; - continue; - } - // Rpath for subos/glibc — rewrite to glibc payload. - if (!glibcLibDir.empty()) { - auto parentDir = std::filesystem::path(oldPath).parent_path(); - // subos rpath lines like -Wl,-rpath,/lib - if (oldPath.find("subos") != std::string::npos) { - result += "-Wl,-rpath," + glibcLibDir.string() + '\n'; - continue; - } - } - } - if (line.starts_with("-Wl,--dynamic-linker=")) { - // Rewrite to glibc payload's ld-linux. - if (!glibcLibDir.empty()) { - result += "-Wl,--dynamic-linker=" + - (glibcLibDir / "ld-linux-x86-64.so.2").string() + '\n'; - continue; - } - } - if (line.starts_with("-Wl,--enable-new-dtags,-rpath,")) { - if (!glibcLibDir.empty()) { - result += "-Wl,--enable-new-dtags,-rpath," + glibcLibDir.string() + '\n'; - continue; - } - } - if (line.starts_with("-Wl,-rpath-link,")) { - if (!glibcLibDir.empty()) { - result += "-Wl,-rpath-link," + glibcLibDir.string() + '\n'; - continue; - } - } - result += line + '\n'; - } - - // Remove trailing newline - while (!result.empty() && result.back() == '\n') result.pop_back(); - result += '\n'; - - std::ofstream os(cfgPath); - os << result; - } -} - -// Post-install fixup for a freshly-installed GNU gcc payload: patchelf -// PT_INTERP/RUNPATH for gcc/binutils binaries + linker-specs wiring against -// the sandbox glibc. ONE pipeline shared by `mcpp toolchain install` and the -// first-run auto-install (the latter previously skipped this, leaving a -// fresh-sandbox glibc gcc unable to find the C library: stdlib.h not found). -void gcc_post_install_fixup(const mcpp::config::GlobalConfig& cfg, - const std::filesystem::path& payloadRoot) { - // Ownership guard: payloads inherited via symlink from another MCPP_HOME - // are not ours to patch — their owner already ran the fixup, and patching - // through the symlink would rewrite the canonical files against OUR - // (possibly ephemeral) paths, bricking the owner's toolchain. - { - std::error_code ec; - auto canonicalRoot = std::filesystem::weakly_canonical(payloadRoot, ec); - auto homeRegistry = std::filesystem::weakly_canonical(cfg.registryDir, ec); - if (!ec && !canonicalRoot.string().starts_with(homeRegistry.string())) { - mcpp::log::verbose("toolchain", std::format( - "skip gcc fixup: payload '{}' resolves outside this home ('{}') — " - "inherited payload, owner is responsible for its fixup", - payloadRoot.string(), canonicalRoot.string())); - return; - } - } - auto xlEnv = mcpp::config::make_xlings_env(cfg); - auto glibcRoot = mcpp::xlings::paths::xim_tool_root(xlEnv, "glibc"); - std::filesystem::path glibcLibDir; - if (std::filesystem::exists(glibcRoot)) { - for (auto& v : std::filesystem::directory_iterator(glibcRoot)) { - auto candidate = v.path() / "lib64"; - if (std::filesystem::exists(candidate / "ld-linux-x86-64.so.2")) { - glibcLibDir = candidate; - break; - } - } - } - auto gccLibDir = payloadRoot / "lib64"; - auto patchelfBin = mcpp::xlings::paths::xim_tool(xlEnv, "patchelf", - mcpp::xlings::pinned::kPatchelfVersion) / "bin" / "patchelf"; - - if (!glibcLibDir.empty() && std::filesystem::exists(gccLibDir) - && std::filesystem::exists(patchelfBin)) - { - auto loader = glibcLibDir / "ld-linux-x86-64.so.2"; - auto rpath = std::format("{}:{}", - glibcLibDir.string(), gccLibDir.string()); - - mcpp::log::verbose("toolchain", std::format( - "gcc fixup: patchelf_walk rpath='{}'", rpath)); - auto binutilsRoot = mcpp::xlings::paths::xim_tool_root(xlEnv, "binutils"); - if (std::filesystem::exists(binutilsRoot)) { - for (auto& v : std::filesystem::directory_iterator(binutilsRoot)) - patchelf_walk(v.path(), loader, rpath, patchelfBin); - } - patchelf_walk(payloadRoot, loader, rpath, patchelfBin); - - mcpp::log::verbose("toolchain", "gcc fixup: fixup_gcc_specs"); - fixup_gcc_specs(payloadRoot, glibcLibDir, gccLibDir); - } else { - mcpp::ui::warning( - "could not locate sandbox glibc/gcc/patchelf paths; " - "gcc-built binaries may have unresolved PT_INTERP/RUNPATH"); - } -} - -// SemVer resolution: a version spec is a "constraint" (vs. exact literal) if -// it starts with one of `^~><=` or contains a comma (multi-part), or is `*` -// or empty. Bare `1.2.3` is treated as exact for back-compat with pre-SemVer -// pinning workflows; users opt into resolution by writing `^1.2.3` etc. -// `is_version_constraint`, `kXpkgPlatform` and `resolve_semver` have moved -// to `mcpp.pm.resolver` (PR-R4 — see -// `.agents/docs/2026-05-08-pm-subsystem-architecture.md`). Call sites -// below reference the `mcpp::pm::` qualified names directly. - -// --- Commands --- - -// ─── Package-based templates (design v2: multi-level --template) ────── -// -// Resolve SPEC's package@version through the index, ensure the package -// sources are installed (same cache as dependencies), and return the -// package root (the directory containing mcpp.toml). -struct FetchedTemplatePackage { - std::filesystem::path root; - std::string name; // short package name (e.g. "imgui") - std::string version; // resolved exact version -}; - -std::expected -fetch_template_package(const mcpp::scaffold::TemplateSpec& spec) { - auto cfg = mcpp::config::load_or_init(/*quiet=*/false, - make_bootstrap_progress_callback()); - if (!cfg) return std::unexpected(cfg.error().message); - mcpp::pm::Fetcher fetcher(*cfg); - - // Namespace candidates mirror dependency lookup: index root first, - // then the compat namespace. - std::optional lua; - for (std::string cand : {std::string{}, std::string{"compat"}}) { - if (auto l = fetcher.read_xpkg_lua(cand, spec.pkg)) { - lua = std::move(*l); - break; - } - } - if (!lua) { - return std::unexpected(std::format( - "template package '{}' not found in the index " - "(check the name, or run `mcpp index update`)", spec.pkg)); - } - - // The filename hit alone does not carry the namespace: a bare spec - // like "llmapi" finds pkgs/l/llmapi.lua even though the descriptor - // declares `namespace = "mcpplibs"`, and xlings resolves install - // targets by the descriptor's qualified name. Derive the structured - // (namespace, shortName) from the descriptor fields. - auto coords = mcpp::pm::compat::descriptor_coordinates( - spec.pkg, - mcpp::manifest::extract_xpkg_namespace(*lua), - mcpp::manifest::extract_xpkg_name(*lua)); - const std::string& ns = coords.namespace_; - const std::string& shortName = coords.shortName; - - std::string version = spec.version; - if (version.empty()) { - auto v = mcpp::pm::resolve_semver(ns, shortName, "*", fetcher); - if (!v) return std::unexpected(v.error()); - version = *v; - } - - auto installed = fetcher.install_path(ns, shortName, version); - if (!installed) { - auto fq = ns.empty() ? shortName : std::format("{}.{}", ns, shortName); - mcpp::ui::info("Downloading", std::format("{} v{}", fq, version)); - CliInstallProgress progress; - std::vector targets{ std::format("{}@{}", fq, version) }; - auto r = fetcher.install(targets, &progress); - if (!r) return std::unexpected(std::format( - "fetch '{}@{}': {}", fq, version, r.error().message)); - if (r->exitCode != 0) return std::unexpected(std::format( - "fetch '{}@{}' failed (exit {})", fq, version, r->exitCode)); - installed = fetcher.install_path(ns, shortName, version); - if (!installed) return std::unexpected(std::format( - "package '{}@{}' install path missing after fetch", fq, version)); - } - - // Package root = the directory holding mcpp.toml (tarballs usually wrap - // everything in a single top-level directory). - std::filesystem::path root = *installed; - if (!std::filesystem::exists(root / "mcpp.toml")) { - std::error_code ec; - for (auto& e : std::filesystem::directory_iterator(root, ec)) { - if (e.is_directory() - && std::filesystem::exists(e.path() / "mcpp.toml")) { - root = e.path(); - break; - } - } - } - if (!std::filesystem::exists(root / "mcpp.toml")) { - return std::unexpected(std::format( - "package '{}@{}' has no mcpp.toml", spec.pkg, version)); - } - return FetchedTemplatePackage{root, shortName, version}; -} - -void print_template_listing(const FetchedTemplatePackage& pkg, - const std::vector& entries) { - std::println("Templates in {}@{}:", pkg.name, pkg.version); - for (auto& t : entries) { - std::println(" {:<14}{}{}", t.name, - t.meta.isDefault ? "(default) " : " ", - t.meta.description); - } - std::println(""); - std::println("usage: mcpp new --template {}[@ver][: