[[!meta date="2026-10-28 12:00"]] [[!tag nix libnix cargo rust cargo+libnix]] [[!series libnix]] [[!summary cargo libnix 1.99.0+v2 and 1.87.0+v5: git dependencies without nix-prefetch-git, faster evaluation, cargo:: build scripts and GC roots that clean up]] [[!title libnix rust-1.99.0+v2 release]] [[!img posts/libnix/libnix-rust-logo.png class="noFancy" style="float: right" width="200px"]] # motivation this is a short status update on `cargo (libnix)` covering what changed since the `libnix-1.99.0+v1` release a week ago. v1 was about following upstream to rust 1.99, v2 is about the daily use: fewer downloads, less waiting on every build, build scripts with the new `cargo::` syntax, and a nix store that doesn't fill up. `libnix-1.87.0+v5` was released together with it and carries the same changes as v2, ported to the 1.87 line. so everything below applies to both, the few things only 1.87 got are listed at the end. the code can be found here: # improvements ## libnix release in `cargo --version` (#30) the first thing to check in a bug report is which cargo someone is running. the release is now part of the version (`LIBNIX_RELEASE`): ``` $ cargo --version cargo 1.99.0 +libnix v2 $ cargo -vV | grep libnix libnix: 1.99.0+v2 ``` ## git dependencies from cargo's checkout (#10) cargo already clones a git dependency to resolve the unit graph (`~/.cargo/git/checkouts//`, submodules included). on top of that, nix mode ran `nix-prefetch-git` for every git unit on every build, only to learn the sha256 for `pkgs.fetchgit`. serde as a `[patch.crates-io]` git dependency was cloned 3 times per build, and a no-op build needed the network. it turns out the NAR hash of cargo's checkout without `.git` and `.cargo-ok` (at every depth, cargo writes both into submodules too) is exactly the hash of the `fetchgit` output with `fetchSubmodules`. so cargo computes it in Rust, derives the fixed-output path and adds its checkout with `nix-store --add-fixed` if the path is missing. the units get a plain: ```nix pkgs.fetchgit { name; url; rev; hash; fetchSubmodules = true; } ``` the repository is downloaded once, by cargo, and ends up in the nix store. the exported build system (`write-nix-buildsystem`) still works on a machine without that path, `fetchgit` just fetches it there.
project no-op before no-op now
itoa (git patch) 3.8 s 2.7 s
serde (git patch, build script) 8.3 s, 3 clones 2.9 s
libz-sys (2 submodules) – 3.2 s
vendored git sources (no checkout of cargo's, e.g. in `test.nix`) still use `nix-prefetch-git`. a side finding: on nixpkgs 26.05 `fetchgit` with submodules isn't deterministic, a detached `git maintenance` raced the removal of `.git`. the generated code works around that, so git sources hash the same every time. ## one shared src for the path crates every path crate had its own `builtins.filterSource … project_root`, so evaluation walked, hashed and copied the whole working tree once per unit (~1.6 s each for cargo's tree, 12 units). the filtered tree is bound once now as `fn.project_src`, and the units use `src = fn.project_src;`. cargo's own build: warm `nix-instantiate` 39 s → 10 s. `.git` is left out by default: it changes with every commit (its index even with a plain `git status`), so having it in the src rebuilt all path crates for nothing. `Cargo.dependencies.nix` adjusts the defaults with top-level names of the workspace: ```nix project_src.include = [ ".git" ]; # back in, e.g. for `git log` in build.rs project_src.exclude = [ "docs" ]; # also out, changes there don't rebuild ``` cargo itself includes `.git`, its `build.rs` runs `git log` for the commit hash of `cargo -vV`. ## no absolute path in the build script runs the build script run derivations contained the absolute path of the output directory, so building the same project from a different directory (or with another `CARGO_TARGET_DIR`) rebuilt them and everything after them. that also made v1's README edits more expensive than needed:
change v1 v2
bat, README edit run, bat, bat (bin) rebuilt, 21 s only bat (build.rs run), 10 s
cargo, README edit run, the 9 path crates, cargo (bin) only cargo (build.rs run), 16 s
the first build after the update rebuilds the build script runs and their dependents once. ## faster generation and evaluation four smaller changes, all on the path every build takes: * **evaluate once**: `nix-instantiate` calls the caller (a function) once per `-A`, so `-A target -A cargo-libnix-retained-paths-…` evaluated the whole build system twice. an expression that imports it once returns both, same drvs, same GC roots. bat: 3.4 s → 2.0 s, a small project 2.0–2.5 s → 1.3 s * **no legacy fingerprints**: nix decides what to build, so nix mode doesn't compute cargo's fingerprints anymore. that skips the mtime scan of path packages with a build script, the most expensive part of the generation: `write-nix-buildsystem` on cargo (432 units) ~1230 → ~880 ms warm, 2.7 → 0.9 s cold. it also fixes units missing from the generated build system after a legacy build in the same target dir * **handlebars templates parsed once** instead of once per unit (~1700 times for a big workspace) * **the lib of a build script run from a map** instead of a search per unit: generation on cargo itself ~736 → ~596 ms ## timings after every build every nix-mode build ends with a status line now, so you see where the time goes: ``` Finished 0 built, 0 fetched, 2 fresh, 0 failed, 0 skipped in 2.8s Status ● generation 0.0s │ ● evaluation 2.4s │ ● build 0.4s │ ● other 0.1s │ Σ total 2.9s ``` * **generation**: writing the nix files * **evaluation**: `nix-instantiate` * **build**: `nix build`, compiling or checking the store * **other**: the rest, resolving the unit graph, the GC roots ## build-rs-libnix 0.1.14 the known issue of v1: build-rs-libnix 0.1.12 ignored the `cargo::` syntax (rust >= 1.77) with a warning. the outlook promised 0.1.13, it became 0.1.14, which parses `cargo::` next to `cargo:`. the first real user was blake3's `cargo::rustc-cfg`, which got lost before. the 1.87 and the 1.99 line pin the same version. ## GC roots that clean up `cargo (libnix)` keeps the outputs of a build alive with GC roots below `target//nix/gc/`, one slot per selection, and deletes what the previous build used but this one doesn't. that left two holes: * **failed or interrupted builds**: their outputs had no root and no list, so nobody deleted them. nix only removes them with a full `nix-collect-garbage` * **paths still in use**: when cargo builds its successor, the old cargo binary is running and can't be deleted, and after that it was forgotten each slot now has a file `built`: before every build, cargo writes the outputs it is about to build into it. after the next successful build, everything that was in `built` or in the old list but isn't in the new one is deleted; what is still in use stays in `built` for the next attempt. `built` is no GC root, a `nix-collect-garbage` in between still works as before. paths another slot uses are never touched, and build-rs-libnix is retained when there are build script runs. that matters because a store fills up fast: every compile of a big crate is a complete new output (`libcargo.rlib` alone is 572 MB). cleaning up the leftovers of the test runs freed 8.7 GB on my machine. `tests-libnix/gc/run.sh` tests it: cargo builds itself twice, and the second build deletes the first one. ## flake the releases are served by `master-libnix`: ```bash nix shell github:nixcloud/cargo/master-libnix cargo --version # cargo 1.99.0 +libnix v2 cargo +1.87 --version # cargo 1.87.0 +libnix v5 ``` the fixed releases are packages of their own (`cargo-libnix-1_99_0_plus_v2`, `cargo-libnix-1_87_0_plus_v5`, the older ones stay). on the branches, v2 and v5 are the default packages and the generators of `cargo-libnix-git-head-via-ifd`. ## regression tests * **1.99 v2**: all 31 projects of `test.nix` build and run with rustc 1.99.0, and cargo builds itself with `CARGO_BACKEND=nix` * **1.87 v5**: 30 of 31, union needs a nightly rustc ## what only 1.87 v5 got the 1.87 line also catches up with things the 1.99 line already had in v1: * the library closure is computed with `builtins.genericClosure`: union's ~1700 units evaluated for over 70 minutes before, now they fail after 12 minutes at the expected nightly `#![feature]` * a pinned rustc 1.87.0 without `rust-src` (which ended up in the runtime closure of every binary) * build-rs-libnix moved to its own repository, the unused crates.io dependency is gone # known issue a `nix-collect-garbage` running while cargo evaluates can break the build. `nix-instantiate` doesn't add temporary GC roots for `fetchTarball` cache hits, so a GC in that window deletes for example the nixpkgs source, and the build fails with a missing path. it happened once during the regression tests. running the build again fixes it, a proper fix is being discussed. # outlook: minimal sources for workspaces (#3, #20) the next big topic is faster iteration, and the branch `rust-1.99.0-libnix-47_minimal_sources_for_workspaces` has a first working version. the problem: every derivation of a workspace crate has the whole workspace as its src. editing a README, a test or another crate's file changes that src, so every crate of the workspace rebuilds. `project_src.exclude` helps a bit, but it can't know which crate reads which file. rustc does know: it writes the files a crate reads to the dep-info file (`.d`) in the unit's output. so a crate gets only those files as its src, and cargo keeps the list with a sha256 per file in `target//nix/src-lists`. getting the list must not cost a second compile: * a crate without a list, with a listed file changed, or depending on such a crate is built once with all files, and the `.d` of that output becomes its pending list * a no-op rebuilds nothing, since the all-files build is still cached * the first time something else changes, the crate would be rebuilt anyway, so it is built with its list instead. from then on changes outside its list don't rebuild it a file read without rustc knowing (a proc macro using `std::fs` instead of `include_bytes!`) would be missing from the list. so the first build with the list is compared with the one with all files, and a crate whose output differs keeps all files, with a warning. `[package.metadata.libnix] src = "full"` opts out. with the experimental nix feature `ca-derivations`, the build script runs of the workspace are content-addressed (#20): a run that produces the same output as before doesn't make the crates after it dirty. without it, the crates of a package with a build script keep the whole workspace. measured on atuin and `just` with that branch (`Status` line, `Σ total`):
step atuin just
no-op 10.9 s 3.6 s
editing a lib (9 units in atuin, 2 in just, once each) 49.8 s 16.0 s
next README edit (the switch to the lists, once) 60.5 s 30.5 s
README edits after that (only the build script runs) 13.4 s 4.7 s
and on cargo itself:
step total generation evaluation build other
a new file in the repo root (lib and bin switch to their lists, compiled once) 79 s 6.9 s 4.0 s 59.5 s 8.8 s
changing that file again (only the build script run reruns) 12.7 s 3.2 s 3.7 s 3.7 s 2.2 s
with v2 every one of these README or repo-root edits rebuilds the workspace's crates (for cargo the lib and the bin, ~60 s of build each time); with the branch it's the build script runs only, after the one switch. I didn't run the same steps side by side with v2, so these numbers show the branch, not an exact v2 comparison. what's still missing: * the fixed cost of every run: 2 s for a small project, 10–13 s for a 430-unit workspace like cargo, even when nothing compiles. reusing the generated build system when nothing changed, evaluating only what changed, and moving the GC work out of the critical path would bring that down to a few seconds * incremental compilation: each compile is a fresh derivation in the sandbox, so editing cargo's lib recompiles it from scratch (~60 s). that comes from nix itself * a crate after a content-addressed build script run compiles twice the one time it switches, when nothing but the run changed the branch passes all 31 regression test projects and will be part of a later release. # summary with v2 `cargo (libnix)` downloads git dependencies once, evaluates faster, understands `cargo::` build scripts, tells you its release and cleans up after failed builds. `libnix-1.87.0+v5` has the same changes for the 1.87 line.
if you like this work, your support is needed! * **consider a star** at to show your support. * **comments** are welcome in * if you're interested in **supporting this development**, please reach out to me, see for details.