[[!meta date="2024-05-10 16:53"]] [[!tag nix nixos libnix]] [[!series libnix]] [[!summary reviewing volth's nix windows port from 2020]] [[!img posts/libnix/Nix_snowflake_windows.svg class="noFancy" style="float: right" width="200px"]] # motivation status of **native windows nix using MinGW** from my series [libnix](https://lastlog.de/blog/timeline.html?filter=series::libnix). [last post](libnix_mingw_status.html) we were looking into nix on windows from the *nix master branch* which uses MinGW. **in this post we'll focus on volth's work running nix on windows back in the year 2020.** # introduction [volth](https://discourse.nixos.org/u/volth) *ported nix to windows* in ~2020. using `nix-build` he was able to build **boost 1.74** using a **msvc 2019 toolchain**: ```cmd \bin\nix-build.exe --keep-failed --option system x86_64-windows -o x86_64-boost -E "with (import { }); pkgsMsvc2019.boost174.override{ staticRuntime=true; static=true; }" ``` sadly it wasn't merged into upstream nix and i (qknight) was never able to run/build his code. at least we got the code, so let's what he did! ## nix windows branch * branch diff (all commits combined): * ## nixpkgs-windows * (called **nixpkgs-windows** in the source) branch diff (all commits combined): * ## volth's windows nix capabilities * **build nix 2.1.3 from native windows** using mingw+meson based toolchain * a [single user setup](https://nixos.org/manual/nix/stable/installation/installing-binary#single-user-installation) -> no nix-daemon implementation * probably using **visual studio** for development * probably used **cmd** (build-meson-32-mt.cmd) because for powershell this would be called (build-meson-32-mt.ps) * **featured msvc and mingw compilers** * **john ericson added meson build system support for nix** (replacing autotools), let's have a closer look at this: there have been proposals for other build systems over the years: * my attempt of autotools replacement using **CMake** in nix - early 2011 * autotools replacement using **CMake** - PR from mid 2017 * **build system discussions** - from late 2018 * **PR: Port Nix to Meson** - late 2019 * autotools replacement using **Meson** - commits from ~2020 ### nix builtins for external programs * `runProgram()` was replaced by `runProgramGetStdout()` which could call `git`, `hg`, `tar`, `unzip`, `xz`, `gzip`, `tr` * `builtins.fetchGit` -> src/libexpr/primops/fetchGit.cc * `builtins.fetchMercurial` -> src/libexpr/primops/fetchMercurial.cc * `builtins.fetchurl` / `nix-prefetch-url` -> src/nix-prefetch-url/nix-prefetch-url.cc - calls `unzip` / `tar` * origin of the tools `git`, `hg`, `tar`, `unzip` is unclear: i guess some are from `cygwin` while others are form `mingw64` installation. ### supported nix commands looking at the build system, checking the the source files for patches i conclude: working: * [x] nix-prefetch-url * [x] nix-instantiate * [x] nix-build * [x] nix-store not working: * [ ] nix-daemon * [ ] nix-channel * [ ] nix-shell * [ ] nix-collect-garbage * [ ] nix-env * [ ] nix-hash * [ ] nix-copy-closure working `nix` subcommands: * [x] run * [x] nar cat * [x] nar ls * [x] nar dump-path * [x] hash * [x] log * [x] search * [x] why-depends * [x] store ping * [x] store verify not working `nix` subcommands * [ ] repl * [ ] store gc **note:** this list is incomplete but still gives a broad overview. for instance, it seems that `nix build` was not supported, but the program `nix-build` was. ### stdenv volth maintained these envs: * [pkgs/stdenv/default.nix](https://github.com/nix-windows/nixpkgs/blob/windows/pkgs/stdenv/default.nix) * [pkgs/stdenv/windows/mingw.nix](https://github.com/nix-windows/nixpkgs/blob/windows/pkgs/stdenv/windows/mingw.nix) * [pkgs/development/mingw-modules/msys-pacman-x86_64.nix](https://github.com/nix-windows/nixpkgs/blob/windows/pkgs/development/mingw-modules/msys-pacman-x86_64.nix) * [pkgs/development/mingw-modules/mingw-pacman-x86_64.nix](https://github.com/nix-windows/nixpkgs/blob/windows/pkgs/development/mingw-modules/mingw-pacman-x86_64.nix) those are used in the file **build-meson-32-mt.cmd** and supposedly buildable: * pkgsMsvc2019.stdenv.cc * pkgsMsvc2019.boost174 * pkgsMsvc2019.openssl * pkgsMsvc2019.xz * pkgsMsvc2019.bzip2 * pkgsMsvc2019.curl * pkgsMsvc2019.sqlite * msysPacman.flex * msysPacman.bison * mingwPacman.meson however: using these *stdenvs* one could attempt a build of any software in nixpkgs so we can't know what builds without trying. it is really worth pointing out that volth's boost's [volths generic.nix](https://github.com/nix-windows/nixpkgs/blob/windows/pkgs/development/libraries/boost/generic.nix) looks nothing like the original [upstream generic.nix](https://github.com/nix-windows/nixpkgs/blob/e3de8a92325620e65039d98281047a83fa07c9dd/pkgs/development/libraries/boost/generic.nix). ### static vs. dynamic linking inside nixpkgs looking at [volths generic.nix](https://github.com/nix-windows/nixpkgs/blob/windows/pkgs/development/libraries/boost/generic.nix): ```nix installPhase = '' print("EXEC: b2 ${b2Args} install\n"); die $! if system("b2 ${b2Args} install"); renameL("$ENV{out}/include/boost-".('${version}' =~ s/^(\d+)\.(\d+).*/$1_$2/r)."/boost", "$ENV{out}/include/boost") or die $!; rmdirL("$ENV{out}/include/boost-".('${version}' =~ s/^(\d+)\.(\d+).*/$1_$2/r)) or die $!; '' + stdenv.lib.optionalString (!static) '' mkdirL("$ENV{out}/bin") or die $!; for my $dll (glob("$ENV{out}/lib/*.dll")) { renameL($dll, "$ENV{out}/bin/".basename($dll)) or die $!; } ''; ``` ## MinGW vs. msvc toolchain i never worked with visual studio / msvc much but **for the purpose of [libnix](https://lastlog.de/blog/timeline.html?filter=series::libnix), i think MinGW is a good choice for porting nix and create a custom MinGW toolchain inside nix** should be sufficient. ### corepkgs/config.nix ```nix let fromEnv = var: def: let val = builtins.getEnv var; in if val != "" then val else def; in rec { shell = "/usr/bin/bash"; coreutils = "/usr/bin"; bzip2 = "/mingw64/bin/bzip2"; gzip = "/usr/bin/gzip"; xz = "/mingw64/bin/xz"; tar = "/usr/bin/tar"; tarFlags = "--warning=no-timestamp"; tr = "/usr/bin/tr"; nixBinDir = fromEnv "NIX_BIN_DIR" "/usr/bin"; nixPrefix = "/usr"; nixLibexecDir = fromEnv "NIX_LIBEXEC_DIR" "/usr/libexec"; nixLocalstateDir = "/nix/var"; nixSysconfDir = "/usr/etc"; nixStoreDir = fromEnv "NIX_STORE_DIR" "/nix/store"; # If Nix is installed in the Nix store, then automatically add it as # a dependency to the core packages. This ensures that they work # properly in a chroot. chrootDeps = if dirOf nixPrefix == builtins.storeDir then [ (builtins.storePath nixPrefix) ] else [ ]; } ``` ### nix.exe #### run.cc volth put some effort into `run(ref store) override` function for getting it to work on windows ```c++ void run(ref store) override { auto outPaths = toStorePaths(store, Build, installables); ... ``` #### main.cc ```c++ #ifdef _WIN32 if (boost::algorithm::iends_with(programName, ".exe")) { programName = programName.substr(0, programName.size()-4); } #endif ``` #### ls.cc ```c++ void listText(ref accessor) { std::function doPath; auto showFile = [&](const Path & curPath, const std::string & relPath) { if (verbose) { auto st = accessor->stat1(curPath); std::string tp = st.type == FSAccessor::Type::tRegular ? ( #ifndef _WIN32 st.isExecutable ? "-r-xr-xr-x" : #endif "-r--r--r--") : ``` ### libexpr * `Path EvalState::coerceToPath(const Pos & pos, Value & v, PathSet & context)` ```c++ Path EvalState::coerceToPath(const Pos & pos, Value & v, PathSet & context) { string path = coerceToString(pos, v, context, false, false); #ifdef _WIN32 if (path.length() >= 7 && path[0] == '\\' && path[1] == '\\' && (path[2] == '.' || path[2] == '?') && path[3] == '\\' && ('A' <= path[4] && path[4] <= 'Z') && path[5] == ':' && isslash(path[6])) { return path; } if (path.length() >= 3 && (('A' <= path[0] && path[0] <= 'Z') || ('a' <= path[0] && path[0] <= 'z')) && path[1] == ':' && isslash(path[2])) { return path; } throwEvalError("string '%1%' doesn't represent an absolute path, at %2%", path, pos); #endif ``` * `std::pair decodeContext(const string & s)` ### libutil src/libutil/util.cc, namely: * `std::string to_bytes(const std::wstring & path) {` * `std::wstring from_bytes(const std::string & s) {` * `optional maybePathW(const Path & path) {` * `std::wstring pathW(const Path & path) {` * `std::wstring handleToFileName(HANDLE handle) {` * `Path handleToPath(HANDLE handle) {` * `std::string WinError::addLastError(const std::string & s)` * `std::wstring getCwdW()` * `std::wstring getArgv0W()` * `std::wstring getEnvW(const std::wstring & key, const std::wstring & def)` * `string getEnv(const string & key, const string & def)` * `std::map getEntireEnvW()` * `Path absPath(Path path, Path dir)` * `Path canonPath(const Path & path, bool resolveSymlinks) {` * `Path canonNarPath(const Path & path)` ### libstore * src/libstore/build.cc * `void DerivationGoal::startBuilder()` doing `tmpDir = tmpDirOrig = createTempDir("", "nix-build-" + drvName, false, false);` prepare build * no chroot on windows... * `void DerivationGoal::startBuilder()` -> `env variables` get exposed to the builder -> uenv -> uenvline -> `CreateProcessW(..)` * `void DerivationGoal::deleteTmpDir(bool force)` * `void DerivationGoal::handleChildOutput(HANDLE handle, const string & data)` * no 'sandboxing builds' * src/libstore/download.cc * a hack for making `libcurl` work with sleep (vs. callback) * a hack for `tar` with MSYS filesystem compatibility * src/libstore/gc.cc * `static void makeSymlink(const Path & link, const Path & target)` * `void LocalStore::removeUnusedLinks(const GCState & state)` * some file locking logic * src/libstore/local-store.cc * `void canonicaliseTimestampAndPermissions(const Path & path)` * `static void canonicalisePathMetaData_(const std::wstring & wpath, const WIN32_FIND_DATAW * wpathFD /* attributes might be known */, InodesSeen & inodesSeen)` * `void canonicalisePathMetaData(const Path & path, InodesSeen & inodesSeen)` * running [compact](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/compact) -> `auto rc = runProgramWithOptions(RunOptions("compact", { "/C", "/S:"+path, "/I" }));` * running [icacls](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/icacls) -> `runProgramWithOptions(RunOptions("icacls", { path2, "/reset", "/C", "/T", "/L" }));` // reset ACL on all children * src/libstore/nar-accessor.cc * hack since windows has no executable bit `i->isExecutable` * src/libstore/optimize-store.cc * `CreateHardLinkW` / `CreateFileW` file deduplication * `LocalStore::InodeHash LocalStore::loadInodeHash()` * `Strings LocalStore::readDirectoryIgnoringInodes(const Path & path, const InodeHash & inodeHash)` * `void LocalStore::optimisePath_(Activity * act, OptimiseStats & stats, const Path & path, InodeHash & inodeHash)` * src/libstore/pathlocks.cc * `AutoCloseWindowsHandle openLockFile(const Path & path, bool create)` * `void deleteLockFile(const Path & path)` * `bool lockFile(HANDLE handle, LockType lockType, bool wait)` * `bool PathLocks::lockPaths(const PathSet & _paths, const string & waitMsg, bool wait)` ### tests tests/local.mk contains some interesting mentions: ```bash # these do not pass (yet) on MINGW: # gc-concurrent.sh <-- fails trying to delete open .lock-file, need runtimeRoots # gc-runtime.sh <-- runtimeRoots not implemented yet # user-envs.sh <-- nix-env is not implemented yet # remote-store.sh <-- not implemented yet # secure-drv-outputs.sh <-- needs nix-daemin which is not ported yet # nix-channel.sh <-- nix-channel is not implemented yet # nix-profile.sh <-- not implemented yet # case-hack.sh <-- not implemented yet (it might have Windows-specific) # nix-shell.sh <-- not implemented yet # linux-sandbox.sh <-- not implemented (Docker can be use on Windows for sandboxing) # plugins.sh <-- not implemented yet # nix-copy-ssh.sh <-- not implemented yet # build-remote.sh <-- not implemented yet # binary-cache.sh <-- \* does not work in MSYS Bash (https://superuser.com/questions/897599/escaping-asterisk-in-bash-on-windows) # nar-access.sh <-- not possible to have args '/foo/data' (paths inside nar) without magic msys path translation (maybe `bash -c '...'` will work?) ``` ## volth's windows bash discussion we need to create a cross-platform `stdenv` and from this discussion i extracted this: * volth proposed these * `ruby` * [rakudo (Perl 6)](https://discourse.nixos.org/t/nix-on-windows/1113/48) + excellent Unicode support and + works well with Windows paths, including symlinks * [lua](https://discourse.nixos.org/t/nix-on-windows/1113/60) is proposed by john ericson * [oil shell](https://discourse.nixos.org/t/nix-on-windows/1113/75) - GSH, a non-interactive POSIX shell for Windows * nushell mentioned in recent times * `nushell` see from luc perkins (see also the [nixcon23 talk on YT](https://www.youtube.com/watch?v=QwElUltNsq0)) * https://github.com/mvdan/sh # summary **volth's work has been kept alive, thanks john ericson!** furthermore, john ericson even managed to already integrate parts of volth's work into nix! **volth's amazing prototyping work shows that it is feasible to use nix on windows. thanks volth**! the code base is a huge inspiration and shows how much is already possible and where we need to focus work! some platform limitations volth faced, and in parts even fixed, are now overcome by microsoft on windows 10 and onwards, namely: * path length limitations * case handling for paths/files per directory * directory/file permissions * proper symlink support * unix domain socket support ([Windows 10 Insider Preview Build 17063 in 2017](https://github.com/rust-lang/libs-team/issues/271)) however, there is still some major issue to be dealt with: * dynamic/static linking (i.e. rpath) but even here we have interesting developments **let's realize the potential of nix by integrating volth's findings into nix!**