Spec M44 -- packages: source distribution, Go-style, from a registry of sources

Owner's request (2026-09-04, translated from the Portuguese): "a package manager for mc, in the style of Go: it provides the source, and the manager is the repository of registered sources." Read as three requirements: (1) a package is SOURCE, fetched as source and compiled by mc like everything else; (2) the model is Go's -- a path-identified source tree, versioned by git tags, minimums in the manifest, a lock that pins content; (3) the manager is a REGISTRY OF SOURCES -- an index that maps a name to where the source is and which versions exist, never a host of binaries.

Goal: [deps] in mc.toml, mc.lock beside it, mc pkg as a sixth bundled part, a registry that is one git repository of TOML files, and a build that never reaches the network. Sequencing: after M41 (the parts and subcommand()), after M42's fix batch if it lands first (no dependency either way); M43 (the sandbox) is the milestone that makes an UNTRUSTED package's compiler module safe to run and is named below as the seam, not as a prerequisite.

What already exists #

Design #

1. Identity, names and versions #

A package is a source tree with an mc.toml at its root carrying a [package] table. Its identity in Go is its import path (github.com/user/pkg, a URL prefix); in mc it is a registry name, because the owner asked for the registry to BE the manager: the name is what the index maps to a location. The location is a detail of the index row, not of the source that uses the package.

Name rule: [a-z][a-z0-9_]*, at most 32 bytes. Why that set: it is a bare TOML key (src/toml.mc: A-Za-z0-9_-, minus - and upper case so the name is also a valid identifier prefix, geo_init), a valid path component on all three hosts, and a name that cannot collide with the bundle's mc/... namespace. A name that the bundle serves is refused (float, sys, i128, prelude, ...): the check is bopen_fn through the lexer, the same pointer limits.mc uses, so <mc/core_build> never depends on <mc/core_bundle>. Reserved outright: mc, deps, build.

Versions are semver X.Y.Z, one git tag vX.Y.Z per version (docs/ci.md § Versioning applied to packages; no pre-releases, for the reason next-version.sh gives). [deps] geo = "1.2.0" means at least 1.2.0 -- Go's require semantics, not an exact pin; the exact version is the lock's business (§ 3). A requested version must exist in the index (Go requires the same).

2. The import spelling changes the language by zero lines #

A dependency geo is included as

#include "geo/geo.mc"

-- the quote form, unchanged. The lexer gains named roots: lex_add_named_root(name, dir), consulted in lex_find_path_from AFTER the includer's own directory and BEFORE the anonymous [include].paths roots: when rel begins with <name>/ and <name> is a registered root, the rest is joined to that root's directory. M14's rule survives verbatim ("a project never shadows a relative include that already resolved"), and the match is exact on the first component, so two packages' files with the same basename cannot shadow each other -- the one failure mode a flat deps/ root added to [include].paths would have.

<name> stays what docs/build.md promises: the copy that shipped with this binary, no filesystem fallback. The boundary is one sentence: angle brackets are the bundle, quotes are files on disk, and a dependency is files on disk. The bundle/package collision question dissolves at the syntax level and is closed at the registry level by the reserved-name rule. Go's analogue: the standard library is not a module; fmt and github.com/x/y share one import syntax but the toolchain never fetches the former.

For a compiler module the same spelling goes into [compiler].modules:

[compiler]
modules = ["teach/mc_teach.mc", "user.mc"]

drv_gen_compiler writes #include "teach/mc_teach.mc" into the generated file (a value with no leading / and a first component that is a named root is emitted as written, not ../-adjusted: ~6 lines in drv_include), and the named roots are registered for BOTH compilations (drv_parse, before drv_apply_config's cfg test).

A package is closed. A file under a package's root may include or #embed only: its own tree, the bundle (<...>), and the roots of the packages its lock row names as deps. A resolved path that lands anywhere else -- the project's files, an absolute path, another package it did not declare -- is geo/vec.mc:3: package geo reaches outside its tree: /etc/hosts, exit 1. ~35 lines in lex.mc (lex_root_of(path) = longest registered root that is a string prefix of the normalised path; an edge list (from_root, to_root) in registration order). This is the M15 "no filesystem fallback, on purpose" stance applied to packages, and it is the cheapest honest answer to #embed as an exfiltration primitive (§ Risks 1).

3. The lock, the tree hash, and what "deterministic" means here #

mc.lock, beside mc.toml, written only by mc pkg, rows sorted by name (bytewise, an insertion sort on unique keys -- rule 2 of docs/determinism.md forbids qsort's tie-breaking, not ordering; the sort key is total):

# written by `mc pkg sync` -- do not edit (docs/reference/packages.md)
[[package]]
name    = "geo"
version = "1.2.0"
sha256  = "9f1c...e2"          # tree hash, below
deps    = ["mathx"]

[[package]]
name    = "mathx"
version = "1.1.0"
sha256  = "41b0...7a"
deps    = []

Not inside mc.toml: the lock is machine-written and complete, mc.toml is hand-written and minimal, and lim_fix_write shows how much care a machine edit of a human file costs. Go keeps go.sum beside go.mod for the same reason.

The tree hash is a dirhash-style content hash -- the shape of Go's dirhash.Hash1, but written in plain hex, not Go's base64 h1: form -- in manifest order: for mc.toml first and then each entry of [package].files in the order written, one line hex(sha256(file bytes)), two spaces, the path, \n; the hash is sha256 of those lines. Go sorts the file list because it derives it from a zip; mc cannot list a directory, so the package AUTHOR lists the files (the precedent is tools/bundle.list and the member column of a sysroot row), and manifest order is the canonical order. The hash is of CONTENT, never of the archive: GitHub's archive/refs/tags/*.tar.gz bytes changed under a git upgrade on 2023-01-30, broke every archive-checksum consumer (Homebrew among them) and were rolled back -- a policy, not a format guarantee. A content hash is also the same function for all three ways a tree can arrive (fetched, vendored, replaced by a path), which is what lets mc pkg verify be one routine.

The lock is checked, not trusted. mc build rehashes every locked package on every build (sha256.mc already hashes the whole output executable per 4 KiB page on every --exe; a dependency's source is smaller than that and is about to be lexed anyway) and refuses on any disagreement:

disagreementmessageexit
a file's bytes differ from the cache manifest's linemc: geo 1.2.0: vec.mc does not match mc.lock2
the tree hash differs but no file line does (the files list changed)mc: geo 1.2.0: mc.toml does not match mc.lock2
[deps] names a package the lock lacks, or asks a minimum above the lockmc: mc.lock is stale: run mc pkg sync --yes2
the lock names a version that is neither vendored nor cachedmc: geo 1.2.0 is not fetched + run: mc pkg sync --yes2
a file the build READ under a package root is not in that package's filesgeo/extra.mc:1: not declared in geo's [package].files1

Exit 2 because every one of these is "the environment is not ready" in the M25 sense, and a script must be able to tell it from "your program does not compile". The last row is the post-parse walk of the lexer's once-only list (inc_at, src/lex.mc:557) and is what makes the files list a real boundary rather than documentation.

Version selection is MVS, precisely Go's (cmd/go/internal/mvs, Russ Cox 2018): the build list starts from the project's [deps]; for every selected (name, version) the requirements of THAT version are added; a name's selected version is the maximum over every minimum that mentions it; repeat to a fixed point. No SAT, no search, no "latest": the answer is a function of the manifests alone, and the lock freezes it so that the index can move afterwards without moving the build. The registry row carries each version's requirements inline (Go's proxy serves .mod files before .zip for the same reason), so MVS runs over the index snapshot with no archive download. Semver comparison is the --gt arithmetic of scripts/next-version.sh, rewritten in .mc (~20 lines).

Majors. Go handles a major bump with semantic import versioning (/v2 in the path). mc does not: if the selected version's major differs from the major of ANY requirement that named it, the resolution is refused -- mc: mathx: 1.1.0 and 2.0.0 are different majors: no solver. Two majors of one name in one build is the case MVS is not designed for, and pricing a /v2 scheme is out of scope.

Yanked. A registry row may gain yanked = true (Go's retract, 1.16) and nothing else may ever change in a row. mc pkg add|update skip yanked versions; sync of a lock that already pins one warns and proceeds, so a build never breaks retroactively.

4. Fetching: the M25 road, generalised #

src/fetch.mc (in <mc/core_build>, included before sysroot.mc) takes the three functions out of sysroot.mc in their general form: fetch_get(src, file) -- an https:// source spawns the downloader exactly as today (--proto '=https' --proto-redir '=https' -fLsS, then the wget fallback), and a source with no scheme is a local path copied with read_file/write_file; fetch_extract(archive, dest, strip, members); hex64(digest). sysroot_extract becomes a four-line call into it. The local-path branch is what makes the test suite need no network and what prices a private registry at zero (§ 5).

Tarballs over git: tarballs. A GitHub tag has a URL (https://github.com/<u>/<r>/archive/refs/tags/v1.2.0.tar.gz, top directory <r>-1.2.0, hence strip = 1), tar -xzf reads it on all three hosts (M25 § 2 verified bsdtar, GNU tar and Windows' libarchive tar.exe on gzip), and git stays off the dependency list -- the same argument M25 made for curl over an HTTP client, one level up. The index row carries the url explicitly, so a non-GitHub forge is a different URL and nothing else.

Cache: host_home()/.mc/pkg/<name>/<version>/ holds the extracted tree and host_home()/.mc/pkg/<name>/<version>.toml (beside, not inside, so a package file named manifest.toml cannot collide) holds [source] name version url sha256 and one [[file]] path sha256 per hashed file. --pkg-dir DIR on mc pkg and mc build overrides the root, as --sysroot-dir does, so CI depends on no HOME. Order of operations, unchanged from M25 because there is no rename: download into <pkgdir>/<name>/<version>.tar.gz, extract into the final directory, hash, and on any failure unlink every listed file (the sysroot_unbless idea) and write no manifest; the manifest is the claim that the directory holds what the lock says, written last. The tree is hashed and compared right after extraction, before anything else: the archive itself is not checksummed (§ 3 says why), so unlike M25 the refusal comes after the bytes are on disk -- but still before any manifest exists and before any build can consume the tree, and a refused tree is unlinked.

Vendoring: mc pkg vendor copies each locked package (mc.toml + files) into deps/<name>/ in the project. When deps/<name>/ exists it is used and the cache is not consulted (Go: a vendor/modules.txt consistent with go.mod makes -mod=vendor the default since 1.14). Both forms are hashed against the lock, so the choice cannot change bytes silently. deps/ plus mc.lock in git is the fully offline project.

Replacement: [replace] geo = "../geo" points a name at a local tree for development (Go's replace directive). A replaced package is registered as a named root from that path, its lock row says path = "../geo" and carries no hash, and mc pkg sync|list print 1 replaced dependency: not pinned by mc.lock. Go's go.sum also omits path-replaced modules.

mc build never downloads (M25, architect's addition (a)). It reads the lock, finds each tree in deps/ or the cache, hashes, registers roots, compiles. Nothing else.

5. The registry: one git repository of TOML #

Compared: Go's proxy.golang.org (immutable zips served by a service) + sum.golang.org (a transparency log of hashes) is the right shape and the wrong size for a one-owner compiler -- it needs a server, a log, and an operator. A Homebrew tap is a git repository of formula files that anyone forks and PRs, and brew reads it as files. A tap of TOML is the choice, with the sumdb's one property kept: a published row never changes.

Repository schivei/mc-registry:

README.md                      the three rules (name, immutability, how to register)
index/<name>.toml              one file per package, added by PR
.github/workflows/check.yml    `mc pkg check` on every changed index file; refuses a diff that
                               edits an existing [[versions]] row except to add `yanked = true`;
                               a scheduled job re-checks every row (link rot, like check-sysroots)
# index/geo.toml
[package]
name        = "geo"
repo        = "https://github.com/schivei/mc-geo"
description = "2-D vectors"

[[versions]]
version = "1.0.0"
url     = "https://github.com/schivei/mc-geo/archive/refs/tags/v1.0.0.tar.gz"
strip   = 1
sha256  = "<tree hash>"
deps    = []

[[versions]]
version = "1.2.0"
url     = "https://github.com/schivei/mc-geo/archive/refs/tags/v1.2.0.tar.gz"
strip   = 1
sha256  = "<tree hash>"
deps    = ["mathx 1.1.0"]

mc reads the index one file at a time: <registry>/index/<name>.toml, where <registry> is https://raw.githubusercontent.com/schivei/mc-registry/main by default (PKG_REGISTRY in src/pkg.mc), [registry].url in mc.toml, or --registry URL|DIR. An https:// registry is fetched with fetch_get into <pkgdir>/index/<name>.toml (the snapshot mc pkg reads offline); a DIR registry is read in place. So: a private registry is a directory or any URL with the same layout -- a git clone of a private tap and [registry] url = "/path/to/it" -- and costs no code. That is the price of "private registries": zero, and no access control beyond the repository's own.

Who can register: anyone, by pull request, which is Go's "anyone with a public repository" translated to a registry that has a human in it. The owner (CODEOWNERS on index/) reviews; the CI check is the gate that matters: it downloads each new row with --yes, recomputes the tree hash, and refuses a row whose hash, [package].name or deps do not match the archive's own mc.toml, a name outside the rule, a name the bundle serves, or a repo change on an existing name without the owner's review. Name squatting is a policy, not a mechanism: first PR wins the name, transfers go through the owner. Registration is versioned in git, so the history of every row is the audit log the sumdb would have provided.

6. Two kinds of package, and both #

The manifest does not enforce a kind; it names entries:

# the package's own mc.toml, at the repository root
[package]
name   = "geo"
files  = ["geo.mc", "vec.mc", "mc_geo.mc"]
lib    = "geo.mc"        # optional: the file a PROGRAM includes
module = "mc_geo.mc"     # optional: the file a COMPILER includes; it provides geo_init()

[deps]
mathx = "1.1.0"

lib/module are informational (mc pkg list and the registry page print them); what a file does is decided by who includes it, as it is for every .mc file today.

7. Commands, and where they live #

A new part, <mc/core_pkg> (src/core_pkg.mc), that includes <mc/core_build> (once-only) and src/pkg.mc, and registers one subcommand. The M41 debloat argument applies twice:

mc pkg sync   [DIR] [--config FILE] [--yes] [--registry URL|DIR] [--pkg-dir DIR]
mc pkg add    NAME[@VERSION] [DIR] [--config FILE] [--yes] [--registry ...] [--pkg-dir DIR]
mc pkg update [NAME] [DIR] [--config FILE] [--yes] [--registry ...] [--pkg-dir DIR]
mc pkg list   [DIR] [--config FILE] [--pkg-dir DIR]
mc pkg vendor [DIR] [--config FILE] [--pkg-dir DIR]
mc pkg verify [DIR] [--config FILE] [--pkg-dir DIR]
mc pkg hash   DIR
mc pkg check  INDEX.toml [--yes] [--pkg-dir DIR]
commandGo analoguewhat it does
syncgo mod tidy + go mod downloadreads [deps], fetches the index rows it needs, runs MVS, writes mc.lock (drops rows nothing requires), fetches missing trees. Without --yes: prints the plan -- each index file and archive it would download, with hash and destination -- and nothing was downloaded: re-run with --yes, exit 0 (M25 D10; mc has no isatty). With a DIR registry and everything cached it completes with no download
addgo get pkg@vwrites NAME = "VERSION" into [deps] (newest non-yanked when no @) by the lim_fix_write method -- one key, one section, every other byte through -- then sync
updatego get -uraises the [deps] minimum(s) to the newest non-yanked index version, then sync
listgo list -m allone line per lock row: name version sha256[0..12] vendored|cache|path -- no absolute path, so it is a golden
vendorgo mod vendorcopies each locked tree into deps/<name>/, then verify
verifygo mod verifyrehashes every locked tree (vendored or cached) and checks lock vs [deps] consistency; exit 0 or 2 with the § 3 messages
hash(dirhash.Hash1)prints the tree hash of a checkout: the author's tool and the registry CI's
check--the registry-side gate of § 5

mc build gains --pkg-dir DIR only. The single-file CLI gains nothing: a mc x.mc build has no mc.toml and therefore no dependencies, and --include=DIR already reaches a vendored tree by hand.

8. Publishing a package #

A package repository contains: mc.toml with [package] (and [deps] if any); the files it lists; optionally a [project] for its own tests (a package that is also a program is fine; a program's mc.toml without [package] is not a package). Cutting a version is git tag v1.2.0 plus git push --tags. Registering it is one PR to mc-registry adding a [[versions]] row whose sha256 the author gets from mc pkg hash . on the tagged checkout; the registry CI re-derives the hash from the tarball and refuses a mismatch, so a moved tag (§ Risks 4) is caught before it is published, and a tag moved AFTER publication is caught by every consumer's fetch.

The bundle is the standard library and is not in the registry. <float>, <i128>, <sys> are versioned with the compiler and need no fetch; they are what "the binary alone is the toolchain" (M15) means. Their names are reserved in the registry. Promoting a registry package into the bundle is a normal tools/bundle.list change, and demoting a bundled library to a package is possible in principle (the [package] manifest describes lib/ as well) but is not this milestone.

9. TOML: one parser, re-entrant #

toml_push()/toml_pop(frame) in src/toml.mc (~30 lines): save the eight table globals (tm_ents tm_entcap tm_n tm_aot_name tm_aot_n tm_aotcap tm_naot tm_file) into an arena frame, zero them, and restore. The lock, an index entry and a package manifest are each parsed inside a push/pop; what is needed is copied into deps.mc's own flat tables (name, version, hash, deps in source order -- rule 1) before the pop. Plus toml_occurrences(name) (~8 lines) so [[package]] and [[versions]] rows can be counted. This replaces the third bespoke reader before it is written; lim_read_usage stays as it is.

Out of scope #

Files and estimated deltas #

filelineswhat
src/fetch.mc~120, newfetch_get (https via downloader, else local copy), fetch_extract, hex64: the M25 three, general
src/sysroot.mc-75 / +12calls fetch_*; sysroot_extract reads its row and delegates
src/deps.mc~340, newname rule, [deps]/[replace]/[registry] reading, lock reader, tree hash over mc.toml + files, cache/vendor/path resolution, per-file verification, named-root and edge registration, the five refusals of § 3
src/pkg.mc~760, newsemver, index fetch/snapshot, MVS, lock writer (sorted), archive fetch + hash + manifest, vendor copy, [deps] line edit, hash, check, list, verify, pkg_cmd dispatch
src/core_pkg.mc~22, newincludes core_build.mc + pkg.mc; mc_pkg_init() = one subcommand("pkg", &pkg_cmd, ...)
src/core_build.mc+2fetch.mc, deps.mc in the list
src/core.mc, src/main.mc+1 / +1the sixth part; mc_pkg_init()
src/toml.mc+38toml_push/toml_pop, toml_occurrences
src/lex.mc+65named roots table (grow, a T_NAMEDROOT tag), lex_add_named_root, the branch in lex_find_path_from, lex_root_of, the closure rule in lex_include and the #embed path, the post-parse files walk's accessor
src/driver.mc+48--pkg-dir; drv_apply_config split into drv_apply_deps (both halves) and the entry-only rest; drv_include keeps a named-root path verbatim; the post-parse files check
src/cli.mc0
tools/bundle.list, src/bundle_data.mc+4, regeneratedmc/fetch, mc/deps, mc/pkg, mc/core_pkg
tests/pkg/registry/index/{geo,mathx,teach,bad}.tomlfixturerows with url = "@ARCHIVES@/geo-1.2.0.tar.gz" rewritten by the script; hashes hard-coded
tests/pkg/src/{geo-1.0.0,geo-1.2.0,mathx-1.0.0,mathx-1.1.0,mathx-2.0.0,teach-1.0.0,bad-1.0.0}/fixtureeach with mc.toml [package]; teach registers a syntax_stmt("unless", ...) as teach_init(); bad includes outside its tree
tests/pkg/app/{mc.toml,obj.toml,main.mc,user.mc,mc.lock.expect,add.toml,add.toml.expect}fixturethe consumer: [deps] geo = "1.2.0", mathx = "1.0.0", teach = "1.0.0"; main.mc uses unless and geo_dot
tests/golden/pkg-list.txtnew
.gitattributes+1tests/pkg/src/** -text so the fixture hashes hold on a Windows checkout
scripts/check-pkg.sh~320, new§ Acceptance 1-17
scripts/check-parts.sh+25case 1b for <mc/core_pkg>; the offline-consumer probe
scripts/check-docs.sh+1the pkg_/deps_ families and the --pkg-dir/--registry flags
Makefile+8check-pkg, inside check
docs/reference/packages.md~420, neweverything above, every message, the manifest, the lock, the registry layout, the closure rule
docs/guide/25-packages.md~240, newusing one; publishing one; the user.mc six lines; vendoring for CI
docs/reference/toml.md+80[deps], [replace], [registry], [package]
docs/reference/cli.md+45mc pkg, --pkg-dir, exit 2 rows
docs/reference/diagnostics.md+55§ 13 packages
docs/reference/bundle.md, docs/build.md, docs/determinism.md, docs/plan.md, docs/README.md, docs/guide/98-recreating-the-compiler.md+15 / +30 / +12 / +1 row / +1 / +10the sixth part, reserved names, the lock as a determinism input
schivei/mc-registry (separate repository)README.md ~80, .github/workflows/check.yml ~70, index/seeded with the owner's first package; the teko port when it exists
stage0/, lib/0

Net new src/ lines ~1 350; the five goldens move once (the bundle grows and core.mc gains a part), under M17 step A's protocol and M41's note 9.

Acceptance (ordered; scripts/check-pkg.sh build/mc1, no network anywhere) #

  1. The fixture registry is built by the script, not checked in as archives: each tests/pkg/src/<name>-<v>/ is tar -czf'd into $tmp/archives/<name>-<v>.tar.gz with the top directory <name>-<v> (strip = 1, GitHub's shape), and tests/pkg/registry/index/*.toml is copied to $tmp/registry/index/ with @ARCHIVES@ replaced. gzip timestamps make the archives non-reproducible; nothing hashes them.
  2. The hash is stable across hosts: mc pkg hash tests/pkg/src/geo-1.2.0 prints the value hard-coded in the fixture index, on macOS, Linux and Windows CI.
  3. The plan is printed and nothing is fetched: mc pkg sync tests/pkg/app --registry $tmp/registry --pkg-dir $tmp/c1 lists three archives with hashes and destinations, prints nothing was downloaded: re-run with --yes, exits 0, and $tmp/c1 holds no <version>.toml.
  4. MVS, not "latest": the same with --yes writes tests/pkg/app/mc.lock byte-identical to mc.lock.expect -- mathx at 1.1.0 (geo 1.2.0's minimum wins over the app's 1.0.0), never at the registered 2.0.0, rows sorted; three <version>.toml manifests exist with [[file]] rows; running sync --yes again downloads nothing and rewrites the lock identically.
  5. A two-package chain builds and runs: mc build tests/pkg/app --pkg-dir $tmp/c1 builds the taught compiler from teach/mc_teach.mc + user.mc, compiles main.mc (which includes "geo/geo.mc", whose own #include "mathx/mathx.mc" resolves through the closure), and the binary's stdout/exit match main.mc's header.
  6. Byte-identical objects from two fetches: a second sync --yes into $tmp/c2 and a build with obj.toml (kind = "obj") from each cache give cmp-identical objects; the two lock files are identical.
  7. The lock refuses a tampered source: append one byte to $tmp/c1/geo/1.2.0/vec.mc; mc build exits 2 with geo 1.2.0: vec.mc does not match mc.lock; mc pkg verify says the same; restore, both exit 0. Then edit files in the cached mc.toml: exit 2 naming mc.toml.
  8. Stale lock, stale tree: raise [deps] geo to "1.9.0" in a copy of the config -> mc build exits 2 with mc.lock is stale: run mc pkg sync --yes; delete $tmp/c1/mathx/ -> exits 2 with mathx 1.1.0 is not fetched and the run: line. mc build must not have spawned curl in either case (the script puts a curl shim on PATH that fails if invoked).
  9. Vendoring is the offline road: mc pkg vendor populates tests/pkg/app/deps/{geo,mathx, teach}/; mc build --pkg-dir $tmp/empty succeeds with no cache at all; its object is cmp-identical to step 6's; mc pkg list prints vendored in every row and matches tests/golden/pkg-list.txt.
  10. A package is closed: add bad = "1.0.0" to a copy of the config, sync, build -> exit 1 with bad/bad.mc:2: package bad reaches outside its tree: ...; a #embed of an absolute path inside bad is refused with the same words; a file geo/extra.mc planted in the cache and included by a planted line is not declared in geo's [package].files.
  11. Names: [deps] float = "1.0.0" -> float: a bundled name; Geo = ... -> invalid package name; both at the key's file:line:col, exit 1.
  12. Majors are refused, not solved: a fixture package requiring mathx 2.0.0 next to the app's 1.x -> mathx: 1.1.0 and 2.0.0 are different majors: no solver, exit 1, no lock written.
  13. A failed fetch leaves no claim behind: an index row whose url names a missing archive -> exit 2 with the M25-shaped message (mc: the download failed for a URL, cannot open for a path), no <version>.toml, and a following mc build says not fetched rather than reading debris. A row whose sha256 is wrong -> checksum mismatch for geo 1.2.0, the listed files unlinked, no manifest.
  14. mc pkg add edits one line: on add.toml (no [deps]), mc pkg add mathx@1.0.0 --yes produces add.toml.expect byte for byte -- every comment and byte outside the new [deps] section untouched; add mathx with no version picks 1.1.0, never the registered-and-yanked 1.2.1 row the fixture carries.
  15. mc pkg check is the registry gate: on $tmp/registry/index/geo.toml --yes --pkg-dir $tmp/chk exit 0; with one hash altered exit 2; with name = "float" exit 1.
  16. Parts: check-parts.sh shows <mc/core_min> + <mc/core_pkg> compiles alone; a probe compiler assembled from core_min + core_machines + core_writers + core_build + core_bundle (no core_pkg) builds the vendored app of step 9 and prints a usage with no pkg line; the measured table gains the + <mc/core_pkg> row.
  17. Inert: scripts/check-inert.sh clean for every object of a project without [deps]; check-standalone, check-obj 32/32, check-build 21/21, check-sysroots and tests/golden/sysroot-list.txt unchanged; mc with no arguments prints today's usage plus exactly the pkg lines; mc sysroot fetch linux-aarch64 --yes --sysroot-dir $tmp/s (the CI step, the only networked check, unchanged) still writes the four files -- proving the fetch.mc move is behaviour-neutral. Goldens rewritten once, after an empty --dump-asm diff between mc1 and mc2 and cmp build/mc2.o build/mc3.o. make check-docs green.

Risks #

  1. Supply chain: a compiler module is code that runs on the developer's machine at build time. Go's go build executes no package code (no build scripts), which is a security property Go chose on purpose; mc's compiler modules are proc-macro-shaped and DO run, inside the spawned taught compiler, with the developer's file system. A library package is only slightly better: #embed reads any file the compiler can. M44's brakes are the closure rule (§ 2: a package reads its own tree, the bundle and its declared deps, nothing else, for #include and #embed alike) and the lock (nothing runs that is not the reviewed bytes). What they do not stop is a module that opens a file by extern open at user_init time. That is M43's job and the seam is the spawn in drv_teach: the taught compiler is already a separate process, so a sandbox wraps one posix_spawnp. Until M43 lands, packages.md says in its first paragraph that a compiler-module package is trusted code.
  2. Name squatting and transfers are policy in a one-owner registry: first PR wins, the owner reviews, repo changes need the owner. Cheap now; revisit if the index passes a few hundred names.
  3. Tarball regeneration (GitHub, 2023-01-30) does not move the tree hash; it would move an archive hash, which is why there is none.
  4. Tag mutability: a tag moved after registration fails every consumer's fetch with checksum mismatch -- loud, and the right outcome. A tag moved BEFORE registration is a registry row that never matched, refused by mc pkg check.
  5. Network in CI: none in make check; the fixture registry is a directory. Only the registry repository's scheduled job and the existing mc sysroot fetch step touch the network, and a dead URL is a maintenance issue there (M25 § Risks), not a red PR.
  6. toml_push/toml_pop regressions: a pop forgotten on an error path leaves the project table swapped out. Every parse of a foreign file is wrapped in one function that pops before returning or dies (_exit, where the table no longer matters).
  7. The files list as the boundary: a package author who forgets a file ships a package that fails with not declared in ... [package].files on the first include -- loud, at the consumer. mc pkg check could cross the list against tar -t output (captured with the drv_sdk file-action trick) and warn; priced at ~40 lines, optional.
  8. Line endings on Windows checkouts move every fixture hash; .gitattributes -text on tests/pkg/src/** is in the file table, and the Windows check subset runs step 2 first.
  9. mc pkg add editing a human file: lim_fix_write's method keeps every other byte, and step 14 cmps the result; the failure mode is a [deps] table written twice in a file that already has it under an unusual spelling ([ deps ]), which the key scan does not recognise. Refuse when the scan finds no [deps] but toml_get("deps.x") says one exists.
  10. No version string in mc: a package needing a hook from a newer mc fails with call to unknown function rather than "needs mc >= 0.9". Acceptable until 1.0.0 (§ Out of scope).
  11. Six parts and a sixth *_init: main.mc's list grows; check-parts.sh case 1b is the regression net M41 built for exactly this.
  12. Diagnostics in cached packages print absolute paths (/Users/me/.mc/pkg/geo/1.2.0/vec.mc:3) while vendored ones print deps/geo/vec.mc:3. Objects carry no path (rule 4, and there is no N_OSO), so determinism of OUTPUT holds -- step 6 and step 9 prove it -- but two machines' error texts differ. A display name (geo/vec.mc, the bundle's "errors point at the bundled name" precedent) is ~15 lines in lex_push and a follow-up; M30's DWARF will want it too.

Decisions (architect) -- to ratify with the owner #

Architect's additions: (a) step 8's curl shim -- mc build must be PROVED never to spawn a downloader, not just documented; (b) the registry repository's README.md is written in the same PR as packages.md, with the three rules and one worked registration; (c) docs/guide/25-packages.md is written for the teko port's author and ends with the six-line user.mc.


Amendment (owner, 2026-09-04): angle brackets, ~/.mc/libs, the slim binary, install/update/upgrade #

The owner's two rulings, translated from the Portuguese:

(1) "For M44 I would change one thing: since these are external libraries referenced in the toml, they should be reachable as #include <pack/lib.mc>. What that changes: even what we ship embedded today could become an unpacking into a directory ~/.mc/libs/pack_name/v<version>/, and with that we gain even more extensibility."

(2) "I would say we even gain a smaller binary: it could carry only the executable and require an mc install, downloading even what is embedded today; an mc update updates the packages' versions; an mc upgrade self-updates the version and downloads the basic libraries for mc; and so on."

Read as four requirements that override D1, D2 and D15 and extend the scope: (a) a dependency is spelled with angle brackets, #include <pack/lib.mc>, so there is ONE resolution model for "a library that did not come from my own tree"; (b) the bundle's entries are a package like any other, materialisable under ~/.mc/libs/<pack>/v<version>/; (c) a second, slimmer release flavour carries no blob and gets its libraries through mc install; (d) three verbs, install, update, upgrade, the last one a self-update. What follows rewrites the draft where the rulings touch it and says, item by item, what does not move.

What survives untouched, and why. The lock (§ 3), the h1 tree hash over mc.toml + [package].files, MVS with the two-majors refusal, the registry as one git repository of TOML (§ 5), the closure rule (a package reads its own tree, the bundle and its declared deps, nothing else), the sixth part <mc/core_pkg> with the read side in <mc/core_build> (§ 7), "mc build never downloads", toml_push/toml_pop (§ 9), D3-D9, D11-D14, D16-D17. None of them depends on HOW an include is spelled or WHERE a tree lives: they are about identity, content and selection. The rulings change the spelling, the directory, and add a distribution channel for the compiler's own libraries; they do not change what a package IS.

A. Angle brackets and one resolution model #

A1. The spelling. A dependency geo is included as

#include <geo/geo.mc>        // a file of the package geo, at the version mc.lock pins
#include <geo>               // the package's `lib` entry (mc.toml: lib = "geo.mc"), if it has one
#include <float>             // unchanged: a name the binary ships
#include <mc/core>           // unchanged

<name> stops meaning "the bundle" and starts meaning "a library that is not in my tree": resolved from the lock, from the bundle, or from the installed copy of the compiler's own package -- in that order, below -- and never from the working directory. Quotes keep meaning "a file on disk relative to me or to [include].paths" (src/lex.mc:536, lex_find_path_from, unchanged). The draft's named roots for the quote form (§ 2, lex_add_named_root, the branch in lex_find_path_from) are DROPPED: one spelling for one thing, which is what the owner asked for. [compiler].modules takes the same spelling -- modules = ["<teach/mc_teach.mc>", "user.mc"] -- and drv_gen_compiler emits a value that starts with < verbatim, the rule src/driver.mc:417-425 already applies to [compiler].core (core = "<mc/core_min>", M41). Nothing in the language changes except one token (A2).

A2. A fact the draft did not check: . is not a token. tok_init (src/lex.mc:246-294) registers the keywords and the operator lexemes; there is no "." among them, and <name> is not tokenised specially -- do_directive (src/parse.mc:1610-1626) reassembles the lexemes between < and >, on purpose, so that --dump-tokens stays byte for byte what stage0/lex.c produces. So #include <geo/geo.mc> fails today at the . with unexpected character (src/lex.mc:902), and the owner's spelling needs one of two things: (i) tok_add(".", 1) appended at the END of tok_init, after the last existing lexeme, so no existing id moves (K_U8..K_EXTERN stay 256..269 and every punct keeps its id), --dump-tokens is unchanged for every file that has no bare . -- which is every file check-lex compares, since float literals are consumed raw by syntax_lit (lib/float.mc:343) before the punct scan and lib/float_rt.mc is already seed-skip -- and examples/lang/lang.mc:44's own tok_add(".", 1) lands on the same id because tok_add is idempotent (the comment two lines above it says so); or (ii) accept only <geo/geo> and append .mc on disk, the bundle's own convention (lex_strip_mc, src/lex.mc:574, exists because bundle names carry no .mc). Recommendation: (i), and lex_include_name strips a trailing .mc from the reassembled name so <geo/geo.mc> and <geo/geo> are one name, exactly as <mc/core> and a relative "core.mc" inside the bundle are today. Cost: one line in tok_init, one call in lex_include_name. stage0/lex.c is untouched: it never sees a <...> with a dot (tests/mc/ is where such files live, and that directory is already outside the mc0 cross-checks).

A3. The resolution order for <X> -- in lex_include_name (src/lex.mc:626), which today is one call to bopen_fn (:498) and one error. It becomes three steps through TWO pointers, the existing bopen_fn and a new lopen_fn registered by lex_set_libs from mc_build_init() (src/core_build.mc:36), so the lexer still depends on nothing and lexdump/astdump keep compiling with mc0:

stepwho answersfor which nameswhere the bytes come from
1lopen_fn(X, 0) -- deps.mc's libs_open, the LOCK roadX's first path component is a package the lock names (geo, teach, or a bundled name pinned in [deps], A5)deps/<pack>/<rest> if vendored, else <libs>/<pack>/v<version>/<rest>; <pack> alone is the lock row's lib entry
2bopen_fn(X, 0) -- the bundle, unchanged (src/bundle.mc, bundle_open; <mc/host> still rewritten by host_bundle_open, src/core_bundle.mc)every name in tools/bundle.list, plus the two synthetic onesthe blob
3lopen_fn(X, 1) -- the INSTALLED mc packagethe same names as step 2, when the binary carries no blob (B)<libs>/mc/v<mc_version()>/ + the path bundle.list maps the name to
--neitheranything elseprog.mc:1: unknown bundled include: no/such/module, unchanged text

<libs> is host_home()/.mc/libs (src/host_macos.mc, src/host_linux.mc, src/host_windows.mc: the HOME=/USERPROFILE walk sysroot_cache_dir already uses, src/sysroot.mc:215), or --libs-dir DIR on every command that reads it (the --sysroot-dir precedent), so CI depends on no HOME. Step 1 exists only when a lock was read (mc build, both halves); the single-file CLI (mc x.mc) has no lock and therefore no step 1: <geo/geo.mc> there is refused with the step-3 miss, and --include=DIR remains the hand road. Never the working directory (a project cannot shadow <float> by dropping a float.mc next to main.mc; that is the M15 stance and it is what lets the answer be a function of (binary, lock, libs content) alone), never an unpinned latest (<libs>/geo/ may hold v1.0.0/ and v1.2.0/; only the lock says which one, and a directory that no lock names is never opened).

A4. The mc package and the layout of ~/.mc/libs/mc/v<version>/. The bundle's 75 entries are ONE package, named mc, at the compiler's own version (C). On disk it keeps the REPOSITORY layout -- lib/float.mc, src/core.mc, src/host_macos.mc, tests/mc/bundle/embed_demo.txt -- with bundle.list (the NAME<TAB>PATH manifest, tools/bundle.list) at the root as the name map. Not a by-name layout, because the bundle's relative-include fallback (bundle_find_base, src/bundle.mc: mc/driver -> "../lib/prelude.mc" -> last component prelude) has no equivalent on a filesystem; in the repository layout src/driver.mc's #include "../lib/prelude.mc" resolves through lex_find_path as a plain relative path, and src/core.mc's "arena.mc" lands on src/arena.mc. The once-only key for a name served from disk is the normalised absolute path (lex_include records paths, lex_include_bundled records canonical names; both go through lex_seen, src/lex.mc:557), so <mc/host>, <mc/host_macos> and core.mc's own "host_macos.mc" coincide on disk exactly as they coincide in the blob. Two consequences worth writing down: the installed src/bundle_data.mc is the mode 1 text (#embed bundle_blob "bundle.bin" + the index, bundle_emit(..., 1), src/bundle.mc) next to a real src/bundle.bin (the blob, bundle_read(BUNDLE_BIN)), so a taught compiler built from an on-disk <mc/core> pays one N_BLOB node and not ~45 000 u64 nodes (M21.5's arena argument, docs/build.md § M15), and both forms produce the same object (that is what check-standalone measures today); and tools/bundle.list itself is bundled as mc/bundle.list (one line in the manifest; its bytes do not depend on its own content, so unlike bundle_data there is no recursion; last component bundle.list collides with nothing), so the FULL binary can write the whole package to disk from its own blob with no network (B3). Diagnostics from a disk-served <mc/core> print the absolute path where the bundle printed the name (draft Risk 12, unchanged and now more visible).

A5. Collisions: the lock wins, except for mc. A registry package MAY carry a bundled name (float, sys, i128 -- the draft's reserved-name rule is withdrawn for lib/'s names) and a project that pins [deps] float = "1.3.0" gets <float> and <float/float_rt.mc> from the locked tree instead of the blob. That is the extensibility the ruling asks for, and it is safe on both counts the task names: determinism -- step 1 answers only from a lock row, the row pins a content hash, mc build rehashes on every build (D7), so two machines with the same binary, lock and tree bytes resolve the same bytes; a project with no [deps] float line is byte for byte what it was; offline -- a locked float that is neither vendored nor fetched is mc: float 1.3.0 is not fetched / run: mc pkg sync --yes, exit 2, the same as any dependency, and a project with no lock never leaves the blob. What stays reserved, in the registry and in [deps]: mc, every mc/... name, deps, build. The mc package can never be pinned by a lock: <mc/core> is the compiler's own source, check-standalone is an equality between a binary and ITS bundle, and a taught compiler assembled from a foreign mc/core would be a different compiler than the one running mc build -- the exact confusion M37's <mc/host> was designed out. mc pkg check refuses an index row named mc; deps.mc refuses [deps] mc = ... at the key (mc: reserved, exit 1).

A6. The closure rule, restated for the new spelling. A file under a package root may include or #embed: its own tree (quotes, relative), <...> names that resolve to the bundle or to the installed mc package, and <dep/...> where dep is in its lock row's deps (or itself). The edge list and lex_root_of (draft § 2, ~35 lines) survive as they were; roots are registered from the lock rows by drv_apply_deps, not from a [include]-like list. The message and the #embed case are unchanged.

A7. Rewrites, itemised. § 2 is replaced by A1-A3 and A6 (named roots gone; the "angle brackets are the bundle, quotes are files" sentence becomes "angle brackets are libraries, quotes are my files"); § 4's cache paragraph becomes ~/.mc/libs/<pack>/v<version>/ + <pack>/v<version>.toml beside it, --pkg-dir becomes --libs-dir everywhere (mc pkg, mc build, mc install, mc upgrade); vendoring keeps deps/<pack>/ and still wins over <libs>; § 6's example becomes #include <geo/geo.mc> and modules = ["<geo/mc_geo.mc>", "user.mc"], and lib stops being informational -- it is the target of a bare <geo> and is copied into the lock row; § 8's last paragraph ("the bundle is the standard library and is not in the registry") becomes: the bundle is the mc package, versioned with the compiler, shipped inside the full binary and installable next to the slim one; its lib/ names are not reserved -- a registry package may replace one, lock-driven -- and its mc/... names are. The M15 promise in docs/build.md § M15 and docs/reference/bundle.md becomes: "the full binary alone is the toolchain; the slim binary plus one mc install is the same toolchain".

B. The slim binary and mc install #

B1. Two release flavours per target. mc (full: today's binary, blob embedded -- the bootstrap SEED on Linux and Windows, scripts/bootstrap-linux.sh:18-26, and the offline/CI default) and mc-slim (the same compiler with an EMPTY bundle). Weight, from M41's measured table (scripts/check-parts.sh, CLAUDE.md § M41): the assembly WITHOUT <mc/core_bundle> is 395 820 B on disk and mc itself is 759 875 B; the bundle part adds ~3.7 KB of __text and ~368 KB of __data (the blob: check-bundle at M41 reports raw 776 601 -> lz 364 543, blob 365 449 B). A slim binary keeps the ~4 KB of reader code and drops the blob, so it weighs ~400 KB, 52% of the full one. The task's "~252 KB" is M15's figure (build/mc-exe 252 316 B without the blob, when the compiler was nine milestones smaller); it is not what a slim binary would weigh today and the spec should not promise it.

B2. How the slim flavour is assembled -- zero core lines. It is docs/reference/bundle.md § "Your own bundle" applied to an empty manifest: src/bundle_empty.mc (checked in, ~8 lines: u64 bundle_blob[1]; i64 bundle_idx[BI_N]; #define BUNDLE_COUNT 0; bundle_end() already returns 0 for a zero count and bundle_cache is sized BUNDLE_COUNT + 2), src/core_bundle_slim.mc (bundle_empty.mc + bundle.mc + bundle_glue.mc), src/bundle_glue.mc (the host_bundle_open

B3. What a slim binary does BEFORE mc install. Everything that needs no <...>: compile a program with no angle-bracket include (i64 main() { return 42; }, or a program with externs and quote includes of its own tree); every --dump-*; mc --host, mc --version; mc build of a project whose sources have no <...> AND no [compiler] section (a taught compiler is #include <mc/host> + <mc/core>, src/driver.mc:414,425, so mc build with [compiler] needs the mc package -- and that is the honest reason mc install must ship mc/core, mc/core_*, mc/host* and every lib/*.mc: check-standalone's equality is the proof that <mc/core> IS the compiler, and a slim binary cannot assemble one without it); mc sysroot, mc limits; mc install itself. What it refuses, with which message, exit 2 in the M25 shape (src/sysroot.mc sysroot_missing's tried:/run: block, docs/reference/cli.md § Exit codes gains one row):

mc: <float> is not installed
  tried: ~/.mc/libs/mc/v0.10.3 (absent)
  run:   mc install --yes

(prog.mc:1: prefix as every include error carries; a stale ~/.mc/libs/mc/v0.10.2/ from another version is not consulted and not mentioned -- per-version directories are the whole point.) The full binary never prints it: step 2 answers first.

B4. mc install [--yes] [--from URL|DIR] [--libs-dir DIR] -- src/install.mc, in <mc/core_build>, registered from mc_build_init() (MAXSUBCMD 16, src/hooks.mc: three used today, seven after this milestone). It populates <libs>/mc/v<mc_version()>/ with the compiler's own package at the compiler's version, from one of two sources:

Without --yes it prints the plan (sysroot_plan's shape: source, size when known, sha256 when known, destination) and nothing was downloaded: re-run with --yes, exit 0 -- M25 D10, same reason (mc has no isatty). A directory whose manifest already exists is mc: mc 0.10.3 is installed (~/.mc/libs/mc/v0.10.3), exit 0, unless --force. mc install never touches another package's directory and never reads a lock: it is about the compiler's own libraries only (mc pkg add is how a registry package arrives; an mc install NAME alias is not adopted, to keep the two meanings apart).

B5. Release assets. scripts/release-assets.sh gains --slim (the archive is mc-<ver>-<target>-slim.tar.gz, the binary inside still named mc/mc.exe -- the ad-hoc signature identifier is the output basename, release.yml "Build the compiler" step) and --libs VERSION BINARY (stages BINARY install --yes --from-bundle --libs-dir OUTDIR/.stage/mc-libs-VERSION, then the same explicit-sorted-members / mtime 0 / ustar / gzip -n -9 rules the script already imposes, producing mc-libs-<ver>.tar.gz + .sha256, host-independent). release.yml: the macOS build job builds dist/mc AND dist-slim/mc (build/mc1 --exe src/mc_slim.mc -o dist-slim/mc, after rm -f), runs scripts/test.sh with the full one, packages both, produces the libs tarball from the full one, cross-compiles the four slim objects next to the four full ones (make mc-linux-slim-obj etc.), and publishes a LATEST file (D2). build-linux / build-windows link the slim object next to the full one and package it with --slim; the bootstrap proof keeps using the FULL binary as its seed (E, risk). publish already globs dist/mc-*.tar.gz -- the slim and libs archives ride along; the release body's install snippet gains the slim road (mc install --yes as the second line). A release therefore carries eleven archives: five full, five slim, one libs, each with its .sha256, plus LATEST.

B6. Does the full binary consult ~/.mc/libs first? Yes, lock-driven only: step 1 runs before step 2 for names the lock pins (A3, A5); step 3 is reached only on a bundle miss, which for the full binary means a name the binary does not ship -- and the installed mc package ships the same names, so it is a miss there too. So the full binary's behaviour with no lock is byte for byte today's, and with a lock it is today's plus the overrides the lock spells out. A full binary that was ALSO mc installed has a ~/.mc/libs/mc/v<ver>/ it never reads for its own includes; what the directory is for on that machine is mc pkg's cache root and the override trees.

C. A version for mc #

C1. Today. The compiler has no version string (docs/ci.md § Versioning: "nothing in the working tree records it", the VERSION file was deleted so that the tags would be the only truth; grep MC_VERSION src/ lib/ is empty). ~/.mc/libs/mc/v<X.Y.Z>/ and mc upgrade both need the running binary to know it.

C2. The baked version. src/version.mc, checked in, six lines:

// the version this binary reports. 0.0.0-dev in the tree; scripts/set-version.sh
// rewrites it from the tag in release.yml and the result is never committed.
uptr mc_version() { return "0.0.0-dev"; }

Included by src/core_min.mc before cli.mc (it is <mc/core_min>'s: cli.mc prints it, install.mc/upgrade.mc in <mc/core_build> consume it) and bundled as mc/version, because <mc/core> includes it and the bundle must stay complete. mc --version prints it and a newline (src/cli.mc:173, next to --host; --host's three lines stay byte-stable -- release.yml and the bootstrap scripts sed them). scripts/set-version.sh X.Y.Z validates the shape with scripts/next-version.sh (the one definition of a version), rewrites the one return line, and runs make bundle so that mc/version inside the blob agrees with mc_version() -- otherwise a taught compiler built by a release binary would say 0.0.0-dev and look for the wrong libs directory. release.yml calls it after make mc1 and before every --exe, cross-compile and --libs step; it does not commit.

C3. The goldens and check-standalone, thought through. src/mc.mc's bytes differ per release -- in the working tree of the release runner only. The tag's TREE keeps 0.0.0-dev, and every golden (tests/golden/mc2.sha256, the two Linux and the two Windows ones) is the hash of an object compiled from the checked-in tree, so they do not move per release and are rewritten only when src/ changes on purpose, as today. The release proofs still hold: scripts/bootstrap-linux.sh takes the versioned binary as SEED, compiles the checked-out tag (dev) to mc1l, which compiles it again to mc2l.o -- the dev object, against the dev golden -- and the script already allows the seed to differ from mc1l ("the seed may legitimately be an older compiler", docs/bootstrap.md § The Linux chain, point 3; what must agree is mc1l --dump-asm vs mc2l --dump-asm, both dev). The macOS release job runs scripts/test.sh dist/mc, no golden. check-standalone compares a binary against an object compiled from the SAME tree by the same mc1; on the dev tree both say 0.0.0-dev, on a release runner (if it were run there) both would say the tag; equality holds either way. Two guards so the sentinel cannot leak: scripts/check-bundle.sh asserts src/version.mc says 0.0.0-dev (a rewritten file fails make check with a message naming set-version.sh), and .github/workflows/ci.yml runs make check on the untouched tree as it does now. docs/ci.md § Versioning is amended, not reversed: the tag is still the only truth for RELEASES; the tree carries a constant sentinel, not a version, and the objection to two sources of truth does not apply to a constant.

C4. The dev build. mc_version() is the constant 0.0.0-dev -- not the commit hash, which would move src/bundle_data.mc, build/mc2.o and all five goldens on every commit. Its libs directory is ~/.mc/libs/mc/v0.0.0-dev/; a dev FULL binary populates it from its blob (make check's slim proof uses --libs-dir in a temporary directory and never touches HOME). For ordering, 0.0.0-dev compares as 0.0.0: every release is newer.

D. mc update and mc upgrade #

D1. Placement. install, update and upgrade are top-level subcommands (subcommand() registrations, one usage line each, mc with no argument lists them after build|limits|sysroot), consistent with mc build and mc sysroot fetch; the package-author and maintenance verbs stay under mc pkg (sync add list vendor verify hash check), and the draft's mc pkg update is REMOVED in favour of mc update [NAME] [DIR] [--config FILE] [--yes] [--registry ...] [--libs-dir DIR], same semantics (go get -u: raise the [deps] minimum(s) to the newest non-yanked index version, then sync). install and upgrade live in <mc/core_build> (src/install.mc, src/upgrade.mc): a compiler without core_pkg -- the CI/consumer shape of D12 -- still installs its own libraries and updates itself. update lives in <mc/core_pkg> (src/pkg.mc, registered from mc_pkg_init() beside pkg): it needs the index and MVS. Semver parsing/comparison moves from pkg.mc to deps.mc (<mc/core_build>), where upgrade can reach it.

D2. mc upgrade [--yes] [VERSION] [--from URL|DIR] [--libs-dir DIR] -- the self-update, in the M25 order (plan, --yes, download, verify BEFORE unpack, act, claim last):

  1. Which version. With no VERSION: fetch https://github.com/schivei/mc/releases/latest/download/LATEST -- GitHub resolves releases/latest/download/<asset> to the newest release's asset by redirect, which -fLsS + --proto-redir =https already follow (sysroot_download) -- a one-line text file X.Y.Z\n that release.yml's publish job attaches (gh release create ... dist/LATEST). Chosen over curl -sI ... -w '%{redirect_url}': the -w form is curl-only (the wget fallback has no equivalent), and a one-line file is parsed by the same ver_parse that validates every version, whereas a Location: header is a URL to be cut. The GitHub API is JSON, which mc does not parse. With a VERSION: that one, validated.
  2. Refusals, before any download: mc: 0.10.3 is the newest release (latest == current), exit 0; mc: 0.10.1 is older than this binary (0.10.3): name it with --yes to downgrade (an explicit older VERSION without --yes), exit 2; a LATEST older than the running version is refused even with --yes (a stale or tampered file, loudly, exit 2); a dev build (0.0.0-dev) refuses unless --yes VERSION names one (so make check never clobbers build/mc-exe, and the acceptance test can).
  3. The plan, then --yes: current, target, the asset URL for this host pair and this FLAVOUR (BUNDLE_COUNT == 0 means slim: fetch -slim), the destination path (the running binary's own path), and nothing was downloaded: re-run with --yes.
  4. Download mc-<ver>-<target>[-slim].tar.gz + .sha256 into <libs>/mc/upgrade.<ver>/ through fetch_get; <target> is the release vocabulary (macos-arm64, linux-arm64, linux-x86_64, windows-arm64, windows-x86_64: host_os() + an aarch64 -> arm64 map of six lines, the "two vocabularies" docs/bootstrap.md records). Verify the archive's SHA-256 before unpacking (src/sha256.mc, against the .sha256 line), then tar -xzf F -C DIR --strip-components=1 mc-<ver>-<target>/mc -- one member, sysroot_extract's member-list shape.
  5. Swap, per host, by host_self_path() (new in the host layer: _NSGetExecutablePath on macOS, readlink("/proc/self/exe") on Linux, GetModuleFileNameA(0, ...) on Windows -- one extern each, the GetEnvironmentVariableA precedent in src/host_windows.mc; argv[0] is a PATH lookup and not a path) and host_retire(path):

    • macOS: unlink(self), then write the new bytes to the SAME path with creat(..., MODE_755). A new inode: the running process keeps the old one, and the kernel's cached-signature Killed: 9 (CLAUDE.md § M12) happens only when a signed file is overwritten IN PLACE -- which is why drv_compile unlinks first (src/driver.mc:355, "never rewrite a signed binary in place") and why release.yml does rm -f dist/mc. The downloaded binary carries its own ad-hoc signature from the release build and was written as mc; no codesign is run and no quarantine attribute is set (the file is written by mc, not by a browser).
    • Linux: the same unlink + write (renaming over is also fine; not needed).
    • Windows: a running .exe cannot be deleted (docs/bootstrap.md § The Windows chain; lib/sys_windows_host.mc:280's unlink is DeleteFileA) but it can be RENAMED: MoveFileExA(self, self + ".old", MOVEFILE_REPLACE_EXISTING) -- one more kernel32 name in scripts/sysroot-windows.sh's list (:79-82) and one extern in src/host_windows.mc -- then write the new bytes to self. The next mc upgrade unlinks a leftover mc.exe.old first; mc --version does not.
  6. Then the libraries: spawn <self> install --yes [--libs-dir DIR] -- the NEW binary, through drv_spawn_ok (posix_spawnp + waitpid), because the running process is the old version and its mc_version() is the wrong directory. A full binary installs from its blob (no second download); a slim one fetches mc-libs-<ver>.tar.gz. Exit is the child's.
  7. Print mc 0.10.2 -> 0.10.3 (/usr/local/bin/mc); remove the download directory.

D3. The supply-chain gap, named. Every check above is against a .sha256 fetched from the SAME origin as the archive: that proves the bytes arrived intact, not that they were published by the owner. Anyone who can serve github.com/schivei/mc/releases/... to this machine can serve a matching checksum. The priced follow-up is a signing key -- minisign or ssh-keygen -Y sign over the .sha256 files, the public key baked into src/version.mc next to the version and rotated with a release, ~90 lines of verification in .mc (Ed25519, no libc) or one more spawned tool -- and mc upgrade refusing an unsigned release once a key ships. Until then packages.md and docs/ci.md say in one sentence what the .sha256 does and does not prove; mc pkg's tree hashes are a different matter (they are pinned in the lock by the developer who reviewed the tree).

E. Files, acceptance, risks -- reworked #

Files and estimated deltas (replaces the draft's table; unchanged rows kept for completeness):

filelineswhat
src/version.mc~6, newmc_version(); the 0.0.0-dev sentinel
src/core_min.mc+1version.mc before cli.mc
src/cli.mc+4--version; the usage line
src/lex.mc+60lopen_fn/lex_set_libs; lex_include_name = three steps + .mc strip; tok_add(".", 1) last in tok_init; lex_root_of + edges + the closure test in lex_include/#embed; the post-parse files accessor. The draft's named-root branch in lex_find_path_from is NOT added
src/fetch.mc~150, newfetch_get (https via the host's downloader, else local copy), fetch_extract, fetch_sha256_line (parse a .sha256 file), hex64
src/sysroot.mc-75 / +12delegates to fetch_*
src/deps.mc~400, newname rule + reserved mc; [deps]/[replace]/[registry]; lock reader (rows carry lib); tree hash; libs_open (steps 1 and 3 of A3, the bundle.list map, lazily read once); semver; edge registration; the § 3 refusals; the B3 message
src/install.mc~260, newinstall_cmd: plan, --yes, --from, --libs-dir, --force; the from-blob road (the one definition of the layout); the tarball road; the manifest
src/upgrade.mc~250, newupgrade_cmd: LATEST, refusals, plan, download + verify + one-member extract, the per-host swap, the spawned install
src/pkg.mc~700, newindex, MVS, lock writer, archive fetch + hash + manifest, vendor, [deps] edit, hash, check, list, verify, pkg_cmd; update_cmd
src/core_pkg.mc~24, newcore_build.mc + pkg.mc; subcommand("pkg", ...), subcommand("update", ...)
src/core_build.mc+6fetch deps install upgrade in the list; install/upgrade registrations
src/bundle_glue.mc, src/core_bundle.mc~14 new / -10 +1host_bundle_open + mc_bundle_init shared by the two bundle parts
src/bundle_empty.mc, src/core_bundle_slim.mc, src/core_slim.mc~8 / ~12 / ~14, newthe slim assembly (B2)
src/mc_slim.mc + 4 host slim entries; 4 *-slim-obj.toml~5 each / ~12 each, newthe ten release flavours' entries
src/host_macos.mc, host_linux.mc, host_windows.mc+10 / +10 / +16host_self_path, host_retire; MoveFileExA, GetModuleFileNameA externs
scripts/sysroot-windows.sh, lib/sys_windows_host.mc+2 / 0the two kernel32 names
src/core.mc, src/main.mc+1 / +1the sixth part; mc_pkg_init()
src/toml.mc+38toml_push/toml_pop, toml_occurrences
src/driver.mc+45--libs-dir; drv_apply_deps for both halves; <...> modules emitted verbatim; the post-parse files check
tools/bundle.list, src/bundle_data.mc+13, regeneratedmc/version, mc/fetch, mc/deps, mc/install, mc/upgrade, mc/pkg, mc/core_pkg, mc/bundle_glue, mc/bundle_empty, mc/core_bundle_slim, mc/core_slim, mc/bundle.list, the five slim entries are NOT bundled (they are entries, like mc/main... mc/main is; add mc/mc_slim only if a recreated slim compiler wants it -- not this milestone)
Makefile+30mc-slim, mc-linux-slim-obj, mc-linux-x86_64-slim-obj, mc-windows-slim-obj, mc-windows-x86_64-slim-obj, libs-tarball, check-pkg, check-slim, check-upgrade inside check
scripts/set-version.sh~40, newrewrite src/version.mc from a tag, make bundle
scripts/release-assets.sh+50--slim suffix; --libs mode via BINARY install --yes --from-bundle
scripts/check-bundle.sh+6the 0.0.0-dev sentinel guard
scripts/check-slim.sh~190, newAcceptance 20-23
scripts/check-upgrade.sh~160, newAcceptance 24-26
scripts/check-pkg.sh~340, newAcceptance 1-19
scripts/check-parts.sh, check-docs.sh+10 / +3<mc/core_pkg>, <mc/core_bundle_slim>; the new families and flags
.github/workflows/release.yml+80set-version.sh; slim builds + cross objects; --libs; LATEST; slim links on the Linux/Windows runners; the install snippet
.github/workflows/ci.yml+2build/mc-slim in the uploaded artifacts
tests/pkg/... fixtures, tests/golden/pkg-list.txt, .gitattributesas in the draftspelling <geo/geo.mc>; the float override fixture (tests/pkg/src/float-1.3.0/)
docs/reference/packages.md~500, neweverything above: the resolution order, the layout of ~/.mc/libs, install/update/upgrade, the slim binary, the version, the trust model
docs/reference/cli.md+70install, update, upgrade, pkg, --version, --libs-dir; the exit-2 rows
docs/reference/bundle.md, docs/build.md+45 / +30"the full binary alone"; the mc package; mc/bundle.list; the slim assembly
docs/ci.md+90§ Versioning amended; the eleven assets; LATEST; set-version.sh; what .sha256 proves
docs/bootstrap.md, tests/golden/README.md, docs/determinism.md+15 / +10 / +12the seed stays full; the dev sentinel and the goldens; the lock and the libs dir as inputs
docs/guide/25-packages.md, docs/reference/toml.md, docs/reference/diagnostics.md~270 / +80 / +75as in the draft, plus the three verbs
schivei/mc-registryas in the draftcheck.yml refuses name = "mc" and any mc/...
stage0/0

Net new src/ lines ~1 700 (the draft's ~1 350 plus install, upgrade, the slim assembly and the host additions). The five goldens move once per gated commit, under M41's note 9.

Acceptance (ordered; scripts/check-pkg.sh, check-slim.sh, check-upgrade.sh; no network anywhere; a curl shim on PATH that fails if invoked, for every step that must not download):

1-4. As in the draft (the fixture registry built by the script; the hash stable across hosts; the plan printed and nothing fetched; MVS not "latest", the lock byte-identical), with --libs-dir and $tmp/libs/<pack>/v<version>/.

  1. <geo/geo.mc> resolves to the locked version: mc build tests/pkg/app --libs-dir $tmp/c1 builds the taught compiler from <teach/mc_teach.mc> + user.mc, compiles main.mc (which includes <geo/geo.mc>, whose #include <mathx/mathx.mc> resolves through the closure; <geo> alone resolves to geo.mc through the lock's lib), and the binary's stdout/exit match main.mc's header. With $tmp/libs/geo/v1.0.0/ ALSO present, the object is cmp-identical to one built with only v1.2.0/ present: the unlocked directory was never opened. 6-9. As in the draft (byte-identical objects from two fetches; the tampered source refused, exit 2; stale lock / stale tree, no curl; vendoring is the offline road, deps/<pack>/ wins).
  2. A package is closed, spelled with <bad/bad.mc>; the #embed case; not declared in geo's [package].files.
  3. Names: [deps] mc = "1.0.0" -> mc: reserved; Geo = ... -> invalid package name; exit 1 at the key. [deps] float = "1.3.0" is ACCEPTED (see 18). 12-17. As in the draft (majors refused; a failed fetch leaves no claim; mc pkg add edits one line; mc pkg check is the gate, and refuses name = "mc"; parts -- a probe compiler without core_pkg builds the vendored app and prints a usage with no pkg/update line but WITH install/upgrade; inert -- check-inert clean, check-standalone, check-obj 32/32, check-build, check-sysroots unchanged, mc sysroot fetch linux-aarch64 --yes in CI still writes the four files).
  4. A bundled name pinned in [deps] overrides the bundle, byte-checked: the fixture package float 1.3.0 (a copy of lib/float.mc + float_rt.mc with one visible change: putf64 prints a ! suffix) locked by tests/pkg/app-float/; mc build with the FULL binary produces a program that prints the !; the object is cmp-identical between the cache road and the vendored road; one byte appended to $tmp/libs/float/v1.3.0/float_rt.mc -> exit 2 float 1.3.0: float_rt.mc does not match mc.lock; removing the [deps] float line and re-syncing gives an object cmp-identical to one built by a checkout with no tests/pkg at all -- the override leaves nothing behind.
  5. No lock, no network, the FULL binary resolves <float>: build/mc-exe --exe on lib/mc_float.mc's shape in an empty directory with HOME unset and the curl shim -- the existing check-standalone steps, re-run with the shim, plus <float>.
  6. The SLIM binary before mc install: build/mc-slim (from src/mc_slim.mc, ~400 KB, asserted < 60% of build/mc-exe's size) compiles i64 main() { return 42; } to an object cmp-identical to build/mc-exe's; --dump-asm of src/arena.mc identical between the two; mc-slim --exe hello.mc (<sys> + <prelude>) exits 2 with <sys> is not installed / tried: / run: mc install --yes; mc-slim install --libs-dir $tmp/libs (no --yes) prints the plan and nothing was downloaded, and $tmp/libs holds no manifest.
  7. mc install from a local tarball, and the standalone proof extended: make libs-tarball (release-assets.sh --libs 0.0.0-dev build/mc-exe dist) writes dist/mc-libs-0.0.0-dev.tar.gz

    • .sha256, reproducibly (two runs cmp equal); mc-slim install --yes --from dist --libs-dir $tmp/libs populates $tmp/libs/mc/v0.0.0-dev/ and writes v0.0.0-dev.toml last; then, in an empty directory with the shim on PATH, mc-slim runs every step of check-standalone.sh with --libs-dir $tmp/libs: hello runs (exit 42), the taught compiler from <mc/host> + <mc/core> + <user_syntax_demo> is built and signed, compiles <syntax_demo_test>, and <mc/host> + <mc/core> + <user_default> compiles to an object cmp-identical to build/mc2.o -- the slim binary plus the installed package is the compiler, byte for byte. A tarball with one flipped byte in its .sha256 -> checksum mismatch for mc-libs-0.0.0-dev.tar.gz, no directory, no manifest; a second install after success says is installed, exit 0.
  8. The FULL binary's mc install needs no network: build/mc-exe install --yes --libs-dir $tmp/libs2 with the shim; the tree is diff -r-identical to the one step 21 unpacked (the tarball came from the same road, so this is a reproducibility check of the road itself); src/bundle_data.mc in it is the mode-1 form and src/bundle.bin has bundle_bin_size() bytes.
  9. mc --version prints 0.0.0-dev for build/mc-exe and build/mc-slim; a tree copied to $tmp/tree with scripts/set-version.sh 9.9.9 applied (in the copy) and compiled with build/mc1 --exe $tmp/tree/src/mc.mc -o $tmp/rel/mc prints 9.9.9; scripts/check-bundle.sh on the COPY fails naming the sentinel, on the tree passes.
  10. mc upgrade against a local release directory, no network: release-assets.sh 9.9.9 macos-arm64 $tmp/rel/mc $tmp/rel and --libs 9.9.9 $tmp/rel/mc $tmp/rel produce the two archives; cp build/mc-exe $tmp/bin/mc; $tmp/bin/mc upgrade --from $tmp/rel --libs-dir $tmp/libs3 (no --yes) prints the plan naming $tmp/bin/mc and downloads nothing; $tmp/bin/mc upgrade --yes 9.9.9 --from $tmp/rel --libs-dir $tmp/libs3 (the explicit form, since the running binary is a dev build) swaps the file, spawns the new binary's install, and afterwards $tmp/bin/mc --version prints 9.9.9, $tmp/libs3/mc/v9.9.9.toml exists, and $tmp/bin/mc compiles hello.mc through --libs-dir $tmp/libs3.
  11. The macOS in-place-overwrite hazard does not occur: stat -f %i $tmp/bin/mc before and after step 24 differ (a new inode), codesign --verify --verbose=4 $tmp/bin/mc passes, and $tmp/bin/mc --version exits 0 -- not Killed: 9. On the Windows CI legs the same script asserts mc.exe.old exists after the swap and is gone after a second upgrade (a no-op one, is the newest release).
  12. Refusals: upgrade --yes 0.0.1 --from $tmp/rel on the 9.9.9 binary without --yes... as specified: an older explicit VERSION without --yes -> older than this binary, exit 2, no file touched (inode unchanged); a LATEST file saying 0.0.1 against the 9.9.9 binary -> refused even with --yes; a tarball whose .sha256 does not match -> checksum mismatch, the binary untouched.
  13. CI builds both flavours and make check is green with check-pkg, check-slim, check-upgrade inside it; check-parts shows <mc/core_min> + <mc/core_pkg> and <mc/core_min> + <mc/core_bundle_slim> compile alone; goldens rewritten once per gated commit after an empty --dump-asm diff and cmp build/mc2.o build/mc3.o; make check-docs green.

Risks (in addition to the draft's 1-12, which stand; 10 is closed by C):

  1. A stale ~/.mc/libs from another version. Closed by construction: the mc package lives under v<mc_version()>/ and nothing else is consulted; the B3 message names the directory it looked for. What is NOT closed: disk growth across versions -- mc install --prune (remove every mc/v* but the running one) is ~25 lines and optional.
  2. The seed of the bootstrap must stay the FULL binary. bootstrap-linux.sh/-windows.sh download mc-<VER>-<target>.tar.gz (the unsuffixed name) and stay as they are; the chain proper (src/mc_linux.mc has no <...>) would in fact close with a slim seed, but the release gate runs the whole suite with the seed and must not depend on ~/.mc or on a second download. docs/bootstrap.md says so in one paragraph.
  3. CI must build both flavours or a slim-only breakage (the empty-bundle assembly, lopen_fn step 3) ships unnoticed; check-slim is inside make check and release.yml links the slim object on every runner. Cost: five more cross-compiles and five more links per release, ~1 minute.
  4. The slim binary and check-standalone. The original script keeps proving the FULL binary; the slim proof (step 21) is a second script because its precondition (a libs tarball) is not the empty directory the original insists on. A drift between the two proofs is a documentation bug to watch: both compare against build/mc2.o.
  5. The . token. Appended last in tok_init, so no id shifts; but a taught module that relied on . being ABSENT (a #token "." of its own is fine -- tok_add is idempotent -- but a syntax_lit/on_stmt that saw 1 . 5 as three tokens on purpose is not) would change behaviour. No module in the tree does; check-lang, check-float, check-surface are the net.
  6. mc upgrade is the widest new attack surface: it writes over the compiler. D3 names what the .sha256 proves; until a signing key ships, packages.md says "mc upgrade trusts the release host". --from with a local directory is also how a reviewer can stage an upgrade.
  7. set-version.sh leaking into a commit moves the goldens and the bundle silently; check-bundle's sentinel guard makes it a red make check.
  8. Windows mc.exe.old: MoveFileExA on a running executable succeeds, but an antivirus holding the file open can make the rename fail; the message names the file and asks the user to re-run, nothing is half-written (the new bytes are written only after the rename returns).
  9. host_self_path() and symlinks: /usr/local/bin/mc -> /opt/mc/0.10.2/mc gets the resolved TARGET replaced (that is what _NSGetExecutablePath/readlink return), not the link; documented, not resolved. A binary on a read-only path fails at creat with cannot write naming the path, exit 2.
  10. A taught compiler built by the slim binary carries the full blob (its src/bundle_data.mc on disk is the whole mc package): expected -- it is the same object the full binary builds (step 21) -- but a user who chose slim for size gets full-size taught compilers unless the project says core = "<mc/core_slim>", which is why core_slim is bundled.

F. Amended decisions -- to ratify with the owner #

Architect's additions, extended: (d) step 24's local-release test is also the documented road for an air-gapped upgrade (--from DIR), and packages.md shows it; (e) the release body's install snippet lists the slim road second, never first -- the full binary stays what a newcomer downloads; (f) the first commit of this milestone is src/version.mc + --version + the sentinel guard alone, so the goldens move once for a six-line change and every later commit can be checked against a versioned binary.


Edit this page