Spec M44 -- packages: source distribution, Go-style, from a registry of sources
Owner's request (2026-09-04, translated from the Portuguese): "a package manager for mc, in the
style of Go: it provides the source, and the manager is the repository of registered sources."
Read as three
requirements: (1) a package is SOURCE, fetched as source and compiled by mc like everything
else; (2) the model is Go's -- a path-identified source tree, versioned by git tags, minimums in
the manifest, a lock that pins content; (3) the manager is a REGISTRY OF SOURCES -- an index that
maps a name to where the source is and which versions exist, never a host of binaries.
Goal: [deps] in mc.toml, mc.lock beside it, mc pkg as a sixth bundled part, a registry
that is one git repository of TOML files, and a build that never reaches the network. Sequencing:
after M41 (the parts and subcommand()), after M42's fix batch if it lands first (no dependency
either way); M43 (the sandbox) is the milestone that makes an UNTRUSTED package's compiler module
safe to run and is named below as the seam, not as a prerequisite.
What already exists #
- The host layer has exactly what M25 needed and nothing more:
open/creat/read/write/close,mkdir,unlink,posix_spawnp/waitpidwith a stdout file action (src/host_macos.mc:35-45,src/driver.mc:220-277),host_home(),host_downloader()/host_downloader_alt(). Noopendir, norename, nogetenv, no HTTP, no TLS. Every design choice below that looks odd -- an explicit file list in the manifest, a hash over listed files, a manifest written after verification instead of an atomic rename -- follows from this line. mc sysroot fetchis already a package fetcher for one fixed package list (src/sysroot.mc:456-749): spawncurl --proto '=https' --proto-redir '=https' -fLsS(orwget),sha256fromsrc/sha256.mc:132over the bytes, size check, onetarspawn with--strip-components, every archive member probed, marker files probed, and only THENmanifest.toml(no date,docs/determinism.md).sysroot_download,sysroot_extract,sysroot_hexare the three functions M44 generalises;sysroot_extractis the only one bound to a row index ofsrc/sysroots.mc.- The pinned-row shape (
src/sysroots.mc:24-40:target host kind url sha size strip member) is a registry version row already.docs/reference/sysroot.md§ 8 mirrors it andscripts/check-sysroots.shdiffs the two with no network. [include].pathsis an anonymous root list (src/lex.mc:509-551): tried in order, only after the includer's own directory fails, "so a project never shadows a relative include that already resolved". Roots are stored with a trailing/and go throughpath_join/path_norm.#embedresolves through the same function.--include=DIRis the CLI form (src/cli.mc:230).#include <name>has no filesystem fallback, on purpose (src/lex.mc:619-626,docs/build.md§ M15: "<name>means the copy that shipped with this binary, and the answer must not depend on the working directory"). The bundle catalogue is 65 names + 2 regenerated (docs/reference/bundle.md), all reachable through the lexer's one pointerbopen_fn.toml_parsefills ONE global table (src/toml.mc:65,394-403); M25 Decision 3 put the sysroot rows in a.mctable for that reason, and M23 wrotelim_read_usage(src/limits.mc:252) as "its own tiny reader on purpose". M44 needs to read four kinds of TOML in one run (project, lock, registry entry, package manifest); a third tiny reader is the road not to take.mc.tomlis already rewritten in place bymc:lim_fix_write(src/limits.mc:515) replaces one key in one section and copies every other byte through. That is the precedent formc pkg addtouching[deps].mc buildis two processes (src/driver.mc:599-652): the taught compiler is generated from[compiler].core+[compiler].modulesand SPAWNED.drv_apply_configruns for the entry only (cfg = 1), never for the compiler (:315-316). A compiler-module package must reach the first half; a library package the second; so the[deps]roots apply to both and[libs]/[externs]keep applying to one.- Subcommands are registrations (
src/hooks.mc:773-830,MAXSUBCMD 16, three used) and a part registers its own (src/core_build.mc:34-47).src/core.mcis six include lines andsrc/main.mc:29-36names every part's*_init. A sixth part is one line in each. - A library and a compiler module are the same thing in two files:
lib/float.mc(registers types, intrinsics; a MODULE) +lib/float_rt.mc(a runtime a PROGRAM includes) + a six-linelib/user_float.mcthat defines the oneuser_init.examples/conc/mc.tomlstacks two module files from two directories (modules = ["../lang/lang.mc", "conc.mc"]) and pays for it with thelg_morechain, because a compiler holds exactly oneuser_init. - Versions are git tags,
vX.Y.Z, no pre-releases (docs/ci.md§ Versioning,scripts/next-version.shrejects0.2.0-rc1); the compiler carries no version string at all (grep -rn MC_VERSION src/ lib/is empty; the tags "are the only source of truth"). - Exit code 2 is "the environment is not ready" and is one message shape (
mc: no sysroot for .../tried:/run:),docs/reference/cli.md§ Exit codes,diagnostics.md§ 11. - The consumer. The
ngenport of teko (M41.5.md§ 1 in the review worktree) is written "againstdocs/against an unmodified compiler": a set of.mccompiler modules plus a runtime. It is the first package of the second kind, and it is exactly whatexamples/conc's../lang/lang.mcrelative path does today by hand.
Design #
1. Identity, names and versions #
A package is a source tree with an mc.toml at its root carrying a [package] table. Its
identity in Go is its import path (github.com/user/pkg, a URL prefix); in mc it is a
registry name, because the owner asked for the registry to BE the manager: the name is what
the index maps to a location. The location is a detail of the index row, not of the source that
uses the package.
Name rule: [a-z][a-z0-9_]*, at most 32 bytes. Why that set: it is a bare TOML key
(src/toml.mc: A-Za-z0-9_-, minus - and upper case so the name is also a valid identifier
prefix, geo_init), a valid path component on all three hosts, and a name that cannot collide
with the bundle's mc/... namespace. A name that the bundle serves is refused (float,
sys, i128, prelude, ...): the check is bopen_fn through the lexer, the same pointer
limits.mc uses, so <mc/core_build> never depends on <mc/core_bundle>. Reserved outright:
mc, deps, build.
Versions are semver X.Y.Z, one git tag vX.Y.Z per version (docs/ci.md § Versioning applied
to packages; no pre-releases, for the reason next-version.sh gives). [deps] geo = "1.2.0"
means at least 1.2.0 -- Go's require semantics, not an exact pin; the exact version is the
lock's business (§ 3). A requested version must exist in the index (Go requires the same).
2. The import spelling changes the language by zero lines #
A dependency geo is included as
#include "geo/geo.mc"
-- the quote form, unchanged. The lexer gains named roots: lex_add_named_root(name, dir),
consulted in lex_find_path_from AFTER the includer's own directory and BEFORE the anonymous
[include].paths roots: when rel begins with <name>/ and <name> is a registered root, the
rest is joined to that root's directory. M14's rule survives verbatim ("a project never shadows a
relative include that already resolved"), and the match is exact on the first component, so two
packages' files with the same basename cannot shadow each other -- the one failure mode a flat
deps/ root added to [include].paths would have.
<name> stays what docs/build.md promises: the copy that shipped with this binary, no
filesystem fallback. The boundary is one sentence: angle brackets are the bundle, quotes are
files on disk, and a dependency is files on disk. The bundle/package collision question
dissolves at the syntax level and is closed at the registry level by the reserved-name rule.
Go's analogue: the standard library is not a module; fmt and github.com/x/y share one import
syntax but the toolchain never fetches the former.
For a compiler module the same spelling goes into [compiler].modules:
[compiler]
modules = ["teach/mc_teach.mc", "user.mc"]
drv_gen_compiler writes #include "teach/mc_teach.mc" into the generated file (a value with no
leading / and a first component that is a named root is emitted as written, not ../-adjusted:
~6 lines in drv_include), and the named roots are registered for BOTH compilations
(drv_parse, before drv_apply_config's cfg test).
A package is closed. A file under a package's root may include or #embed only: its own
tree, the bundle (<...>), and the roots of the packages its lock row names as deps. A resolved
path that lands anywhere else -- the project's files, an absolute path, another package it did
not declare -- is geo/vec.mc:3: package geo reaches outside its tree: /etc/hosts, exit 1. ~35
lines in lex.mc (lex_root_of(path) = longest registered root that is a string prefix of the
normalised path; an edge list (from_root, to_root) in registration order). This is the M15
"no filesystem fallback, on purpose" stance applied to packages, and it is the cheapest honest
answer to #embed as an exfiltration primitive (§ Risks 1).
3. The lock, the tree hash, and what "deterministic" means here #
mc.lock, beside mc.toml, written only by mc pkg, rows sorted by name (bytewise, an insertion
sort on unique keys -- rule 2 of docs/determinism.md forbids qsort's tie-breaking, not
ordering; the sort key is total):
# written by `mc pkg sync` -- do not edit (docs/reference/packages.md)
[[package]]
name = "geo"
version = "1.2.0"
sha256 = "9f1c...e2" # tree hash, below
deps = ["mathx"]
[[package]]
name = "mathx"
version = "1.1.0"
sha256 = "41b0...7a"
deps = []
Not inside mc.toml: the lock is machine-written and complete, mc.toml is hand-written and
minimal, and lim_fix_write shows how much care a machine edit of a human file costs. Go keeps
go.sum beside go.mod for the same reason.
The tree hash is a dirhash-style content hash -- the shape of Go's dirhash.Hash1, but written
in plain hex, not Go's base64 h1: form -- in manifest order: for mc.toml first and then each entry of [package].files in the order written, one
line hex(sha256(file bytes)), two spaces, the path, \n; the hash is sha256 of those lines.
Go sorts the file list because it derives it from a zip; mc cannot list a directory, so the
package AUTHOR lists the files (the precedent is tools/bundle.list and the member column of a
sysroot row), and manifest order is the canonical order. The hash is of CONTENT, never of the
archive: GitHub's archive/refs/tags/*.tar.gz bytes changed under a git upgrade on 2023-01-30,
broke every archive-checksum consumer (Homebrew among them) and were rolled back -- a policy,
not a format guarantee. A content hash is also the same function for all three ways a tree can
arrive (fetched, vendored, replaced by a path), which is what lets mc pkg verify be one
routine.
The lock is checked, not trusted. mc build rehashes every locked package on every build
(sha256.mc already hashes the whole output executable per 4 KiB page on every --exe; a
dependency's source is smaller than that and is about to be lexed anyway) and refuses on any
disagreement:
| disagreement | message | exit |
|---|---|---|
| a file's bytes differ from the cache manifest's line | mc: geo 1.2.0: vec.mc does not match mc.lock | 2 |
the tree hash differs but no file line does (the files list changed) | mc: geo 1.2.0: mc.toml does not match mc.lock | 2 |
[deps] names a package the lock lacks, or asks a minimum above the lock | mc: mc.lock is stale: run mc pkg sync --yes | 2 |
| the lock names a version that is neither vendored nor cached | mc: geo 1.2.0 is not fetched + run: mc pkg sync --yes | 2 |
a file the build READ under a package root is not in that package's files | geo/extra.mc:1: not declared in geo's [package].files | 1 |
Exit 2 because every one of these is "the environment is not ready" in the M25 sense, and a
script must be able to tell it from "your program does not compile". The last row is the
post-parse walk of the lexer's once-only list (inc_at, src/lex.mc:557) and is what makes the
files list a real boundary rather than documentation.
Version selection is MVS, precisely Go's (cmd/go/internal/mvs, Russ Cox 2018): the build
list starts from the project's [deps]; for every selected (name, version) the requirements of
THAT version are added; a name's selected version is the maximum over every minimum that
mentions it; repeat to a fixed point. No SAT, no search, no "latest": the answer is a function of
the manifests alone, and the lock freezes it so that the index can move afterwards without
moving the build. The registry row carries each version's requirements inline (Go's proxy serves
.mod files before .zip for the same reason), so MVS runs over the index snapshot with no
archive download. Semver comparison is the --gt arithmetic of scripts/next-version.sh,
rewritten in .mc (~20 lines).
Majors. Go handles a major bump with semantic import versioning (/v2 in the path). mc does
not: if the selected version's major differs from the major of ANY requirement that named it,
the resolution is refused -- mc: mathx: 1.1.0 and 2.0.0 are different majors: no solver. Two
majors of one name in one build is the case MVS is not designed for, and pricing a /v2 scheme
is out of scope.
Yanked. A registry row may gain yanked = true (Go's retract, 1.16) and nothing else may
ever change in a row. mc pkg add|update skip yanked versions; sync of a lock that already
pins one warns and proceeds, so a build never breaks retroactively.
4. Fetching: the M25 road, generalised #
src/fetch.mc (in <mc/core_build>, included before sysroot.mc) takes the three functions out
of sysroot.mc in their general form: fetch_get(src, file) -- an https:// source spawns the
downloader exactly as today (--proto '=https' --proto-redir '=https' -fLsS, then the wget
fallback), and a source with no scheme is a local path copied with read_file/write_file;
fetch_extract(archive, dest, strip, members); hex64(digest). sysroot_extract becomes a
four-line call into it. The local-path branch is what makes the test suite need no network and
what prices a private registry at zero (§ 5).
Tarballs over git: tarballs. A GitHub tag has a URL
(https://github.com/<u>/<r>/archive/refs/tags/v1.2.0.tar.gz, top directory <r>-1.2.0, hence
strip = 1), tar -xzf reads it on all three hosts (M25 § 2 verified bsdtar, GNU tar and
Windows' libarchive tar.exe on gzip), and git stays off the dependency list -- the same
argument M25 made for curl over an HTTP client, one level up. The index row carries the url
explicitly, so a non-GitHub forge is a different URL and nothing else.
Cache: host_home()/.mc/pkg/<name>/<version>/ holds the extracted tree and
host_home()/.mc/pkg/<name>/<version>.toml (beside, not inside, so a package file named
manifest.toml cannot collide) holds [source] name version url sha256 and one [[file]] path
sha256 per hashed file. --pkg-dir DIR on mc pkg and mc build overrides the root, as
--sysroot-dir does, so CI depends on no HOME. Order of operations, unchanged from M25 because
there is no rename: download into <pkgdir>/<name>/<version>.tar.gz, extract into the final
directory, hash, and on any failure unlink every listed file (the sysroot_unbless idea) and
write no manifest; the manifest is the claim that the directory holds what the lock says, written
last. The tree is hashed and compared right after extraction, before anything else: the
archive itself is not checksummed (§ 3 says why), so unlike M25 the refusal comes after the
bytes are on disk -- but still before any manifest exists and before any build can consume the
tree, and a refused tree is unlinked.
Vendoring: mc pkg vendor copies each locked package (mc.toml + files) into deps/<name>/
in the project. When deps/<name>/ exists it is used and the cache is not consulted (Go: a
vendor/modules.txt consistent with go.mod makes -mod=vendor the default since 1.14). Both
forms are hashed against the lock, so the choice cannot change bytes silently. deps/ plus
mc.lock in git is the fully offline project.
Replacement: [replace] geo = "../geo" points a name at a local tree for development (Go's
replace directive). A replaced package is registered as a named root from that path, its lock
row says path = "../geo" and carries no hash, and mc pkg sync|list print
1 replaced dependency: not pinned by mc.lock. Go's go.sum also omits path-replaced modules.
mc build never downloads (M25, architect's addition (a)). It reads the lock, finds each tree
in deps/ or the cache, hashes, registers roots, compiles. Nothing else.
5. The registry: one git repository of TOML #
Compared: Go's proxy.golang.org (immutable zips served by a service) + sum.golang.org (a
transparency log of hashes) is the right shape and the wrong size for a one-owner compiler --
it needs a server, a log, and an operator. A Homebrew tap is a git repository of formula files
that anyone forks and PRs, and brew reads it as files. A tap of TOML is the choice, with
the sumdb's one property kept: a published row never changes.
Repository schivei/mc-registry:
README.md the three rules (name, immutability, how to register)
index/<name>.toml one file per package, added by PR
.github/workflows/check.yml `mc pkg check` on every changed index file; refuses a diff that
edits an existing [[versions]] row except to add `yanked = true`;
a scheduled job re-checks every row (link rot, like check-sysroots)
# index/geo.toml
[package]
name = "geo"
repo = "https://github.com/schivei/mc-geo"
description = "2-D vectors"
[[versions]]
version = "1.0.0"
url = "https://github.com/schivei/mc-geo/archive/refs/tags/v1.0.0.tar.gz"
strip = 1
sha256 = "<tree hash>"
deps = []
[[versions]]
version = "1.2.0"
url = "https://github.com/schivei/mc-geo/archive/refs/tags/v1.2.0.tar.gz"
strip = 1
sha256 = "<tree hash>"
deps = ["mathx 1.1.0"]
mc reads the index one file at a time: <registry>/index/<name>.toml, where <registry> is
https://raw.githubusercontent.com/schivei/mc-registry/main by default (PKG_REGISTRY in
src/pkg.mc), [registry].url in mc.toml, or --registry URL|DIR. An https:// registry is
fetched with fetch_get into <pkgdir>/index/<name>.toml (the snapshot mc pkg reads offline);
a DIR registry is read in place. So: a private registry is a directory or any URL with the
same layout -- a git clone of a private tap and [registry] url = "/path/to/it" -- and
costs no code. That is the price of "private registries": zero, and no access control beyond the
repository's own.
Who can register: anyone, by pull request, which is Go's "anyone with a public repository"
translated to a registry that has a human in it. The owner (CODEOWNERS on index/) reviews; the
CI check is the gate that matters: it downloads each new row with --yes, recomputes the tree
hash, and refuses a row whose hash, [package].name or deps do not match the archive's own
mc.toml, a name outside the rule, a name the bundle serves, or a repo change on an existing
name without the owner's review. Name squatting is a policy, not a mechanism: first PR wins the
name, transfers go through the owner. Registration is versioned in git, so the history of every
row is the audit log the sumdb would have provided.
6. Two kinds of package, and both #
The manifest does not enforce a kind; it names entries:
# the package's own mc.toml, at the repository root
[package]
name = "geo"
files = ["geo.mc", "vec.mc", "mc_geo.mc"]
lib = "geo.mc" # optional: the file a PROGRAM includes
module = "mc_geo.mc" # optional: the file a COMPILER includes; it provides geo_init()
[deps]
mathx = "1.1.0"
- A library is
#include "geo/geo.mc"from a program; it lands through the named root the driver registers for the entry compilation (what[include].pathsdoes today, named). - A compiler module is
modules = ["geo/mc_geo.mc", "user.mc"]; it lands in the generated compiler source. The rule a package must follow, stated once inpackages.md: a package never definesuser_init. It exports<name>_init()and the project's own module calls it --lib/user_float.mcis the six-line precedent, andexamples/conc'slg_morechain is what happens without the rule.mc buildgenerates nothing here: the project writes the six lines. (D8 prices the alternative.) - Both is
<float>:float.mcteaches the compiler,float_rt.mcis included by programs. The teko package is this shape --ngen/modules plus a runtime -- and needs nothing beyond two entries in one manifest.
lib/module are informational (mc pkg list and the registry page print them); what a file
does is decided by who includes it, as it is for every .mc file today.
7. Commands, and where they live #
A new part, <mc/core_pkg> (src/core_pkg.mc), that includes <mc/core_build> (once-only) and
src/pkg.mc, and registers one subcommand. The M41 debloat argument applies twice:
- The READ side --
[deps]/[replace]parsing, the lock reader, the tree hash, root registration, the build-time refusals -- issrc/deps.mcinside<mc/core_build>. A compiler withmc buildand withoutmc pkgstill builds a project from its lock and itsdeps/tree. That is the CI/consumer shape and it matches "mc build never downloads". - The WRITE and network side -- index fetch, MVS, lock writing, archive fetch, vendor copy,
[deps]editing,hash,check-- issrc/pkg.mcinside<mc/core_pkg>. A recreated compiler that will never resolve a dependency omits the part and thepkgusage line disappears with it (subcommand_usage).
mc pkg sync [DIR] [--config FILE] [--yes] [--registry URL|DIR] [--pkg-dir DIR]
mc pkg add NAME[@VERSION] [DIR] [--config FILE] [--yes] [--registry ...] [--pkg-dir DIR]
mc pkg update [NAME] [DIR] [--config FILE] [--yes] [--registry ...] [--pkg-dir DIR]
mc pkg list [DIR] [--config FILE] [--pkg-dir DIR]
mc pkg vendor [DIR] [--config FILE] [--pkg-dir DIR]
mc pkg verify [DIR] [--config FILE] [--pkg-dir DIR]
mc pkg hash DIR
mc pkg check INDEX.toml [--yes] [--pkg-dir DIR]
| command | Go analogue | what it does |
|---|---|---|
sync | go mod tidy + go mod download | reads [deps], fetches the index rows it needs, runs MVS, writes mc.lock (drops rows nothing requires), fetches missing trees. Without --yes: prints the plan -- each index file and archive it would download, with hash and destination -- and nothing was downloaded: re-run with --yes, exit 0 (M25 D10; mc has no isatty). With a DIR registry and everything cached it completes with no download |
add | go get pkg@v | writes NAME = "VERSION" into [deps] (newest non-yanked when no @) by the lim_fix_write method -- one key, one section, every other byte through -- then sync |
update | go get -u | raises the [deps] minimum(s) to the newest non-yanked index version, then sync |
list | go list -m all | one line per lock row: name version sha256[0..12] vendored|cache|path -- no absolute path, so it is a golden |
vendor | go mod vendor | copies each locked tree into deps/<name>/, then verify |
verify | go mod verify | rehashes every locked tree (vendored or cached) and checks lock vs [deps] consistency; exit 0 or 2 with the § 3 messages |
hash | (dirhash.Hash1) | prints the tree hash of a checkout: the author's tool and the registry CI's |
check | -- | the registry-side gate of § 5 |
mc build gains --pkg-dir DIR only. The single-file CLI gains nothing: a mc x.mc build has
no mc.toml and therefore no dependencies, and --include=DIR already reaches a vendored tree
by hand.
8. Publishing a package #
A package repository contains: mc.toml with [package] (and [deps] if any); the files it
lists; optionally a [project] for its own tests (a package that is also a program is fine; a
program's mc.toml without [package] is not a package). Cutting a version is git tag v1.2.0
plus git push --tags. Registering it is one PR to mc-registry adding a [[versions]] row
whose sha256 the author gets from mc pkg hash . on the tagged checkout; the registry CI
re-derives the hash from the tarball and refuses a mismatch, so a moved tag (§ Risks 4) is caught
before it is published, and a tag moved AFTER publication is caught by every consumer's fetch.
The bundle is the standard library and is not in the registry. <float>, <i128>, <sys>
are versioned with the compiler and need no fetch; they are what "the binary alone is the
toolchain" (M15) means. Their names are reserved in the registry. Promoting a registry package
into the bundle is a normal tools/bundle.list change, and demoting a bundled library to a
package is possible in principle (the [package] manifest describes lib/ as well) but is not
this milestone.
9. TOML: one parser, re-entrant #
toml_push()/toml_pop(frame) in src/toml.mc (~30 lines): save the eight table globals
(tm_ents tm_entcap tm_n tm_aot_name tm_aot_n tm_aotcap tm_naot tm_file) into an arena frame,
zero them, and restore. The lock, an index entry and a package manifest are each parsed inside a
push/pop; what is needed is copied into deps.mc's own flat tables (name, version, hash, deps in
source order -- rule 1) before the pop. Plus toml_occurrences(name) (~8 lines) so [[package]]
and [[versions]] rows can be counted. This replaces the third bespoke reader before it is
written; lim_read_usage stays as it is.
Out of scope #
- Binary artifacts of any kind: no prebuilt objects, no compiled compilers in the registry.
- A server. The registry is a git repository read as files; no API, no search service, no transparency log. The git history is the log.
- Dependency solving beyond MVS; two majors of one name in one build are refused, not
solved. Semantic import versioning (
/v2) unpriced. - Private registries as a feature: a directory or a URL with the index layout is one, at
zero lines; authentication is the transport's business (a private URL
curlcan reach with the user's own~/.netrc). - Git as a fetch transport,
git+https://sources inmc.toml, unregistered URL deps. A package is registered or replaced by a path; nothing in between. Priced as a follow-up: ~60 lines inpkg.mcfor a[deps.x] url = ...form, and a second spelling of the same list. - A minimum-
mc-version key. The compiler has no version string; a package that uses a hook the running compiler lacks fails ascall to unknown function(src/gen_resolve.mc:389), which is loud. Revisit when 1.0.0 givesmca version to check. - The sandbox itself (M43). M44 runs a package's compiler module exactly where a
[compiler].modulesfile runs today: inside the spawned taught compiler. - Any change to
stage0/. The lexer changes are insrc/lex.mconly;stage0/lex.ccannot see named roots and never has to (it compilessrc/mc.mc, which has no dependencies). - Directory listing,
rename,getenv: not added.
Files and estimated deltas #
| file | lines | what |
|---|---|---|
src/fetch.mc | ~120, new | fetch_get (https via downloader, else local copy), fetch_extract, hex64: the M25 three, general |
src/sysroot.mc | -75 / +12 | calls fetch_*; sysroot_extract reads its row and delegates |
src/deps.mc | ~340, new | name rule, [deps]/[replace]/[registry] reading, lock reader, tree hash over mc.toml + files, cache/vendor/path resolution, per-file verification, named-root and edge registration, the five refusals of § 3 |
src/pkg.mc | ~760, new | semver, index fetch/snapshot, MVS, lock writer (sorted), archive fetch + hash + manifest, vendor copy, [deps] line edit, hash, check, list, verify, pkg_cmd dispatch |
src/core_pkg.mc | ~22, new | includes core_build.mc + pkg.mc; mc_pkg_init() = one subcommand("pkg", &pkg_cmd, ...) |
src/core_build.mc | +2 | fetch.mc, deps.mc in the list |
src/core.mc, src/main.mc | +1 / +1 | the sixth part; mc_pkg_init() |
src/toml.mc | +38 | toml_push/toml_pop, toml_occurrences |
src/lex.mc | +65 | named roots table (grow, a T_NAMEDROOT tag), lex_add_named_root, the branch in lex_find_path_from, lex_root_of, the closure rule in lex_include and the #embed path, the post-parse files walk's accessor |
src/driver.mc | +48 | --pkg-dir; drv_apply_config split into drv_apply_deps (both halves) and the entry-only rest; drv_include keeps a named-root path verbatim; the post-parse files check |
src/cli.mc | 0 | |
tools/bundle.list, src/bundle_data.mc | +4, regenerated | mc/fetch, mc/deps, mc/pkg, mc/core_pkg |
tests/pkg/registry/index/{geo,mathx,teach,bad}.toml | fixture | rows with url = "@ARCHIVES@/geo-1.2.0.tar.gz" rewritten by the script; hashes hard-coded |
tests/pkg/src/{geo-1.0.0,geo-1.2.0,mathx-1.0.0,mathx-1.1.0,mathx-2.0.0,teach-1.0.0,bad-1.0.0}/ | fixture | each with mc.toml [package]; teach registers a syntax_stmt("unless", ...) as teach_init(); bad includes outside its tree |
tests/pkg/app/{mc.toml,obj.toml,main.mc,user.mc,mc.lock.expect,add.toml,add.toml.expect} | fixture | the consumer: [deps] geo = "1.2.0", mathx = "1.0.0", teach = "1.0.0"; main.mc uses unless and geo_dot |
tests/golden/pkg-list.txt | new | |
.gitattributes | +1 | tests/pkg/src/** -text so the fixture hashes hold on a Windows checkout |
scripts/check-pkg.sh | ~320, new | § Acceptance 1-17 |
scripts/check-parts.sh | +25 | case 1b for <mc/core_pkg>; the offline-consumer probe |
scripts/check-docs.sh | +1 | the pkg_/deps_ families and the --pkg-dir/--registry flags |
Makefile | +8 | check-pkg, inside check |
docs/reference/packages.md | ~420, new | everything above, every message, the manifest, the lock, the registry layout, the closure rule |
docs/guide/25-packages.md | ~240, new | using one; publishing one; the user.mc six lines; vendoring for CI |
docs/reference/toml.md | +80 | [deps], [replace], [registry], [package] |
docs/reference/cli.md | +45 | mc pkg, --pkg-dir, exit 2 rows |
docs/reference/diagnostics.md | +55 | § 13 packages |
docs/reference/bundle.md, docs/build.md, docs/determinism.md, docs/plan.md, docs/README.md, docs/guide/98-recreating-the-compiler.md | +15 / +30 / +12 / +1 row / +1 / +10 | the sixth part, reserved names, the lock as a determinism input |
schivei/mc-registry (separate repository) | README.md ~80, .github/workflows/check.yml ~70, index/ | seeded with the owner's first package; the teko port when it exists |
stage0/, lib/ | 0 |
Net new src/ lines ~1 350; the five goldens move once (the bundle grows and core.mc gains a
part), under M17 step A's protocol and M41's note 9.
Acceptance (ordered; scripts/check-pkg.sh build/mc1, no network anywhere) #
- The fixture registry is built by the script, not checked in as archives: each
tests/pkg/src/<name>-<v>/istar -czf'd into$tmp/archives/<name>-<v>.tar.gzwith the top directory<name>-<v>(strip = 1, GitHub's shape), andtests/pkg/registry/index/*.tomlis copied to$tmp/registry/index/with@ARCHIVES@replaced. gzip timestamps make the archives non-reproducible; nothing hashes them. - The hash is stable across hosts:
mc pkg hash tests/pkg/src/geo-1.2.0prints the value hard-coded in the fixture index, on macOS, Linux and Windows CI. - The plan is printed and nothing is fetched:
mc pkg sync tests/pkg/app --registry $tmp/registry --pkg-dir $tmp/c1lists three archives with hashes and destinations, printsnothing was downloaded: re-run with --yes, exits 0, and$tmp/c1holds no<version>.toml. - MVS, not "latest": the same with
--yeswritestests/pkg/app/mc.lockbyte-identical tomc.lock.expect--mathxat1.1.0(geo 1.2.0's minimum wins over the app's 1.0.0), never at the registered2.0.0, rows sorted; three<version>.tomlmanifests exist with[[file]]rows; runningsync --yesagain downloads nothing and rewrites the lock identically. - A two-package chain builds and runs:
mc build tests/pkg/app --pkg-dir $tmp/c1builds the taught compiler fromteach/mc_teach.mc+user.mc, compilesmain.mc(which includes"geo/geo.mc", whose own#include "mathx/mathx.mc"resolves through the closure), and the binary's stdout/exit matchmain.mc's header. - Byte-identical objects from two fetches: a second
sync --yesinto$tmp/c2and a build withobj.toml(kind = "obj") from each cache givecmp-identical objects; the two lock files are identical. - The lock refuses a tampered source: append one byte to
$tmp/c1/geo/1.2.0/vec.mc;mc buildexits 2 withgeo 1.2.0: vec.mc does not match mc.lock;mc pkg verifysays the same; restore, both exit 0. Then editfilesin the cachedmc.toml: exit 2 namingmc.toml. - Stale lock, stale tree: raise
[deps] geoto"1.9.0"in a copy of the config ->mc buildexits 2 withmc.lock is stale: run mc pkg sync --yes; delete$tmp/c1/mathx/-> exits 2 withmathx 1.1.0 is not fetchedand therun:line.mc buildmust not have spawnedcurlin either case (the script puts acurlshim onPATHthat fails if invoked). - Vendoring is the offline road:
mc pkg vendorpopulatestests/pkg/app/deps/{geo,mathx, teach}/;mc build --pkg-dir $tmp/emptysucceeds with no cache at all; its object iscmp-identical to step 6's;mc pkg listprintsvendoredin every row and matchestests/golden/pkg-list.txt. - A package is closed: add
bad = "1.0.0"to a copy of the config, sync, build -> exit 1 withbad/bad.mc:2: package bad reaches outside its tree: ...; a#embedof an absolute path insidebadis refused with the same words; a filegeo/extra.mcplanted in the cache and included by a planted line isnot declared in geo's [package].files. - Names:
[deps] float = "1.0.0"->float: a bundled name;Geo = ...->invalid package name; both at the key'sfile:line:col, exit 1. - Majors are refused, not solved: a fixture package requiring
mathx 2.0.0next to the app's1.x->mathx: 1.1.0 and 2.0.0 are different majors: no solver, exit 1, no lock written. - A failed fetch leaves no claim behind: an index row whose
urlnames a missing archive -> exit 2 with the M25-shaped message (mc: the download failedfor a URL,cannot openfor a path), no<version>.toml, and a followingmc buildsaysnot fetchedrather than reading debris. A row whosesha256is wrong ->checksum mismatch for geo 1.2.0, the listed files unlinked, no manifest. mc pkg addedits one line: onadd.toml(no[deps]),mc pkg add mathx@1.0.0 --yesproducesadd.toml.expectbyte for byte -- every comment and byte outside the new[deps]section untouched;add mathxwith no version picks1.1.0, never the registered-and-yanked1.2.1row the fixture carries.mc pkg checkis the registry gate: on$tmp/registry/index/geo.toml --yes --pkg-dir $tmp/chkexit 0; with one hash altered exit 2; withname = "float"exit 1.- Parts:
check-parts.shshows<mc/core_min>+<mc/core_pkg>compiles alone; a probe compiler assembled fromcore_min+core_machines+core_writers+core_build+core_bundle(nocore_pkg) builds the vendored app of step 9 and prints a usage with nopkgline; the measured table gains the+ <mc/core_pkg>row. - Inert:
scripts/check-inert.shclean for every object of a project without[deps];check-standalone,check-obj32/32,check-build21/21,check-sysrootsandtests/golden/sysroot-list.txtunchanged;mcwith no arguments prints today's usage plus exactly thepkglines;mc sysroot fetch linux-aarch64 --yes --sysroot-dir $tmp/s(the CI step, the only networked check, unchanged) still writes the four files -- proving thefetch.mcmove is behaviour-neutral. Goldens rewritten once, after an empty--dump-asmdiff betweenmc1andmc2andcmp build/mc2.o build/mc3.o.make check-docsgreen.
Risks #
- Supply chain: a compiler module is code that runs on the developer's machine at build
time. Go's
go buildexecutes no package code (no build scripts), which is a security property Go chose on purpose; mc's compiler modules are proc-macro-shaped and DO run, inside the spawned taught compiler, with the developer's file system. A library package is only slightly better:#embedreads any file the compiler can. M44's brakes are the closure rule (§ 2: a package reads its own tree, the bundle and its declared deps, nothing else, for#includeand#embedalike) and the lock (nothing runs that is not the reviewed bytes). What they do not stop is a module that opens a file byextern openatuser_inittime. That is M43's job and the seam is the spawn indrv_teach: the taught compiler is already a separate process, so a sandbox wraps oneposix_spawnp. Until M43 lands,packages.mdsays in its first paragraph that a compiler-module package is trusted code. - Name squatting and transfers are policy in a one-owner registry: first PR wins, the
owner reviews,
repochanges need the owner. Cheap now; revisit if the index passes a few hundred names. - Tarball regeneration (GitHub, 2023-01-30) does not move the tree hash; it would move an archive hash, which is why there is none.
- Tag mutability: a tag moved after registration fails every consumer's fetch with
checksum mismatch-- loud, and the right outcome. A tag moved BEFORE registration is a registry row that never matched, refused bymc pkg check. - Network in CI: none in
make check; the fixture registry is a directory. Only the registry repository's scheduled job and the existingmc sysroot fetchstep touch the network, and a dead URL is a maintenance issue there (M25 § Risks), not a red PR. toml_push/toml_popregressions: a pop forgotten on an error path leaves the project table swapped out. Every parse of a foreign file is wrapped in one function that pops before returning or dies (_exit, where the table no longer matters).- The
fileslist as the boundary: a package author who forgets a file ships a package that fails withnot declared in ... [package].fileson the first include -- loud, at the consumer.mc pkg checkcould cross the list againsttar -toutput (captured with thedrv_sdkfile-action trick) and warn; priced at ~40 lines, optional. - Line endings on Windows checkouts move every fixture hash;
.gitattributes-textontests/pkg/src/**is in the file table, and the Windowschecksubset runs step 2 first. mc pkg addediting a human file:lim_fix_write's method keeps every other byte, and step 14cmps the result; the failure mode is a[deps]table written twice in a file that already has it under an unusual spelling ([ deps ]), which the key scan does not recognise. Refuse when the scan finds no[deps]buttoml_get("deps.x")says one exists.- No version string in
mc: a package needing a hook from a newermcfails withcall to unknown functionrather than "needs mc >= 0.9". Acceptable until 1.0.0 (§ Out of scope). - Six parts and a sixth
*_init:main.mc's list grows;check-parts.shcase 1b is the regression net M41 built for exactly this. - Diagnostics in cached packages print absolute paths (
/Users/me/.mc/pkg/geo/1.2.0/vec.mc:3) while vendored ones printdeps/geo/vec.mc:3. Objects carry no path (rule 4, and there is noN_OSO), so determinism of OUTPUT holds -- step 6 and step 9 prove it -- but two machines' error texts differ. A display name (geo/vec.mc, the bundle's "errors point at the bundled name" precedent) is ~15 lines inlex_pushand a follow-up; M30's DWARF will want it too.
Decisions (architect) -- to ratify with the owner #
- D1 -- import spelling:
#include "geo/geo.mc"through a named root, not<geo/...>and not a new directive. Zero language change; the bundle's "no filesystem fallback" promise stays intact; collisions with the bundle become a registry rule instead of a lexer rule. The alternative -- extending<name>to packages -- reads well and breaks a documented promise. - D2 -- identity is a registry name, and
[deps] name = "min.version". The Go path (github.com/u/r) is the location; the owner asked for the registry to be the manager, so the name is the identity and the row is the location. A[deps.x] url = ...form for unregistered sources is out of scope, priced at ~60 lines. - D3 -- registry = one git repository of TOML, one file per package, PR to register, owner
review,
mc pkg checkas the CI gate, rows immutable exceptyanked. Not a proxy, not a sumdb: no operator, no server; the git history is the log. - D4 --
mc.lockbesidemc.toml, machine-written, rows sorted by name, one content hash per package, edges inline. Not insidemc.toml. - D5 -- the hash is Go's h1 over
mc.toml+[package].filesin manifest order, in hex; never the archive. The author lists the files becausemchas noopendir, and the list is also the vendor-copy list and the build-time boundary. - D6 -- MVS, precisely Go's; two majors in one graph are refused. No solver.
- D7 --
mc buildrehashes every dependency on every build and refuses any disagreement (exit 2). "Checked, not trusted." Cost is linear in dependency source size, which the lexer reads anyway. Alternative: trust the cache manifest and rehash only inverify-- faster, and it makes the tamper acceptance averify-only proof; recommend against. - D8 -- a package never defines
user_init; it exports<name>_init()and the project's own six-line module calls it. Alternative:mc buildgeneratesuser_initfrom a[package].initkey -- ~40 driver lines and a second place a compiler's init order is decided; recommend against until a second real consumer wants it. - D9 -- tarballs by URL,
tar -xzf, nogitdependency; GitHub'sarchive/refs/tagsas the documented default shape, any https tarball accepted. - D10 -- cache
~/.mc/pkg/<name>/<version>/+<version>.tomlbeside it;--pkg-dir DIRoverride onmc pkgandmc build;deps/is the vendor directory (owner's word; Go's isvendor/) and wins over the cache when present. - D11 --
[replace] name = "path"for development, unhashed and announced. Go'sreplace. - D12 --
mc pkgis a sixth part<mc/core_pkg>; the read side (deps.mc,fetch.mc) lives in<mc/core_build>so a compiler withoutcore_pkgstill builds from a lock anddeps/. - D13 --
sync|add|updaterequire--yesto download and print the plan otherwise (M25 D10, the same reason: noisatty, no prompt). - D14 --
toml_push/toml_popinsrc/toml.mcrather than a third bespoke reader; M25 D3 stands for its own case (sysroots.mcstays a.mctable). - D15 -- the bundle is the standard library, outside the registry; its names are reserved.
- D16 -- the closure rule is in M44, the sandbox is M43;
packages.mdstates the trust model plainly until M43 lands. - D17 -- no 1.0.0 on the back of this milestone (M42 D8): 1.0.0 waits for the roadmap and
for the teko/
ngenconsumer, which is also the first package of the second kind this spec should be validated against before it is called done.
Architect's additions: (a) step 8's curl shim -- mc build must be PROVED never to spawn a
downloader, not just documented; (b) the registry repository's README.md is written in the same
PR as packages.md, with the three rules and one worked registration; (c) docs/guide/25-packages.md
is written for the teko port's author and ends with the six-line user.mc.
Amendment (owner, 2026-09-04): angle brackets, ~/.mc/libs, the slim binary, install/update/upgrade #
The owner's two rulings, translated from the Portuguese:
(1) "For M44 I would change one thing: since these are external libraries referenced in the toml, they should be reachable as
#include <pack/lib.mc>. What that changes: even what we ship embedded today could become an unpacking into a directory~/.mc/libs/pack_name/v<version>/, and with that we gain even more extensibility."(2) "I would say we even gain a smaller binary: it could carry only the executable and require an
mc install, downloading even what is embedded today; anmc updateupdates the packages' versions; anmc upgradeself-updates the version and downloads the basic libraries for mc; and so on."
Read as four requirements that override D1, D2 and D15 and extend the scope: (a) a dependency is
spelled with angle brackets, #include <pack/lib.mc>, so there is ONE resolution model for "a
library that did not come from my own tree"; (b) the bundle's entries are a package like any other,
materialisable under ~/.mc/libs/<pack>/v<version>/; (c) a second, slimmer release flavour carries
no blob and gets its libraries through mc install; (d) three verbs, install, update,
upgrade, the last one a self-update. What follows rewrites the draft where the rulings touch it and
says, item by item, what does not move.
What survives untouched, and why. The lock (§ 3), the h1 tree hash over mc.toml +
[package].files, MVS with the two-majors refusal, the registry as one git repository of TOML
(§ 5), the closure rule (a package reads its own tree, the bundle and its declared deps, nothing
else), the sixth part <mc/core_pkg> with the read side in <mc/core_build> (§ 7), "mc build
never downloads", toml_push/toml_pop (§ 9), D3-D9, D11-D14, D16-D17. None of them depends on
HOW an include is spelled or WHERE a tree lives: they are about identity, content and selection.
The rulings change the spelling, the directory, and add a distribution channel for the compiler's own
libraries; they do not change what a package IS.
A. Angle brackets and one resolution model #
A1. The spelling. A dependency geo is included as
#include <geo/geo.mc> // a file of the package geo, at the version mc.lock pins
#include <geo> // the package's `lib` entry (mc.toml: lib = "geo.mc"), if it has one
#include <float> // unchanged: a name the binary ships
#include <mc/core> // unchanged
<name> stops meaning "the bundle" and starts meaning "a library that is not in my tree":
resolved from the lock, from the bundle, or from the installed copy of the compiler's own package --
in that order, below -- and never from the working directory. Quotes keep meaning "a file on disk
relative to me or to [include].paths" (src/lex.mc:536, lex_find_path_from, unchanged). The
draft's named roots for the quote form (§ 2, lex_add_named_root, the branch in
lex_find_path_from) are DROPPED: one spelling for one thing, which is what the owner asked for.
[compiler].modules takes the same spelling -- modules = ["<teach/mc_teach.mc>", "user.mc"] --
and drv_gen_compiler emits a value that starts with < verbatim, the rule src/driver.mc:417-425
already applies to [compiler].core (core = "<mc/core_min>", M41). Nothing in the language
changes except one token (A2).
A2. A fact the draft did not check: . is not a token. tok_init (src/lex.mc:246-294)
registers the keywords and the operator lexemes; there is no "." among them, and <name> is not
tokenised specially -- do_directive (src/parse.mc:1610-1626) reassembles the lexemes between
< and >, on purpose, so that --dump-tokens stays byte for byte what stage0/lex.c produces.
So #include <geo/geo.mc> fails today at the . with unexpected character (src/lex.mc:902),
and the owner's spelling needs one of two things: (i) tok_add(".", 1) appended at the END of
tok_init, after the last existing lexeme, so no existing id moves (K_U8..K_EXTERN stay 256..269
and every punct keeps its id), --dump-tokens is unchanged for every file that has no bare . --
which is every file check-lex compares, since float literals are consumed raw by syntax_lit
(lib/float.mc:343) before the punct scan and lib/float_rt.mc is already seed-skip -- and
examples/lang/lang.mc:44's own tok_add(".", 1) lands on the same id because tok_add is
idempotent (the comment two lines above it says so); or (ii) accept only <geo/geo> and append
.mc on disk, the bundle's own convention (lex_strip_mc, src/lex.mc:574, exists because bundle
names carry no .mc). Recommendation: (i), and lex_include_name strips a trailing .mc from
the reassembled name so <geo/geo.mc> and <geo/geo> are one name, exactly as <mc/core> and a
relative "core.mc" inside the bundle are today. Cost: one line in tok_init, one call in
lex_include_name. stage0/lex.c is untouched: it never sees a <...> with a dot (tests/mc/ is
where such files live, and that directory is already outside the mc0 cross-checks).
A3. The resolution order for <X> -- in lex_include_name (src/lex.mc:626), which today is
one call to bopen_fn (:498) and one error. It becomes three steps through TWO pointers, the
existing bopen_fn and a new lopen_fn registered by lex_set_libs from mc_build_init()
(src/core_build.mc:36), so the lexer still depends on nothing and lexdump/astdump keep
compiling with mc0:
| step | who answers | for which names | where the bytes come from |
|---|---|---|---|
| 1 | lopen_fn(X, 0) -- deps.mc's libs_open, the LOCK road | X's first path component is a package the lock names (geo, teach, or a bundled name pinned in [deps], A5) | deps/<pack>/<rest> if vendored, else <libs>/<pack>/v<version>/<rest>; <pack> alone is the lock row's lib entry |
| 2 | bopen_fn(X, 0) -- the bundle, unchanged (src/bundle.mc, bundle_open; <mc/host> still rewritten by host_bundle_open, src/core_bundle.mc) | every name in tools/bundle.list, plus the two synthetic ones | the blob |
| 3 | lopen_fn(X, 1) -- the INSTALLED mc package | the same names as step 2, when the binary carries no blob (B) | <libs>/mc/v<mc_version()>/ + the path bundle.list maps the name to |
| -- | neither | anything else | prog.mc:1: unknown bundled include: no/such/module, unchanged text |
<libs> is host_home()/.mc/libs (src/host_macos.mc, src/host_linux.mc, src/host_windows.mc:
the HOME=/USERPROFILE walk sysroot_cache_dir already uses, src/sysroot.mc:215), or
--libs-dir DIR on every command that reads it (the --sysroot-dir precedent), so CI depends on
no HOME. Step 1 exists only when a lock was read (mc build, both halves); the single-file CLI
(mc x.mc) has no lock and therefore no step 1: <geo/geo.mc> there is refused with the step-3
miss, and --include=DIR remains the hand road. Never the working directory (a project cannot
shadow <float> by dropping a float.mc next to main.mc; that is the M15 stance and it is what
lets the answer be a function of (binary, lock, libs content) alone), never an unpinned
latest (<libs>/geo/ may hold v1.0.0/ and v1.2.0/; only the lock says which one, and a
directory that no lock names is never opened).
A4. The mc package and the layout of ~/.mc/libs/mc/v<version>/. The bundle's 75 entries
are ONE package, named mc, at the compiler's own version (C). On disk it keeps the REPOSITORY
layout -- lib/float.mc, src/core.mc, src/host_macos.mc, tests/mc/bundle/embed_demo.txt --
with bundle.list (the NAME<TAB>PATH manifest, tools/bundle.list) at the root as the name map.
Not a by-name layout, because the bundle's relative-include fallback (bundle_find_base,
src/bundle.mc: mc/driver -> "../lib/prelude.mc" -> last component prelude) has no
equivalent on a filesystem; in the repository layout src/driver.mc's #include "../lib/prelude.mc"
resolves through lex_find_path as a plain relative path, and src/core.mc's "arena.mc" lands on
src/arena.mc. The once-only key for a name served from disk is the normalised absolute path
(lex_include records paths, lex_include_bundled records canonical names; both go through
lex_seen, src/lex.mc:557), so <mc/host>, <mc/host_macos> and core.mc's own
"host_macos.mc" coincide on disk exactly as they coincide in the blob. Two consequences worth
writing down: the installed src/bundle_data.mc is the mode 1 text (#embed bundle_blob
"bundle.bin" + the index, bundle_emit(..., 1), src/bundle.mc) next to a real src/bundle.bin
(the blob, bundle_read(BUNDLE_BIN)), so a taught compiler built from an on-disk <mc/core> pays
one N_BLOB node and not ~45 000 u64 nodes (M21.5's arena argument, docs/build.md § M15), and
both forms produce the same object (that is what check-standalone measures today); and
tools/bundle.list itself is bundled as mc/bundle.list (one line in the manifest; its bytes do
not depend on its own content, so unlike bundle_data there is no recursion; last component
bundle.list collides with nothing), so the FULL binary can write the whole package to disk from its
own blob with no network (B3). Diagnostics from a disk-served <mc/core> print the absolute path
where the bundle printed the name (draft Risk 12, unchanged and now more visible).
A5. Collisions: the lock wins, except for mc. A registry package MAY carry a bundled name
(float, sys, i128 -- the draft's reserved-name rule is withdrawn for lib/'s names) and a
project that pins [deps] float = "1.3.0" gets <float> and <float/float_rt.mc> from the locked
tree instead of the blob. That is the extensibility the ruling asks for, and it is safe on both
counts the task names: determinism -- step 1 answers only from a lock row, the row pins a
content hash, mc build rehashes on every build (D7), so two machines with the same binary, lock
and tree bytes resolve the same bytes; a project with no [deps] float line is byte for byte what
it was; offline -- a locked float that is neither vendored nor fetched is mc: float 1.3.0
is not fetched / run: mc pkg sync --yes, exit 2, the same as any dependency, and a project with no
lock never leaves the blob. What stays reserved, in the registry and in [deps]: mc, every
mc/... name, deps, build. The mc package can never be pinned by a lock: <mc/core> is the
compiler's own source, check-standalone is an equality between a binary and ITS bundle, and a
taught compiler assembled from a foreign mc/core would be a different compiler than the one
running mc build -- the exact confusion M37's <mc/host> was designed out. mc pkg check refuses
an index row named mc; deps.mc refuses [deps] mc = ... at the key (mc: reserved, exit 1).
A6. The closure rule, restated for the new spelling. A file under a package root may include
or #embed: its own tree (quotes, relative), <...> names that resolve to the bundle or to the
installed mc package, and <dep/...> where dep is in its lock row's deps (or itself). The
edge list and lex_root_of (draft § 2, ~35 lines) survive as they were; roots are registered from
the lock rows by drv_apply_deps, not from a [include]-like list. The message and the #embed
case are unchanged.
A7. Rewrites, itemised. § 2 is replaced by A1-A3 and A6 (named roots gone; the "angle brackets
are the bundle, quotes are files" sentence becomes "angle brackets are libraries, quotes are my
files"); § 4's cache paragraph becomes ~/.mc/libs/<pack>/v<version>/ + <pack>/v<version>.toml
beside it, --pkg-dir becomes --libs-dir everywhere (mc pkg, mc build, mc install,
mc upgrade); vendoring keeps deps/<pack>/ and still wins over <libs>; § 6's example becomes
#include <geo/geo.mc> and modules = ["<geo/mc_geo.mc>", "user.mc"], and lib stops being
informational -- it is the target of a bare <geo> and is copied into the lock row; § 8's last
paragraph ("the bundle is the standard library and is not in the registry") becomes: the bundle is
the mc package, versioned with the compiler, shipped inside the full binary and installable next
to the slim one; its lib/ names are not reserved -- a registry package may replace one, lock-driven
-- and its mc/... names are. The M15 promise in docs/build.md § M15 and docs/reference/bundle.md
becomes: "the full binary alone is the toolchain; the slim binary plus one mc install is the
same toolchain".
B. The slim binary and mc install #
B1. Two release flavours per target. mc (full: today's binary, blob embedded -- the bootstrap
SEED on Linux and Windows, scripts/bootstrap-linux.sh:18-26, and the offline/CI default) and
mc-slim (the same compiler with an EMPTY bundle). Weight, from M41's measured table
(scripts/check-parts.sh, CLAUDE.md § M41): the assembly WITHOUT <mc/core_bundle> is
395 820 B on disk and mc itself is 759 875 B; the bundle part adds ~3.7 KB of __text
and ~368 KB of __data (the blob: check-bundle at M41 reports raw 776 601 -> lz 364 543, blob
365 449 B). A slim binary keeps the ~4 KB of reader code and drops the blob, so it weighs
~400 KB, 52% of the full one. The task's "~252 KB" is M15's figure (build/mc-exe 252 316 B
without the blob, when the compiler was nine milestones smaller); it is not what a slim binary would
weigh today and the spec should not promise it.
B2. How the slim flavour is assembled -- zero core lines. It is docs/reference/bundle.md
§ "Your own bundle" applied to an empty manifest: src/bundle_empty.mc (checked in, ~8 lines:
u64 bundle_blob[1]; i64 bundle_idx[BI_N]; #define BUNDLE_COUNT 0; bundle_end() already returns 0
for a zero count and bundle_cache is sized BUNDLE_COUNT + 2), src/core_bundle_slim.mc
(bundle_empty.mc + bundle.mc + bundle_glue.mc), src/bundle_glue.mc (the host_bundle_open
mc_bundle_initpair moved out ofsrc/core_bundle.mc, which then includes it too), andsrc/core_slim.mc= the five parts withcore_bundle_slim.mcin place ofcore_bundle.mc, plusmain.mc. Five entries of three lines each (src/mc_slim.mc,mc_linux_slim.mc,mc_linux_x86_64_slim.mc,mc_windows_slim.mc,mc_windows_x86_64_slim.mc: host file +core_slim.mc+user.mc, thesrc/mc_linux.mcshape) and four*-slim-obj.tomlconfigs (src/mc.linux-aarch64-obj.toml's shape,entrychanged) giverelease.yml's cross-compile steps the objects the Linux and Windows runners link with the samescripts/link-linux.sh/link-windows.sh.check-parts.sh's per-part case covers<mc/core_bundle_slim>for free. Themain()is the same file:mc_bundle_init()registers an opener that always misses.
B3. What a slim binary does BEFORE mc install. Everything that needs no <...>: compile a
program with no angle-bracket include (i64 main() { return 42; }, or a program with externs and
quote includes of its own tree); every --dump-*; mc --host, mc --version; mc build of a
project whose sources have no <...> AND no [compiler] section (a taught compiler is
#include <mc/host> + <mc/core>, src/driver.mc:414,425, so mc build with [compiler] needs
the mc package -- and that is the honest reason mc install must ship mc/core, mc/core_*,
mc/host* and every lib/*.mc: check-standalone's equality is the proof that <mc/core> IS
the compiler, and a slim binary cannot assemble one without it); mc sysroot, mc limits;
mc install itself. What it refuses, with which message, exit 2 in the M25 shape (src/sysroot.mc
sysroot_missing's tried:/run: block, docs/reference/cli.md § Exit codes gains one row):
mc: <float> is not installed
tried: ~/.mc/libs/mc/v0.10.3 (absent)
run: mc install --yes
(prog.mc:1: prefix as every include error carries; a stale ~/.mc/libs/mc/v0.10.2/ from another
version is not consulted and not mentioned -- per-version directories are the whole point.) The
full binary never prints it: step 2 answers first.
B4. mc install [--yes] [--from URL|DIR] [--libs-dir DIR] -- src/install.mc, in
<mc/core_build>, registered from mc_build_init() (MAXSUBCMD 16, src/hooks.mc: three used
today, seven after this milestone). It populates <libs>/mc/v<mc_version()>/ with the compiler's
own package at the compiler's version, from one of two sources:
- the binary's own blob (default for the FULL binary, no network, no downloader): read
<mc/bundle.list>(A4), for each entrybundle_readthe source andwrite_fileit under its path; writesrc/bundle_data.mcin mode 1 andsrc/bundle.binfrombundle_read(BUNDLE_BIN); writebundle.listat the root. ~50 lines, and it is the ONE definition of the on-disk layout:scripts/release-assets.sh --libs(B5) produces the release tarball by running exactly this into a staging directory, so "the same bytes the full binary embeds" holds by construction; - a release asset (default for the SLIM binary, which has no blob to read):
https://github.com/schivei/mc/releases/download/v<ver>/mc-libs-<ver>.tar.gzand its.sha256, throughfetch_get(draft § 4:sysroot_download,src/sysroot.mc:456, generalised), sha256 bysrc/sha256.mccompared to the first 64 hex characters of the.sha256file (thesha256sum -clinerelease-assets.shwrites) BEFORE anytarspawn,tar -xzf --strip-components=1into the version directory (sysroot_extract's one-spawn shape,:484), every path of the extractedbundle.listprobed withlex_readable, and only then<libs>/mc/v<ver>.toml([source] name = "mc" version url sha256+ one[[file]]per entry:sysroot_manifest's shape,:607, no date). On any failure:unlinkwhat was listed, no manifest, exit 2 with the M25 texts (the download failed,checksum mismatch for mc-libs-0.10.3.tar.gz,the archive did not carry src/core.mc).--from DIRreadsDIR/mc-libs-<ver>.tar.gz+.sha256as local files (the test road; also how an air-gapped machine is fed).
Without --yes it prints the plan (sysroot_plan's shape: source, size when known, sha256 when
known, destination) and nothing was downloaded: re-run with --yes, exit 0 -- M25 D10, same reason
(mc has no isatty). A directory whose manifest already exists is mc: mc 0.10.3 is installed
(~/.mc/libs/mc/v0.10.3), exit 0, unless --force. mc install never touches another package's
directory and never reads a lock: it is about the compiler's own libraries only (mc pkg add is how
a registry package arrives; an mc install NAME alias is not adopted, to keep the two meanings
apart).
B5. Release assets. scripts/release-assets.sh gains --slim (the archive is
mc-<ver>-<target>-slim.tar.gz, the binary inside still named mc/mc.exe -- the ad-hoc signature
identifier is the output basename, release.yml "Build the compiler" step) and --libs VERSION
BINARY (stages BINARY install --yes --from-bundle --libs-dir OUTDIR/.stage/mc-libs-VERSION, then
the same explicit-sorted-members / mtime 0 / ustar / gzip -n -9 rules the script already
imposes, producing mc-libs-<ver>.tar.gz + .sha256, host-independent). release.yml: the macOS
build job builds dist/mc AND dist-slim/mc (build/mc1 --exe src/mc_slim.mc -o dist-slim/mc,
after rm -f), runs scripts/test.sh with the full one, packages both, produces the libs tarball
from the full one, cross-compiles the four slim objects next to the four full ones (make
mc-linux-slim-obj etc.), and publishes a LATEST file (D2). build-linux / build-windows link
the slim object next to the full one and package it with --slim; the bootstrap proof keeps using
the FULL binary as its seed (E, risk). publish already globs dist/mc-*.tar.gz -- the slim and
libs archives ride along; the release body's install snippet gains the slim road (mc install
--yes as the second line). A release therefore carries eleven archives: five full, five slim,
one libs, each with its .sha256, plus LATEST.
B6. Does the full binary consult ~/.mc/libs first? Yes, lock-driven only: step 1 runs
before step 2 for names the lock pins (A3, A5); step 3 is reached only on a bundle miss, which for
the full binary means a name the binary does not ship -- and the installed mc package ships the
same names, so it is a miss there too. So the full binary's behaviour with no lock is byte for byte
today's, and with a lock it is today's plus the overrides the lock spells out. A full binary that
was ALSO mc installed has a ~/.mc/libs/mc/v<ver>/ it never reads for its own includes; what the
directory is for on that machine is mc pkg's cache root and the override trees.
C. A version for mc #
C1. Today. The compiler has no version string (docs/ci.md § Versioning: "nothing in the
working tree records it", the VERSION file was deleted so that the tags would be the only truth;
grep MC_VERSION src/ lib/ is empty). ~/.mc/libs/mc/v<X.Y.Z>/ and mc upgrade both need the
running binary to know it.
C2. The baked version. src/version.mc, checked in, six lines:
// the version this binary reports. 0.0.0-dev in the tree; scripts/set-version.sh
// rewrites it from the tag in release.yml and the result is never committed.
uptr mc_version() { return "0.0.0-dev"; }
Included by src/core_min.mc before cli.mc (it is <mc/core_min>'s: cli.mc prints it,
install.mc/upgrade.mc in <mc/core_build> consume it) and bundled as mc/version, because
<mc/core> includes it and the bundle must stay complete. mc --version prints it and a newline
(src/cli.mc:173, next to --host; --host's three lines stay byte-stable -- release.yml and
the bootstrap scripts sed them). scripts/set-version.sh X.Y.Z validates the shape with
scripts/next-version.sh (the one definition of a version), rewrites the one return line, and
runs make bundle so that mc/version inside the blob agrees with mc_version() -- otherwise a
taught compiler built by a release binary would say 0.0.0-dev and look for the wrong libs
directory. release.yml calls it after make mc1 and before every --exe, cross-compile and
--libs step; it does not commit.
C3. The goldens and check-standalone, thought through. src/mc.mc's bytes differ per release
-- in the working tree of the release runner only. The tag's TREE keeps 0.0.0-dev, and every
golden (tests/golden/mc2.sha256, the two Linux and the two Windows ones) is the hash of an object
compiled from the checked-in tree, so they do not move per release and are rewritten only when
src/ changes on purpose, as today. The release proofs still hold: scripts/bootstrap-linux.sh
takes the versioned binary as SEED, compiles the checked-out tag (dev) to mc1l, which compiles it
again to mc2l.o -- the dev object, against the dev golden -- and the script already allows the seed
to differ from mc1l ("the seed may legitimately be an older compiler", docs/bootstrap.md § The
Linux chain, point 3; what must agree is mc1l --dump-asm vs mc2l --dump-asm, both dev). The
macOS release job runs scripts/test.sh dist/mc, no golden. check-standalone compares a binary
against an object compiled from the SAME tree by the same mc1; on the dev tree both say
0.0.0-dev, on a release runner (if it were run there) both would say the tag; equality holds
either way. Two guards so the sentinel cannot leak: scripts/check-bundle.sh asserts
src/version.mc says 0.0.0-dev (a rewritten file fails make check with a message naming
set-version.sh), and .github/workflows/ci.yml runs make check on the untouched tree as it
does now. docs/ci.md § Versioning is amended, not reversed: the tag is still the only truth for
RELEASES; the tree carries a constant sentinel, not a version, and the objection to two sources of
truth does not apply to a constant.
C4. The dev build. mc_version() is the constant 0.0.0-dev -- not the commit hash, which
would move src/bundle_data.mc, build/mc2.o and all five goldens on every commit. Its libs
directory is ~/.mc/libs/mc/v0.0.0-dev/; a dev FULL binary populates it from its blob
(make check's slim proof uses --libs-dir in a temporary directory and never touches HOME).
For ordering, 0.0.0-dev compares as 0.0.0: every release is newer.
D. mc update and mc upgrade #
D1. Placement. install, update and upgrade are top-level subcommands
(subcommand() registrations, one usage line each, mc with no argument lists them after
build|limits|sysroot), consistent with mc build and mc sysroot fetch; the package-author and
maintenance verbs stay under mc pkg (sync add list vendor verify hash check), and the draft's
mc pkg update is REMOVED in favour of mc update [NAME] [DIR] [--config FILE] [--yes]
[--registry ...] [--libs-dir DIR], same semantics (go get -u: raise the [deps] minimum(s) to the
newest non-yanked index version, then sync). install and upgrade live in <mc/core_build>
(src/install.mc, src/upgrade.mc): a compiler without core_pkg -- the CI/consumer shape of
D12 -- still installs its own libraries and updates itself. update lives in <mc/core_pkg>
(src/pkg.mc, registered from mc_pkg_init() beside pkg): it needs the index and MVS. Semver
parsing/comparison moves from pkg.mc to deps.mc (<mc/core_build>), where upgrade can reach
it.
D2. mc upgrade [--yes] [VERSION] [--from URL|DIR] [--libs-dir DIR] -- the self-update, in
the M25 order (plan, --yes, download, verify BEFORE unpack, act, claim last):
- Which version. With no
VERSION: fetchhttps://github.com/schivei/mc/releases/latest/download/LATEST-- GitHub resolvesreleases/latest/download/<asset>to the newest release's asset by redirect, which-fLsS+--proto-redir =httpsalready follow (sysroot_download) -- a one-line text fileX.Y.Z\nthatrelease.yml'spublishjob attaches (gh release create ... dist/LATEST). Chosen overcurl -sI ... -w '%{redirect_url}': the-wform iscurl-only (thewgetfallback has no equivalent), and a one-line file is parsed by the samever_parsethat validates every version, whereas aLocation:header is a URL to be cut. The GitHub API is JSON, whichmcdoes not parse. With aVERSION: that one, validated. - Refusals, before any download:
mc: 0.10.3 is the newest release(latest == current), exit 0;mc: 0.10.1 is older than this binary (0.10.3): name it with --yes to downgrade(an explicit olderVERSIONwithout--yes), exit 2; aLATESTolder than the running version is refused even with--yes(a stale or tampered file, loudly, exit 2); a dev build (0.0.0-dev) refuses unless--yes VERSIONnames one (somake checknever clobbersbuild/mc-exe, and the acceptance test can). - The plan, then
--yes: current, target, the asset URL for this host pair and this FLAVOUR (BUNDLE_COUNT == 0means slim: fetch-slim), the destination path (the running binary's own path), andnothing was downloaded: re-run with --yes. - Download
mc-<ver>-<target>[-slim].tar.gz+.sha256into<libs>/mc/upgrade.<ver>/throughfetch_get;<target>is the release vocabulary (macos-arm64,linux-arm64,linux-x86_64,windows-arm64,windows-x86_64:host_os()+ anaarch64 -> arm64map of six lines, the "two vocabularies"docs/bootstrap.mdrecords). Verify the archive's SHA-256 before unpacking (src/sha256.mc, against the.sha256line), thentar -xzf F -C DIR --strip-components=1 mc-<ver>-<target>/mc-- one member,sysroot_extract's member-list shape. -
Swap, per host, by
host_self_path()(new in the host layer:_NSGetExecutablePathon macOS,readlink("/proc/self/exe")on Linux,GetModuleFileNameA(0, ...)on Windows -- oneexterneach, theGetEnvironmentVariableAprecedent insrc/host_windows.mc;argv[0]is a PATH lookup and not a path) andhost_retire(path):- macOS:
unlink(self), then write the new bytes to the SAME path withcreat(..., MODE_755). A new inode: the running process keeps the old one, and the kernel's cached-signatureKilled: 9(CLAUDE.md§ M12) happens only when a signed file is overwritten IN PLACE -- which is whydrv_compileunlinks first (src/driver.mc:355, "never rewrite a signed binary in place") and whyrelease.ymldoesrm -f dist/mc. The downloaded binary carries its own ad-hoc signature from the release build and was written asmc; nocodesignis run and no quarantine attribute is set (the file is written bymc, not by a browser). - Linux: the same
unlink+ write (renaming over is also fine; not needed). - Windows: a running
.execannot be deleted (docs/bootstrap.md§ The Windows chain;lib/sys_windows_host.mc:280'sunlinkisDeleteFileA) but it can be RENAMED:MoveFileExA(self, self + ".old", MOVEFILE_REPLACE_EXISTING)-- one more kernel32 name inscripts/sysroot-windows.sh's list (:79-82) and oneexterninsrc/host_windows.mc-- then write the new bytes toself. The nextmc upgradeunlinks a leftovermc.exe.oldfirst;mc --versiondoes not.
- macOS:
- Then the libraries: spawn
<self> install --yes [--libs-dir DIR]-- the NEW binary, throughdrv_spawn_ok(posix_spawnp+waitpid), because the running process is the old version and itsmc_version()is the wrong directory. A full binary installs from its blob (no second download); a slim one fetchesmc-libs-<ver>.tar.gz. Exit is the child's. - Print
mc 0.10.2 -> 0.10.3 (/usr/local/bin/mc); remove the download directory.
D3. The supply-chain gap, named. Every check above is against a .sha256 fetched from the
SAME origin as the archive: that proves the bytes arrived intact, not that they were published by
the owner. Anyone who can serve github.com/schivei/mc/releases/... to this machine can serve a
matching checksum. The priced follow-up is a signing key -- minisign or ssh-keygen -Y sign over
the .sha256 files, the public key baked into src/version.mc next to the version and rotated
with a release, ~90 lines of verification in .mc (Ed25519, no libc) or one more spawned tool --
and mc upgrade refusing an unsigned release once a key ships. Until then packages.md and
docs/ci.md say in one sentence what the .sha256 does and does not prove; mc pkg's tree hashes
are a different matter (they are pinned in the lock by the developer who reviewed the tree).
E. Files, acceptance, risks -- reworked #
Files and estimated deltas (replaces the draft's table; unchanged rows kept for completeness):
| file | lines | what |
|---|---|---|
src/version.mc | ~6, new | mc_version(); the 0.0.0-dev sentinel |
src/core_min.mc | +1 | version.mc before cli.mc |
src/cli.mc | +4 | --version; the usage line |
src/lex.mc | +60 | lopen_fn/lex_set_libs; lex_include_name = three steps + .mc strip; tok_add(".", 1) last in tok_init; lex_root_of + edges + the closure test in lex_include/#embed; the post-parse files accessor. The draft's named-root branch in lex_find_path_from is NOT added |
src/fetch.mc | ~150, new | fetch_get (https via the host's downloader, else local copy), fetch_extract, fetch_sha256_line (parse a .sha256 file), hex64 |
src/sysroot.mc | -75 / +12 | delegates to fetch_* |
src/deps.mc | ~400, new | name rule + reserved mc; [deps]/[replace]/[registry]; lock reader (rows carry lib); tree hash; libs_open (steps 1 and 3 of A3, the bundle.list map, lazily read once); semver; edge registration; the § 3 refusals; the B3 message |
src/install.mc | ~260, new | install_cmd: plan, --yes, --from, --libs-dir, --force; the from-blob road (the one definition of the layout); the tarball road; the manifest |
src/upgrade.mc | ~250, new | upgrade_cmd: LATEST, refusals, plan, download + verify + one-member extract, the per-host swap, the spawned install |
src/pkg.mc | ~700, new | index, MVS, lock writer, archive fetch + hash + manifest, vendor, [deps] edit, hash, check, list, verify, pkg_cmd; update_cmd |
src/core_pkg.mc | ~24, new | core_build.mc + pkg.mc; subcommand("pkg", ...), subcommand("update", ...) |
src/core_build.mc | +6 | fetch deps install upgrade in the list; install/upgrade registrations |
src/bundle_glue.mc, src/core_bundle.mc | ~14 new / -10 +1 | host_bundle_open + mc_bundle_init shared by the two bundle parts |
src/bundle_empty.mc, src/core_bundle_slim.mc, src/core_slim.mc | ~8 / ~12 / ~14, new | the slim assembly (B2) |
src/mc_slim.mc + 4 host slim entries; 4 *-slim-obj.toml | ~5 each / ~12 each, new | the ten release flavours' entries |
src/host_macos.mc, host_linux.mc, host_windows.mc | +10 / +10 / +16 | host_self_path, host_retire; MoveFileExA, GetModuleFileNameA externs |
scripts/sysroot-windows.sh, lib/sys_windows_host.mc | +2 / 0 | the two kernel32 names |
src/core.mc, src/main.mc | +1 / +1 | the sixth part; mc_pkg_init() |
src/toml.mc | +38 | toml_push/toml_pop, toml_occurrences |
src/driver.mc | +45 | --libs-dir; drv_apply_deps for both halves; <...> modules emitted verbatim; the post-parse files check |
tools/bundle.list, src/bundle_data.mc | +13, regenerated | mc/version, mc/fetch, mc/deps, mc/install, mc/upgrade, mc/pkg, mc/core_pkg, mc/bundle_glue, mc/bundle_empty, mc/core_bundle_slim, mc/core_slim, mc/bundle.list, the five slim entries are NOT bundled (they are entries, like mc/main... mc/main is; add mc/mc_slim only if a recreated slim compiler wants it -- not this milestone) |
Makefile | +30 | mc-slim, mc-linux-slim-obj, mc-linux-x86_64-slim-obj, mc-windows-slim-obj, mc-windows-x86_64-slim-obj, libs-tarball, check-pkg, check-slim, check-upgrade inside check |
scripts/set-version.sh | ~40, new | rewrite src/version.mc from a tag, make bundle |
scripts/release-assets.sh | +50 | --slim suffix; --libs mode via BINARY install --yes --from-bundle |
scripts/check-bundle.sh | +6 | the 0.0.0-dev sentinel guard |
scripts/check-slim.sh | ~190, new | Acceptance 20-23 |
scripts/check-upgrade.sh | ~160, new | Acceptance 24-26 |
scripts/check-pkg.sh | ~340, new | Acceptance 1-19 |
scripts/check-parts.sh, check-docs.sh | +10 / +3 | <mc/core_pkg>, <mc/core_bundle_slim>; the new families and flags |
.github/workflows/release.yml | +80 | set-version.sh; slim builds + cross objects; --libs; LATEST; slim links on the Linux/Windows runners; the install snippet |
.github/workflows/ci.yml | +2 | build/mc-slim in the uploaded artifacts |
tests/pkg/... fixtures, tests/golden/pkg-list.txt, .gitattributes | as in the draft | spelling <geo/geo.mc>; the float override fixture (tests/pkg/src/float-1.3.0/) |
docs/reference/packages.md | ~500, new | everything above: the resolution order, the layout of ~/.mc/libs, install/update/upgrade, the slim binary, the version, the trust model |
docs/reference/cli.md | +70 | install, update, upgrade, pkg, --version, --libs-dir; the exit-2 rows |
docs/reference/bundle.md, docs/build.md | +45 / +30 | "the full binary alone"; the mc package; mc/bundle.list; the slim assembly |
docs/ci.md | +90 | § Versioning amended; the eleven assets; LATEST; set-version.sh; what .sha256 proves |
docs/bootstrap.md, tests/golden/README.md, docs/determinism.md | +15 / +10 / +12 | the seed stays full; the dev sentinel and the goldens; the lock and the libs dir as inputs |
docs/guide/25-packages.md, docs/reference/toml.md, docs/reference/diagnostics.md | ~270 / +80 / +75 | as in the draft, plus the three verbs |
schivei/mc-registry | as in the draft | check.yml refuses name = "mc" and any mc/... |
stage0/ | 0 |
Net new src/ lines ~1 700 (the draft's ~1 350 plus install, upgrade, the slim assembly and the host
additions). The five goldens move once per gated commit, under M41's note 9.
Acceptance (ordered; scripts/check-pkg.sh, check-slim.sh, check-upgrade.sh; no network
anywhere; a curl shim on PATH that fails if invoked, for every step that must not download):
1-4. As in the draft (the fixture registry built by the script; the hash stable across hosts; the
plan printed and nothing fetched; MVS not "latest", the lock byte-identical), with --libs-dir and
$tmp/libs/<pack>/v<version>/.
<geo/geo.mc>resolves to the locked version:mc build tests/pkg/app --libs-dir $tmp/c1builds the taught compiler from<teach/mc_teach.mc>+user.mc, compilesmain.mc(which includes<geo/geo.mc>, whose#include <mathx/mathx.mc>resolves through the closure;<geo>alone resolves togeo.mcthrough the lock'slib), and the binary's stdout/exit matchmain.mc's header. With$tmp/libs/geo/v1.0.0/ALSO present, the object iscmp-identical to one built with onlyv1.2.0/present: the unlocked directory was never opened. 6-9. As in the draft (byte-identical objects from two fetches; the tampered source refused, exit 2; stale lock / stale tree, nocurl; vendoring is the offline road,deps/<pack>/wins).- A package is closed, spelled with
<bad/bad.mc>; the#embedcase;not declared in geo's [package].files. - Names:
[deps] mc = "1.0.0"->mc: reserved;Geo = ...->invalid package name; exit 1 at the key.[deps] float = "1.3.0"is ACCEPTED (see 18). 12-17. As in the draft (majors refused; a failed fetch leaves no claim;mc pkg addedits one line;mc pkg checkis the gate, and refusesname = "mc"; parts -- a probe compiler withoutcore_pkgbuilds the vendored app and prints a usage with nopkg/updateline but WITHinstall/upgrade; inert --check-inertclean,check-standalone,check-obj32/32,check-build,check-sysrootsunchanged,mc sysroot fetch linux-aarch64 --yesin CI still writes the four files). - A bundled name pinned in
[deps]overrides the bundle, byte-checked: the fixture packagefloat1.3.0 (a copy oflib/float.mc+float_rt.mcwith one visible change:putf64prints a!suffix) locked bytests/pkg/app-float/;mc buildwith the FULL binary produces a program that prints the!; the object iscmp-identical between the cache road and the vendored road; one byte appended to$tmp/libs/float/v1.3.0/float_rt.mc-> exit 2float 1.3.0: float_rt.mc does not match mc.lock; removing the[deps] floatline and re-syncing gives an objectcmp-identical to one built by a checkout with notests/pkgat all -- the override leaves nothing behind. - No lock, no network, the FULL binary resolves
<float>:build/mc-exe --exeonlib/mc_float.mc's shape in an empty directory withHOMEunset and thecurlshim -- the existingcheck-standalonesteps, re-run with the shim, plus<float>. - The SLIM binary before
mc install:build/mc-slim(fromsrc/mc_slim.mc, ~400 KB, asserted< 60%ofbuild/mc-exe's size) compilesi64 main() { return 42; }to an objectcmp-identical tobuild/mc-exe's;--dump-asmofsrc/arena.mcidentical between the two;mc-slim --exe hello.mc(<sys>+<prelude>) exits 2 with<sys> is not installed/tried:/run: mc install --yes;mc-slim install --libs-dir $tmp/libs(no--yes) prints the plan andnothing was downloaded, and$tmp/libsholds no manifest. -
mc installfrom a local tarball, and the standalone proof extended:make libs-tarball(release-assets.sh --libs 0.0.0-dev build/mc-exe dist) writesdist/mc-libs-0.0.0-dev.tar.gz.sha256, reproducibly (two runscmpequal);mc-slim install --yes --from dist --libs-dir $tmp/libspopulates$tmp/libs/mc/v0.0.0-dev/and writesv0.0.0-dev.tomllast; then, in an empty directory with the shim onPATH,mc-slimruns every step ofcheck-standalone.shwith--libs-dir $tmp/libs:helloruns (exit 42), the taught compiler from<mc/host>+<mc/core>+<user_syntax_demo>is built and signed, compiles<syntax_demo_test>, and<mc/host>+<mc/core>+<user_default>compiles to an objectcmp-identical tobuild/mc2.o-- the slim binary plus the installed package is the compiler, byte for byte. A tarball with one flipped byte in its.sha256->checksum mismatch for mc-libs-0.0.0-dev.tar.gz, no directory, no manifest; a secondinstallafter success saysis installed, exit 0.
- The FULL binary's
mc installneeds no network:build/mc-exe install --yes --libs-dir $tmp/libs2with the shim; the tree isdiff -r-identical to the one step 21 unpacked (the tarball came from the same road, so this is a reproducibility check of the road itself);src/bundle_data.mcin it is the mode-1 form andsrc/bundle.binhasbundle_bin_size()bytes. mc --versionprints0.0.0-devforbuild/mc-exeandbuild/mc-slim; a tree copied to$tmp/treewithscripts/set-version.sh 9.9.9applied (in the copy) and compiled withbuild/mc1 --exe $tmp/tree/src/mc.mc -o $tmp/rel/mcprints9.9.9;scripts/check-bundle.shon the COPY fails naming the sentinel, on the tree passes.mc upgradeagainst a local release directory, no network:release-assets.sh 9.9.9 macos-arm64 $tmp/rel/mc $tmp/reland--libs 9.9.9 $tmp/rel/mc $tmp/relproduce the two archives;cp build/mc-exe $tmp/bin/mc;$tmp/bin/mc upgrade --from $tmp/rel --libs-dir $tmp/libs3(no--yes) prints the plan naming$tmp/bin/mcand downloads nothing;$tmp/bin/mc upgrade --yes 9.9.9 --from $tmp/rel --libs-dir $tmp/libs3(the explicit form, since the running binary is a dev build) swaps the file, spawns the new binary'sinstall, and afterwards$tmp/bin/mc --versionprints9.9.9,$tmp/libs3/mc/v9.9.9.tomlexists, and$tmp/bin/mccompileshello.mcthrough--libs-dir $tmp/libs3.- The macOS in-place-overwrite hazard does not occur:
stat -f %i $tmp/bin/mcbefore and after step 24 differ (a new inode),codesign --verify --verbose=4 $tmp/bin/mcpasses, and$tmp/bin/mc --versionexits 0 -- notKilled: 9. On the Windows CI legs the same script assertsmc.exe.oldexists after the swap and is gone after a secondupgrade(a no-op one,is the newest release). - Refusals:
upgrade --yes 0.0.1 --from $tmp/relon the 9.9.9 binary without--yes... as specified: an older explicitVERSIONwithout--yes->older than this binary, exit 2, no file touched (inode unchanged); aLATESTfile saying0.0.1against the 9.9.9 binary -> refused even with--yes; a tarball whose.sha256does not match ->checksum mismatch, the binary untouched. - CI builds both flavours and
make checkis green withcheck-pkg,check-slim,check-upgradeinside it;check-partsshows<mc/core_min>+<mc/core_pkg>and<mc/core_min>+<mc/core_bundle_slim>compile alone; goldens rewritten once per gated commit after an empty--dump-asmdiff andcmp build/mc2.o build/mc3.o;make check-docsgreen.
Risks (in addition to the draft's 1-12, which stand; 10 is closed by C):
- A stale
~/.mc/libsfrom another version. Closed by construction: themcpackage lives underv<mc_version()>/and nothing else is consulted; the B3 message names the directory it looked for. What is NOT closed: disk growth across versions --mc install --prune(remove everymc/v*but the running one) is ~25 lines and optional. - The seed of the bootstrap must stay the FULL binary.
bootstrap-linux.sh/-windows.shdownloadmc-<VER>-<target>.tar.gz(the unsuffixed name) and stay as they are; the chain proper (src/mc_linux.mchas no<...>) would in fact close with a slim seed, but the release gate runs the whole suite with the seed and must not depend on~/.mcor on a second download.docs/bootstrap.mdsays so in one paragraph. - CI must build both flavours or a slim-only breakage (the empty-bundle assembly,
lopen_fnstep 3) ships unnoticed;check-slimis insidemake checkandrelease.ymllinks the slim object on every runner. Cost: five more cross-compiles and five more links per release, ~1 minute. - The slim binary and
check-standalone. The original script keeps proving the FULL binary; the slim proof (step 21) is a second script because its precondition (a libs tarball) is not the empty directory the original insists on. A drift between the two proofs is a documentation bug to watch: both compare againstbuild/mc2.o. - The
.token. Appended last intok_init, so no id shifts; but a taught module that relied on.being ABSENT (a#token "."of its own is fine --tok_addis idempotent -- but asyntax_lit/on_stmtthat saw1.5as three tokens on purpose is not) would change behaviour. No module in the tree does;check-lang,check-float,check-surfaceare the net. mc upgradeis the widest new attack surface: it writes over the compiler. D3 names what the.sha256proves; until a signing key ships,packages.mdsays "mc upgradetrusts the release host".--fromwith a local directory is also how a reviewer can stage an upgrade.set-version.shleaking into a commit moves the goldens and the bundle silently;check-bundle's sentinel guard makes it a redmake check.- Windows
mc.exe.old:MoveFileExAon a running executable succeeds, but an antivirus holding the file open can make the rename fail; the message names the file and asks the user to re-run, nothing is half-written (the new bytes are written only after the rename returns). host_self_path()and symlinks:/usr/local/bin/mc -> /opt/mc/0.10.2/mcgets the resolved TARGET replaced (that is what_NSGetExecutablePath/readlinkreturn), not the link; documented, not resolved. A binary on a read-only path fails atcreatwithcannot writenaming the path, exit 2.- A taught compiler built by the slim binary carries the full blob (its
src/bundle_data.mcon disk is the wholemcpackage): expected -- it is the same object the full binary builds (step 21) -- but a user who chose slim for size gets full-size taught compilers unless the project sayscore = "<mc/core_slim>", which is whycore_slimis bundled.
F. Amended decisions -- to ratify with the owner #
- D1' -- import spelling:
#include <pack/file.mc>, angle brackets, one resolution model (lock, bundle, installedmcpackage; never the working directory);<pack>alone is the package'slibentry;.becomes a core lexeme appended last intok_init, and a trailing.mcis stripped from every<...>name. Rejects: the draft's named roots for the quote form (two spellings for one thing) and<pack/file>without the extension as the only form (the owner's spelling, one token away). - D2' -- identity is a registry name;
[deps] name = "min.version";mc,mc/...,deps,buildreserved;lib/'s bundled names are NOT reserved. Rejects: the draft's blanket reservation of every bundled name (it forecloses the extensibility the ruling asks for). - D10' -- the cache is
~/.mc/libs/<pack>/v<version>/with<pack>/v<version>.tomlbeside;--libs-dir DIRon every reader;deps/<pack>/vendoring wins when present. Rejects:~/.mc/pkg/<name>/<version>/and--pkg-dir. - D15' -- the bundle is the
mcpackage: versioned with the compiler, embedded in the full binary, installable beside the slim one under~/.mc/libs/mc/v<version>/in the repository layout withbundle.listas the name map; a bundledlib/name pinned in[deps]overrides the blob, lock-driven and byte-checked; themcpackage itself can never be pinned. Rejects: a by-name layout (breaks../lib/prelude.mc), and refusing collisions (loses the override with no gain: the lock already pins content). - D18 -- two release flavours per target:
mc(full, ~760 KB, the seed and the default) andmc-slim(~400 KB by M41's table, an empty bundle assembled from<mc/core_bundle_slim>at zero core lines); eleven archives per release plusLATEST. Rejects: slim as the only flavour (the bootstrap seed and offline CI need the blob) and a slim binary that carrieslib/but notmc/core(a[compiler]section would then fail, andcheck-standalone's equality is the property being sold). - D19 --
mc install [--yes] [--from URL|DIR] [--libs-dir DIR], top-level, in<mc/core_build>: the compiler's own package at the compiler's version, from the blob (full binary, no network) or frommc-libs-<ver>.tar.gz+.sha256(verified before unpack, manifest written last);release-assets.sh --libsis that same road into a staging directory. Rejects: a hand-maintained member list in the script (two definitions of one layout) and anmc install NAMEalias formc pkg add. - D20 -- the version is baked:
src/version.mc(mc_version(), in<mc/core_min>, bundled asmc/version),0.0.0-devin the tree, rewritten byscripts/set-version.sh+make bundleinrelease.ymland never committed; the goldens are recorded for the dev tree and do not move per release;mc --version;check-bundleguards the sentinel. Rejects: the commit hash as the dev version (moves every golden every commit), a version outside the bundle (a taught compiler would report and install the wrong one), and aVERSIONfile (the reason it was deleted stands). - D21 --
install,update,upgradeare top-level subcommands;mc pkgkeepssync add list vendor verify hash check;mc pkg updateis removed;updateis<mc/core_pkg>'s, the other two are<mc/core_build>'s. Rejects: three moremc pkgverbs (the user-facing ones read likemc build, not like maintenance). - D22 --
mc upgrade [--yes] [VERSION] [--from URL|DIR]: the newest version from a one-lineLATESTasset atreleases/latest/download/LATEST; the host pair's archive of the running flavour plus.sha256, verified before unpack; the swap isunlink+ write on macOS and Linux (a new inode, never in place) andMoveFileExAto.old+ write on Windows; then the NEW binary'sinstall --yesis spawned; a downgrade needs--yes VERSION, a dev build needs the same, a staleLATESTis refused. Rejects: the GitHub API (JSON),curl -w '%{redirect_url}'(curl-only), in-place overwrite (Killed: 9), and running the old process's own install logic for the new version's libraries. - D23 -- the host layer gains
host_self_path()andhost_retire(path)(three externs:_NSGetExecutablePath,readlink,GetModuleFileNameA/MoveFileExA);rename,opendirandgetenvare still not added. Rejects:argv[0]as the binary's path (it is aPATHlookup). - D24 -- the full binary consults the lock before its bundle (the override) and never consults
the installed
mcpackage for its own names; with no lock its behaviour is byte for byte today's. Rejects: an unconditional~/.mc/libslookup (the working-directory-independence argument of M15, one level up). - D25 --
.sha256from the release origin is integrity, not authenticity; a signing key (minisign orssh-keygen -Y, public key insrc/version.mc) is the priced follow-up and not in this milestone. Rejects: shippingmc upgradewithout saying so inpackages.mdandci.md.
Architect's additions, extended: (d) step 24's local-release test is also the documented road for an
air-gapped upgrade (--from DIR), and packages.md shows it; (e) the release body's install snippet
lists the slim road second, never first -- the full binary stays what a newcomer downloads; (f) the
first commit of this milestone is src/version.mc + --version + the sentinel guard alone, so the
goldens move once for a six-line change and every later commit can be checked against a versioned
binary.