Spec M20 -- Windows x64: COFF x64 relocations and the Win64 ABI on the x86-64 machine

Goal: [target] os = "windows", arch = "x86_64" in mc.toml makes mc build emit a COFF .obj for AMD64, link it with lld-link against kernel32.lib, and the result runs on Windows on x64. Nothing in stage0/. Depends on M17 (walker + machine table, the x86-64 machine) and M19 (the COFF writer, the kernel32 layer, the sysroot and test scripts). Acceptance is "same as M19" (docs/plan.md). Line references below are to the tree at the M19 merge (77c4a62).

What is already done, and what is not #

Three of the five pieces M19 built are already architecture-neutral or already parameterised.

Design #

1. The Win64 ABI in src/machine_x86_64.mc #

The SysV machine partitions registers at machine_x86_64.mc:26-41: depths 0..3 in r8..r11 (XREG_BASE 8, XREG_MAX 3, x86_in_reg at :205), scratch rax/rcx/rdx, locals at [rbp - off], result in rax, callp pointer in rax (x86_callp:458-469). The calling convention lives in five places and nowhere else:

whereSysV todayWin64
x86_argreg[] (:174)rdi rsi rdx rcx r8 r9rcx rdx r8 r9
x86_param (:262-266)i < 6; 7th/8th at [rbp+16], [rbp+24]i < 4; 5th at [rbp+48], i.e. 16 + 32 + (i-4)*8
x86_push_args (:413-427)pushes args 7..n, pads 8 when the count is oddthe same, plus sub rsp, 32 after the pushes
x86_reg_args (:432-441)if (n > 6) n = 6if (n > 4) n = 4
x86_call/x86_callpif (back) add rsp, backunchanged -- back now includes the 32

The register partition does not change. rax, rcx, rdx, r8-r11 are volatile in both ABIs, so depths stay in r8..r11, scratch stays rax/rcx/rdx, and the callp pointer stays in rax. rdi and rsi become callee-saved on Win64 and the machine simply stops naming them: they appear only as SysV argument registers 0 and 1, and the Win64 x86_argreg table does not contain them. rbx, rbp, r12..r15 were already never written.

r8 and r9 being argument registers 3 and 4 on Win64 while also holding depths 0 and 1 is safe, and the proof is the SysV comment at machine_x86_64.mc:429-431 run the other way: x86_reg_args writes argreg[i] in ascending i, reading depth dbase + i. argreg[2] is r8, the register of depth 0, which can only be an argument source at index -dbase <= 0 < 2, so it has already been consumed. argreg[3] is r9, depth 1, index 1 - dbase <= 1 < 3. For x86_callp the pointer moves to rax before any argument register is written (:460), so the same argument holds with dbase + 1. The acceptance section asks for a TEST of this, not only the argument.

Shadow space. Win64 requires the CALLER to reserve 32 bytes below the return address before every call, and rsp 16-byte aligned at the call. Folding it into x86_push_args' return value is the smallest change and puts the bytes in the right place by construction: the pushes go first (so stack argument 5 lands at [rsp+32] once the shadow is subtracted), then sub rsp, 32. The alignment rule is unchanged -- 8*np + 32 == 0 (mod 16) iff np is even -- so the existing "pad 8 when the count is odd" branch stays, and if (back) ei(X_SPADD, ..., back) after the call gives all of it back at once. When na <= 4, np == 0 and back == 32, which is why x86_push_args must return non-zero even for a call with no stack arguments.

The frame is already 16-aligned (gen_walk.mc:892) and the prologue is already push rbp; mov rbp, rsp; sub rsp, N, so rsp is 0 mod 16 after the prologue and the call alignment holds unchanged.

How the two ABIs are selected. Three globals -- x86_nargreg (6 / 4), x86_shadow (0 / 32) and the argument table pointer -- read by x86_param, x86_push_args and x86_reg_args. They are set by MTASK_PROLOGUE, which gen_func (gen_walk.mc:876) always runs before the first MTASK_PARAM and before any MTASK_CALL, so they can never be stale. Registration becomes:

machine("x86_64", m_x86_64);            // MTASK_PROLOGUE = x86_prologue      (sets SysV)
machine("x86_64-win", m_x86_64_win);    // MTASK_PROLOGUE = x86_prologue_win  (sets Win64)

m_x86_64_win is a copy of m_x86_64 with one slot replaced; the other thirty entries are the same &fn. MTASK_INS_SIZE, MTASK_ENCODE, MTASK_DUMP, MTASK_RELOC_KIND and MTASK_RELOC_OFF are pure functions of the Ins record and are ABI-blind. MAXMACHINES is 8 (hooks.mc:373) and three are in use, so the ceiling holds.

This keeps the M17 rule intact -- the object backend names its machine as its first statement (docs/reference/machine.md) -- and --machine=x86_64-win makes --dump-asm show the Win64 sequence, which a runtime flag set only by the backend could not.

2. coff-obj-x86_64 in src/backend_coff.mc #

Parameterised the way backend_elf.mc was, not duplicated: an i64 coff_machine global (the exact counterpart of elf_em, backend_elf.mc:125), set by each entry point before gen_lower.

3. The entry point #

lib/sys_windows.mc:188-191 is the only AArch64 code in the layer, and the reason it is written that way is stated at :182-187: a file cannot both declare main extern and define it, and the "self" link mode (tests/windows/070-kernel32.mc) includes the layer into a program that defines main. It cannot simply be re-encoded for x86-64: emit() writes exactly four bytes (gen_walk.mc:578-592), a pending reloc() is pinned to the START of that word (gen_walk.mc:586, :857), and gen_reloc only accepts the four Mach-O kinds (gen_walk.mc:615). An x86 call rel32 is five bytes with its field one byte in.

The fix removes the raw words instead of doubling them. mc_start moves into its own bundled file, lib/sys_windows_start.mc (sys_windows_start), compiled once into winstart.obj and linked into every Windows executable:

extern i64  main(i64 argc, uptr argv);
extern i64  win_setup();          // splits GetCommandLineA(), returns argc
extern uptr win_argv();
extern void ExitProcess(i64 code);

i64 mc_start() { i64 argc = win_setup(); ExitProcess(main(argc, win_argv())); }

lib/sys_windows.mc keeps the wrappers and win_split and gains the two accessors; it loses win_call_main and mc_start. main is called through the ordinary MTASK_CALL path, so the relocation is BRANCH26 on ARM64 and R_X86_PLT32 -> IMAGE_REL_AMD64_REL32 on x64, both already correct. The layer becomes genuinely arch-neutral and tests/windows/070-kernel32.mc needs no // skip-x86_64:.

lld-link /entry:mc_start on x64 calls the entry point normally, so rsp is 8 mod 16 at its first instruction -- the same invariant every mc function is compiled against -- and the entry receives no arguments, which is why the command line is fetched rather than read off the stack.

Link lines become test.obj [+ winrt.obj] + winstart.obj + kernel32.lib; the self mode drops winrt.obj because the source carries the wrappers, and both modes now carry winstart.obj. This is a behaviour change for windows/arm64 too, and the existing ARM64 CI leg is its gate.

4. Scripts, tests, Makefile #

5. CI #

6. Bundle and goldens #

7. Docs #

docs/reference/objects.md (a coff-obj-x86_64 row in the backend table, a § 4c Win64 contract mirroring § 4b, and one paragraph on why COFF REL32 needs no addend where ELF needs -4), docs/reference/machine.md (a Win64 column in the "x86-64 implementation" table and the x86_64-win machine name), docs/build.md § Windows targets, docs/guide/50-cross-compile.md, docs/reference/bundle.md, docs/reference/cli.md (--machine=x86_64-win), docs/ci.md, CLAUDE.md § State, docs/plan.md M20 row.

Out of scope #

Files (estimated deltas) #

filenowdeltawhat
src/machine_x86_64.mc775+45Win64 arg table, x86_nargreg/x86_shadow, x86_prologue_win, the second task table
src/backend_coff.mc380+45coff_machine, AMD64 + REL32/ADDR64, coff_rel_type_x86, backend_coff_x86
src/main.mc234+2one backend(), one target()
lib/sys_windows.mc196-8drop win_call_main/mc_start, add win_setup/win_argv
lib/sys_windows_start.mcnew+30mc_start, four externs, the comment that explains the split
tools/bundle.list42+1sys_windows_start
scripts/test-windows.sh350+45--arch x86_64, two-level skip_reason, winstart.obj in every link
scripts/sysroot-windows.sh810already parameterised
Makefile+14sysroot-windows-x86_64, test-windows-x86_64, check, .PHONY, check-skipped
.github/workflows/ci.yml+60cross-compile step + artifact + the windows-x86_64 job
tests/windows/+2 filesthe two ABI tests below
docs/*+120the pages in § 7
tests/golden/*.sha256rewrittenthree files, once

Acceptance (same as M19, plus two tests) #

scripts/test-windows.sh --arch x86_64 --build-only produces objects for every non-skipped test; llvm-readobj --coff-* --relocs --symbols --sections output matches clang --target=x86_64-windows-msvc -c of equivalent C for headers, sections, symbols and relocation types; lld-link -machine:x64 links 001, 013 and 070-kernel32 into .exe files on macOS; the CI leg Link and run the suite (windows/x86_64) runs the suite on windows-latest with the expected stdout and exit codes; the existing windows/arm64 leg stays green with winstart.obj in its link lines; make check green; --dump-asm unchanged for arm64 and for --machine=x86_64; goldens rewritten once.

Two tests under tests/windows/, portable to both Windows architectures and run by both legs:

Additional verification, copying M17 step B: every distinct instruction the Win64 machine emits while compiling src/mc.mc fed back through llvm-mc -triple=x86_64-windows-msvc and required byte-identical.

Risks #

  1. Shadow space forgotten on one path. A missing sub rsp, 32 corrupts nothing on a call into mc code (mc callees do not spill to their home space) and corrupts the caller's frame only on a call into kernel32 -- so it can pass every test that does not do I/O. Mitigation: the shadow lives in x86_push_args alone, which both x86_call and x86_callp go through, plus test 072.
  2. Stack-parameter offset off by the shadow. [rbp+48] vs [rbp+16]; only functions with more than four parameters are affected. Test 072.
  3. lld-link on windows-latest under Git Bash. The dash form is already used and must stay.
  4. The bundle/golden churn touches three goldens and the blob. Regenerate the bundle before bootstrapping, or check-bundle reports a stale bundle as a golden mismatch.
  5. The layer split is a behaviour change for windows/arm64 too. Every existing Windows link line gains winstart.obj; the ARM64 CI leg is the regression gate and must be green in the same PR.

Decisions (architect, 2026-09-04) #

  1. ABI selection: a second machine x86_64-win registered from the same file, sharing every encoder, with the three ABI globals set by that machine's MTASK_PROLOGUE. Not a runtime flag: --machine=x86_64-win must be able to dump the Win64 sequence.
  2. Shadow space: folded into x86_push_args' return value. Not reserved in the frame.
  3. Entry shim: mc_start moves to lib/sys_windows_start.mc with extern main; the raw AArch64 words are deleted, not duplicated. Allowing an extern-declared function to be defined later is a language change and is not taken here.
  4. COFF writer: parameterised with coff_machine, mirroring elf_em. Not duplicated.
  5. Skip headers: reuse // skip-x86_64: with a two-level skip_reason. No new header.
  6. Machine name: x86_64-win.
  7. .pdata/.xdata: the same accepted gap, recorded for x64 in the existing section.
  8. CI runner: windows-latest; release.yml untouched.
  9. Win64 register contract: documented in objects.md § 4c and verified by the llvm-mc sweep and by tests 071/072; check-surface does not grow.
  10. Depth registers: r8..r11 kept. The ascending-order argument is required to be backed by test 071, not asserted alone.

Edit this page