Architecture

Why the tree looks the way it does. Read when changing structure, not every session.

Goal

Re-implement the OpenBSD kernel in Rust, file by file, preserving OpenBSD’s structure, names and semantics, as a standalone #![no_std] kernel for amd64 and arm64, booted by Limine, run in QEMU. The C tree under reference/openbsd-src/sys is the specification.

Four crates: hal, kernel, abi and the LZ tree bsd (decision 20, 2026-10-16)

sys/ holds four Cargo packages (docs/ZERO_UNSAFE.md section 0, the one diagram): sys/hal/, the hardware and the runtime, the only crate where unsafe may live; sys/kernel/, every decision of the operating system, #![forbid(unsafe_code)] at its root; sys/abi/, OpenBSD’s userland interface as the loadable module abi_openbsd.ko, forbid, translation only; and bsd at sys/ itself (OpenBSD’s kernel image is /bsd), the LZ tree with its subsystems as modules (kern, uvm, dev, ddb, sys, machine, arch), replaced subsystem by subsystem and deleted at N9. Dependencies go one way: bsd -> kernel -> hal, abi -> hal and abi -> KernelApi; hal depends on nothing of ours but libkern.

The cycles OpenBSD resolves at link time (trap → uvm_fault → pmap → tsleep → sched) were the reason bsd was one crate until 2026-10-16 (v1: a module could become a crate only if it used nothing from crate::{kern, uvm, arch, machine}). Under v2 an upward call is never a dependency: it is a trait kernel or hal declares and bsd implements and registers in hal’s registry at boot, one row of bridges.toml each (section 0.3). sys/lib/libkern and sys/lib/libz stay leaves (OpenBSD builds both as libraries too; libz holds what the kernel compiles of zlib: crc32, adler32, deflate, inflate; zlib licence).

No src/ directory ([lib] path = "lib.rs"), so C and Rust paths differ only by extension.

The runtime API (N2.3)

What kernel builds on, all safe to call, every unsafe inside hal with its proof (docs/ZERO_UNSAFE.md decisions 30, 32-36): hal::once::OnceStatic<T> (a global set once at boot), hal::mutex::Mutexed<T> and hal::rwlock::Rwlocked<T> (locks that own their data; the write guard sleeps with rwsleep and runs a closure unlocked), hal::percpu::PerCpu<T>, hal::refcnt::Refcnt, hal::intr::splraise (a guard that restores only a level it raised), hal::cpu (the CPU’s identity), hal::kmbuf::KmBuf (pageable kernel memory it owns), hal::pool::TypedPool<T> with PoolArc<T> (one pool(9) per type, so vmstat -m and systat pool name it), hal::malloc::MBox<T> (a T in a malloc(9) block of the C’s type), hal::user::UserPtr<T> with abi_pod! (typed user copies), hal::sleep (wakeup) and hal::cons (the console primitive). What still lives in bsd is reached through hal::bridges, a registry of safe traits bsd fills at boot (sys/kern/hal_bridges.rs, called by stand::start_kernel before main) and checked on every answer (sys/hal/trust.rs). At the N3 close, 21: IntrLevel, CpuId, Sleep, UserCopy, ModulePlatform (the boot entry), TlbCpus, KernelLock, SysInfo, VmProc, VmSysctl and pipe’s PipeGlue, UioMove, PipeFiles (paid by N4); BufCacheStats, VnodeOps (N5); NetLock (N6); Console, Random (N7); PoolBackend, Malloc, Printf (N9). N3 paid PageMemory, PhysPages’s row, KmMaps, KmPages and FaultMap with the kernel modules that implement them. Each is a row of bridges.toml; lz check counts them.

Path and name mapping

OpenBSD Here
sys/kern/subr_prf.c sys/kern/subr_prf.rs
sys/sys/proc.h (struct proc) sys/sys/proc.rs (pub struct Proc)
sys/kern/kern_fork.c (fork1()) sys/kern/kern_fork.rs (impl Proc { pub fn fork1 } or pub fn fork1)
sys/arch/amd64/amd64/pmap.c sys/arch/amd64/amd64/pmap.rs
sys/arch/amd64/include/pte.h sys/arch/amd64/include/pte.rs
sys/lib/libkern/strlcpy.c sys/lib/libkern/strlcpy.rs
sys/net/if.h, sys/netinet/in.h (Rust keywords) sys/net/if_.rs, sys/netinet/in_.rs (docs/C_TO_RUST.md)
sys/arch/*/stand/, boot(8), efiboot sys/stand/ (Limine glue) until M14, when the user decided (2026-10-03) to port boot(8)/efiboot and sys/lib/libsa/; Limine is scaffolding until boot(8) boots the same kernel in QEMU
sys/conf/, config(8), Makefiles, newvers.sh Cargo features and tools/xtask; ioconf.c is sys/arch/<arch>/conf/ioconf.rs, by hand (M7b); the vers.c that newvers.sh generates is sys/conf/vers.rs, fed by sys/build.rs (“The system’s identity”, below); not ported

Types live where the header is; functions live where the .c is. Rust allows inherent impl blocks in any module of the defining crate, which is exactly the header/implementation split.

Zones in a file (M15)

Every .rs under sys/ and tools/ (not init/) is split into zones by comment lines of their own: /* <LICENSES> */, /* <CODE> */, /* <TESTS> */, each with its closing marker, in that order (.claude/rules/rust-kernel.md has the layout, tools/xtask/src/layout.rs the validator that cargo xtask ports check runs). The reason is reading cost: a ported file is mostly licence text on top and tests at the bottom, and an agent that wants the code reads sed -n '/<CODE>/,/<\/CODE>/p' file.rs. The tests are inline in the same file, however long: the former <name>/tests.rs files (324 of them) became #[cfg(test)] mod tests { .. } in the TESTS zone, so the module path of every test (crate::x::tests::name) is unchanged and one file holds a module and its tests. Every LICENSES zone starts with the author’s ISC block (the user’s decision of 2026-10-09): before the original notices of a port, or alone in the ports whose C file has no licence text (or is generated, license = "none" in lineage.toml), the [[extra]] files and the files outside lineage.toml. One exception is written down: the test helpers other modules share (mod testutil;, fn foo_reset() under #[cfg(test)]) stay in CODE, since they are code for other files’ tests. init/ (a stand-in) is outside.

The machine contract

sys/machine/<header>.rs holds the traits standing in for <machine/*.h> and cpufunc.h, one module per OpenBSD header (param.rs, cpu.rs with boot(9), delay(9) and, since M5, curcpu() with the cpu_info accessors the clock code needs, cons.rs for consinit(), bus.rs for bus_space(9), db_machdep.rs for what ddb needs; later pmap.rs, intr.rs, …; autoconf.rs is what ioconf.c and the machine’s autoconf.c give subr_autoconf.c; signal.rs is <machine/signal.h> plus sendsig, sys_sigreturn and the signal trampoline), all re-exported from sys/machine/mod.rs, which also re-exports consinit(), bus.rs for bus_space(9) and (M7b) bus_dma(9), db_machdep.rs for what ddb needs; later pmap.rs, intr.rs, …; autoconf.rs is what ioconf.c and the machine’s autoconf.c give subr_autoconf.c; pci_machdep.rs (M7b) is <machine/pci_machdep.h>; conf.rs (M8) is the device switch each arch’s conf.c fills; isa_machdep.rs (M8) is <machine/isa_machdep.h>; disklabel.rs (M8) is <machine/disklabel.h> plus the machine’s disksubr.c, readdisklabel and writedisklabel; acpi_machdep.rs (M13) is the machine half of <dev/acpi/acpivar.h>, each arch’s acpi_machdep.c: acpi_map, the register maps, the SCI, the global lock, pwr_action, ci_acpi_proc_id, cpu_suspended, and the ACPI_PRT/ACPI_SECTWO constants that stand for acpi.c’s #ifdef __amd64__/__arm64__ walks; arm64 answers as a machine without ACPI until M14), all re-exported from sys/machine/mod.rs, which also re-exports crate::arch::current::Machine and asserts at compile time that it implements every trait. Generic code names only crate::machine. bus.rs also carries the C names as free functions (bus_space_read_1(t, h, o), bus_dmamap_load(t, map, ...)), so a driver reads like its original; the tag and handle types are the architecture’s (X86BusSpace/BusSpaceHandle on amd64, &'static BusSpace on arm64), and so are the DMA tag, map and segment types (each arch’s include/bus.rs; the tag is a table of functions, as in C). MI code reads a map’s public members by their C names (dm_nsegs, dm_segs(), ds_addr, …), which every arch must define (bus_dma_public_members checks it at compile time).

param.rs, vmparam.rs and exec.rs (the traits MachineParam, VmParam, MachineExec) live in sys/hal/machine/ since N2.2, and sys/machine/mod.rs re-exports them under their old names.

pmap.rs is an adapter over hal::pmap since N3.1 (docs/ZERO_UNSAFE.md 0.10): the Pmap trait’s methods call the safe API of hal, whose two backends (sys/hal/arch/amd64/pmap/, sys/hal/arch/arm64/pmap/) replaced the LZ pmaps, and the host double’s pmap is hal’s host backend (sys/hal/arch/host/pmap.rs). The adapter has no unsafe block: it holds the LegacyLend of the kernel map’s token (kernel::vm::km), and its three legacy unsafe fn signatures (pmap_kenter_pa, pmap_kremove and arm64’s pmap_kenter_cache) stay until their last caller moves. The page provider bootstrap takes as a value (decision 39) is the kernel’s since N3.3 (kernel::vm::page, sys/kernel/vm/page.rs); bsd’s BsdPages went with sys/uvm/.

Since N3, memory is the kernel’s (docs/ZERO_UNSAFE.md 0.10): sys/kernel/vm/ holds the page array and its free ranges, the owners (anon, amap, object, aobj, pager, vnode), the fault handler, the map and the vmspace, the vm system calls (Native entries of the module) and the rest of uvm’s 26 files, all #![forbid(unsafe_code)]. Their locks are hal::rwlock’s SharedRwlock with Guarded cells: a cell opens only for a witness of its own lock, and the fault carries its witnesses through its loop; the lock order (map, amap, object, anon) stays the code’s, as in the C, each acquisition saying where it stands. What they reach of the hardware is hal’s safe API: hal::pmap (the kernel pmap through the KernelMapper token, held by kernel::vm::km), MappedBytes for the pager’s windows, ForeignWindow for uvm_io’s copies, the intrusive lists and trees of the page system and the map. What they reach of bsd is a bridge of bridges.toml (VmProc and VmSysctl until N4’s Proc, VnodeOps until N5).

Constants travel the same way as functions: MachineParam (M1) carries <machine/param.h> and the alignment rules of <machine/_types.h> as associated consts, each arch defines them in arch/<arch>/include/{param,_types}.rs, and sys/sys/param.rs re-exports them, so generic code imports PAGE_SIZE from sys::param exactly as C includes <sys/param.h>.

Three implementors:

Boot flow

Limine (UEFI, both archs) → _start in sys/stand/mod.rs (protocol structs in sys/stand/limine.rs) → machine::BootInfo (bootloader-neutral: memory map, HHDM offset, kernel load addresses, DTB/RSDP pointers, the UEFI system table and memory map (efiboot’s openbsd,uefi-* properties), command line; it lives in sys/machine/bootinfo.rs so the machine traits can name it) → boothowto from the command line (BootInfo::boothowto, the boot(8) letters -a -c -d -s as arm64’s initarm parses them) → machine::Machine::early_init(&BootInfo) (OpenBSD’s init_x86_64 / initarm as far as they are ported: the message buffer, consinit(), and db_enter() for boot -d) → the bsd: booted on banner → kern::init_main::main (OpenBSD’s main() in the C’s order; every step whose subsystem is not here yet reports itself with unported!). Under feature qemu, main ends with the success exit where proc0 would go to sleep.

A panic anywhere (panic! is kern::subr_prf::panic through the crate’s panic handler) prints panic: <message> through db_printf, a frame-pointer stack trace (db_stack_dump → machine::DbMachdep::db_stack_trace_print, addresses only; cargo xtask symbolize --arch A names them offline from the ELF symbol table) and reaches reboot → machine::Cpu::boot, which, cold, halts; under feature qemu the “press any key” wait is the failure exit (status 35).

Leaving the machine goes through machine::Exit: under feature qemu, amd64 uses the isa-debug-exit device and arm64 the semihosting SYS_EXIT call, both making QEMU exit with status 33 for success and 35 for failure, which xtask smoke checks. Without the feature the CPU halts.

At entry Limine (protocol base revision 6) guarantees: 64-bit mode, MMU on, kernel mapped at 0xffffffff80000000, a higher-half direct map of physical memory (HHDM), a memory map, a stack of at least 64 KiB, interrupts masked, secondary CPUs parked. Limine’s page tables live in bootloader-reclaimable memory, so the kernel owns its own pmap and trap vectors before reclaiming (M3/M4).

Why Limine: it is the only option with an identical boot contract on amd64 and aarch64, which keeps the arch split focused on real kernel work (traps, pmap, interrupts). OpenBSD also keeps the bootloader separate from the kernel, so this is faithful in spirit. bootloader (crate) is x86_64-only; QEMU -kernel raw loading on virt would mean two unrelated early-boot paths.

The protocol is specified in https://github.com/limine-bootloader/limine-protocol (PROTOCOL.md, include/limine.h); sys/stand/limine.rs implements the subset the kernel asks for, at base revision 6, with no crate in between (see “Dependencies”).

Toolchain and targets

Stable Rust, pinned in rust-toolchain.toml.

No nightly: asm!/global_asm!, #[panic_handler], #[global_allocator], #[unsafe(no_mangle)] and #[unsafe(link_section)] are stable; core and alloc ship precompiled for both targets. extern "x86-interrupt" is unstable, so interrupt stubs are assembly, like OpenBSD’s vector.S. custom_test_frameworks is unstable, so in-QEMU tests are serial smoke tests driven by xtask.

Linking

sys/arch/<arch>/conf/kernel.ld (arm64: base 0xffffffff80000000 with physical addresses from 0 and the entry at _start’s offset, from M14; amd64: OpenBSD’s layout from M14, KERNTEXTOFF with physical addresses from 0x1000000, “Boot loaders”), PHDRS text/rodata/data, Limine request sections kept, .eh_frame/.note discarded. M16d: both scripts add OpenBSD’s openbsd_randomize header (PT_OPENBSD_RANDOMIZE) over .openbsd.randomdata, page-aligned inside the rodata segment, which boot(8)’s loadfile fills with its seed (dev/rnd.c’s entropy_pool0/rs_buf0, __guard_local); Limine fills nothing there, so a Limine boot starts the generator from a zero seed and says so (random_start’s “warning: no entropy supplied by boot loader”, the boot glue’s unported: line). arm64’s script provides etext too. sys/build.rs passes it with cargo:rustc-link-arg-bins only when target_os = "none".

Per-target rustflags in .cargo/config.toml: relocation-model=static (non-PIE higher-half kernel) and force-frame-pointers=yes (backtraces in panic). No [build] target, so host builds stay the default and cargo test just works.

abi_openbsd.ko (N2.4, sys/abi/ko, just build-abi-<arch>) is linked as efiboot is: RUSTFLAGS="-C relocation-model=pie" (replacing the kernel’s static), its own target directory (target/abi), CARGO_PROFILE_DEV_STRIP=debuginfo (boot(8) reads the whole file into efiboot’s 1 MiB heap), and sys/abi/ko/abi.ld: a static PIE at 0 with -pie --no-dynamic-linker -z notext -Bsymbolic --pack-dyn-relocs=none -z norelro, three page-aligned PT_LOADs (text R X; rodata R with the module header first at the ELF entry, the GOT and .data.rel.ro, relocated by the loader before it is mapped; data R W) and no import (section “abi_openbsd.ko”).

abi_openbsd.ko (N2.4)

The ABI crate is a loadable module (decision 22, docs/ZERO_UNSAFE.md 0.1). The pieces:

Cargo features ↔ option(4)

Feature OpenBSD Effect
alloc — extern crate alloc and the GlobalAlloc over malloc(9); default since M3
diagnostic option DIAGNOSTIC kassert! active
debug option DEBUG kdassert! active
kmemstats option KMEMSTATS malloc(9) statistics and per-type limits
pool_debug option POOL_DEBUG pool_debug = 1 (poisoning, once subr_poison.c is here)
ffs option FFS the fast file system (sys/ufs) and its vfsconflist[] entry; default
ffs2 option FFS2 FFS2 (UFS2 dinodes, the 64 KB super-block) in ffs; default
qemu — QEMU-only exits (isa-debug-exit, semihosting), the boot self-tests, the TSC under TCG and the uptime went backwards check
inet6 option INET6 IPv6: the #ifdef INET6 sites outside sys/netinet6 and inet6domain in domains[]; default, as in GENERIC. sys/netinet6 itself (and the IPv6 tables and usrreqs it names, route6_cache, tcp6_usrreqs, …) always compiles, like a library nothing reaches without the option, so the tree builds both ways
multiprocessor option MULTIPROCESSOR a default since decision 40 (the user, 2026-10-17): the MULTIPROCESSOR kernel is the only one built (every recipe, image, smoke, abi_openbsd.ko and host test; hal, kernel and abi have no such option, their code is MULTIPROCESSOR only), and bsd’s lib.rs refuses a build without it. Until then it was off by default and smoke-up booted a kernel built without it; now smoke-up boots this one with -smp 1. The uniprocessor branches still in bsd’s inherited and adapted modules are unbuilt and go when N4 redesigns those files. Since M11e every just smoke recipe boots it, since 2026-10-07 with -smp 2 but for the smp4 group (“Parallel smokes”). M11a: MAXCPUS 255/256, the kernel lock and the spinning mutex (kern_lock.c), the Limine MP request and the application processors’ start (see “Deviations”); just smoke-mp boots it with -smp 4. M11d: NET_TASKQ 8 softnet queues, softnet_percpu keeps one per CPU; just smoke-net-mp runs both VMs of the two-VM smokes on it
ntfs option NTFS the read-only NTFS file system (sys/ntfs) and its vfsconflist[] entry; default, but compiled only where the architecture’s GENERIC has it (amd64): see below
fuse option FUSE FUSE (sys/miscfs/fuse), its vfsconflist[] entry, cdevsw[] 92 (/dev/fuse0) and fuseattach in pdevinit[]; default, as in GENERIC
viocon amd64 GENERIC’s commented-out #viocon* at virtio? viocon(4) (M16d) in neither GENERIC: the feature stands for uncommenting the line, so amd64’s ioconf.rs gets the viocon* at virtio? entry and cdevsw[] 94 its entry points (NVIOCON 1, 0 without the feature as config(8) counts it: ENXIO); arm64’s GENERIC has no such line, so nothing changes there. dev/pv/viocon.rs compiles on every build, so just clippy and the host tests cover it either way; off by default

More appear as they are needed (small_kernel, …), one per option(4).

An option that only some architectures’ GENERIC sets (M10d: option NTFS, in arch/amd64/conf/GENERIC alone) cannot be a per-target cargo feature, so sys/build.rs plays config(8)’s part: its ARCH_OPTIONS table names the feature, a cfg and the architectures, and it emits the cfg (option_ntfs) when the feature is on and the target is one of them, or a host build (so the host tests cover the code). The code is gated on the cfg, not on the feature: the arm64 kernel has no NTFS, as OpenBSD’s arm64 GENERIC has none, and generic code still never names an architecture.

A machine-independent driver that files.<arch> lists only for some architectures and whose C uses those machines’ headers directly (M12: dev/fdt/pciecam.c, written against struct machine_pci_chipset, struct bus_space, struct machine_intr_handle) is gated the same way: sys/build.rs’s ARCH_MACHINE table emits cfg machine_pci_chipset for arm64 bare-metal builds (never for host, whose double has none of these headers), the driver’s module is #[cfg(machine_pci_chipset)], and it reaches the machine items through sys/machine/pci_chipset.rs, which re-exports them from crate::arch::current under the same cfg. Generic code still never names an architecture; the alternative, a dozen contract methods with fake amd64 and host implementations, would make the driver unlike its C and the fakes untestable anyway. The price: such a driver has no host tests, like arch code. M16e adds the x86 counterpart, cfg machine_x86 (amd64 only) with sys/machine/x86.rs, for dev/acpi/acpidmar.c (the VT-d and AMD-Vi IOMMUs), which builds its own struct pic and struct bus_dma_tag over the _bus_dma* functions, walks page tables through the direct map with pmap_flush_cache, reads bios_memmap and builds an MSI pci_intr_handle_t. There the driver’s items carry the cfg one by one, so the header part of the file (registers, table entries, source ids, the device scope parser) still compiles on every target and keeps its host tests. dev/acpi/acpicpu_x86.c (acpicpu(4), M16e) is written against it the same way: it sets the machine’s cpu_idle_cycle_fcn and cpu_suspend_cycle_fcn, links itself into struct cpu_info (ci_acpicpudev, ci_mwait) and idles with hlt, inb or monitor/mwait; its _CST/_PSS parsing and its choice of an idle state are plain functions with host tests. M16a’s ISA floppy stack is the third user: isadma(4)’s maps ask for BUS_DMA_24BIT (its attachment carries the cfg, the rest of isadma.c is generic), and fdc(4), fd(4) and fdlink.h are gated as whole modules, since nearly every item names <machine/ioctl_fd.h>’s struct fd_type or the RTC’s NVRAM diskette byte; their register headers (fdreg.h, nec765reg.h) compile everywhere with their host tests.

Dependencies

Crate Where Why it is not OpenBSD code
(none for Limine) sys/stand/limine.rs the limine crate was dropped: 0.6+ needs nightly (ptr_metadata), 0.5 is stable but frozen at base revision 3, which Limine has already tried to drop once. The protocol is about twenty #[repr(C)] structs; they are written from PROTOCOL.md
bitflags sys/ typed flag sets for #define groups; a macro, no runtime
(none for lists and trees) sys/sys/queue.rs, sys/sys/tree.rs intrusive-collections was dropped at M1: the OpenBSD macros are short, their semantics are the project’s to keep, and a crate’s policy changes would bind us as the limine crate’s did
libsa, boot, efi sys/lib/libsa, sys/stand/boot, sys/stand/efi not crates.io crates: OpenBSD code ported here (M14), split into crates as OpenBSD builds libsa as a library and compiles sys/stand/boot into every boot program; the boot loaders (sys/arch/amd64/stand/efiboot, package efiboot-amd64) depend on them, the kernel does not. libsa dev-depends on bsd for the host tests that pin its header layouts to the kernel’s
proptest dev-only property tests for libkern
serde, toml tools/xtask tracker parsing
fatfs tools/xtask writes the FAT boot image; a host tool, not kernel code

Not allowed: crates that replace OpenBSD code (x86_64, aarch64-cpu, spin, uart_16550, fdt, linked_list_allocator, buddy_system_allocator). Porting that code is the project.

The userland build (M8)

The userland is OpenBSD’s own C, cross-compiled unmodified (the user’s M8 decision), not ported. cargo xtask userland --arch A (tools/xtask/src/userland.rs, just userland) builds, into target/userland/<arch>/: the /usr/include sysroot as include/Makefile installs it (FILES, DIRS, LFILES/MFILES links, the kernel headers of LDIRS, <machine/*>, and of the RDIRS only lib/libutil’s headers and lib/librpcsvc’s rpcgen output, which libc’s YP code includes); lib/csu; libc.a (988 objects on amd64, 989 on arm64), libutil.a, libm.a (259 objects on both; the programs’ -lm), libkvm.a (kvm_getprocs, kvm_getfiles, … over sysctl(2) when no kernel image is named, the way ps, fstat and vmstat use it) and libcompiler_rt.a; and sbin/init, bin/ksh, bin/cat, bin/echo, bin/ls, usr.bin/uname, sbin/mount, sbin/mount_ffs, libexec/getty, usr.bin/login, libexec/login_passwd, the network tools sbin/ifconfig, sbin/ping (with its ping6 link, setuid root), sbin/route, sbin/pfctl and sbin/ipsecctl, the diagnostic tools bin/ps, bin/df, usr.bin/fstat (and its fuser link) and usr.bin/vmstat, the disk tools sbin/umount, sbin/newfs (with its mount_mfs link), sbin/fsck, sbin/fsck_ffs, sbin/disklabel and sbin/fdisk (M10a), the quota tools sbin/quotacheck, usr.sbin/quotaon (and quotaoff), usr.sbin/edquota, usr.sbin/repquota and usr.bin/quota (over librpcsvc.a, whose sources rpcgen makes from its .x files) with usr.bin/su, bin/mkdir and bin/chmod (with its chgrp and /sbin/chown links; M10b), and a few more as static PIE executables, the form OpenBSD’s cc -static gives /bin and /sbin (rcrt0.o relocates the program itself; no PT_INTERP).

A git worktree’s first build starts from the main checkout’s (the user’s decision of 2026-10-09: the OpenBSD sources and the toolchain are pinned, so compiling them again in every agent’s worktree is wasted time). When target/userland/<arch> (or, for comp, target/comp) is missing in a worktree and the main checkout has it, userland/seed.rs copies it with cp -Rcp: an APFS clone, instant and free of space until a file changes, that keeps the modification times the up-to-date checks compare. The build writes absolute paths into its records, so the copy’s rule stamps (*.cmd), dependency files (*.d), archive member lists, include manifests and absolute symbolic links are rebased from the main checkout’s path to the worktree’s, except paths into reference/, which a worktree shares with the main checkout. The build then runs as usual and redoes only what differs. Measured on 2026-10-09: a fresh worktree’s cargo xtask userland --arch amd64 took 23 s, every OpenBSD object (0 rebuilt), only the three tools/ programs (checked out after the main checkout’s objects) recompiled. A worktree whose own changes touch the userland rebuilds what they touch; a copy that cannot be made or rebased is removed and the build starts from nothing, as before.

Nothing is listed by hand. tools/xtask/src/bsdmake.rs evaluates the subset of make(1) the Makefiles use (assignments, lazy expansion, the :L :M :N :R :S :old=new modifiers, .if, .for, .include, .PATH, explicit rules) and fails on anything else. SRCS, OBJS, .PATH and CFLAGS come from it; explicit rules are run through /bin/sh as make would, so the system-call stubs are made exactly as lib/libc/sys/Makefile.inc makes them (GENERATE.* piped into FINISH.*), and so are the generated hash helpers and the rpcsvc headers. share/mk is not in the reference clone: sys.mk and bsd.own.mk are stood in for by predefined variables (CFLAGS?= -O2 -pipe ${DEBUG}, COMPILE.c, YP=yes, STATIC=-static, …), bsd.prog.mk/bsd.lib.mk by their variable effects (../Makefile.inc, COPTS) and the implicit .c.o/.S.o rules; CDIAGFLAGS (warnings only) is empty. Apple clang’s OpenBSD target supplies the rest of OpenBSD’s defaults by itself: PIE, -fstack-protector-strong, IBT (-fcf-protection=branch) on amd64, BTI and return-address signing on arm64, emulated TLS.

Workarounds, each printed by the build (flags only; no source is edited):

M9+ adds LibreSSL (lib/libcrypto, lib/libssl, lib/libtls), lib/libcurses (ncurses) and lib/libedit (LIBRARIES in userland.rs, the code in userland/libraries.rs), for usr.bin/ftp and usr.bin/nc (static, like login). Nothing new is listed by hand either:

Every OpenBSD file compiled or included is classified by licence into licences.txt; LibreSSL’s two licences are named OpenSSL and SSLeay there (accepted by the user on 2026-10-03), not counted as BSD-4-Clause.

The ramdisk image

The ffs image the kernel boots from (target/userland/<arch>/ramdisk.ffs, the rd(4) root) is made by OpenBSD’s own makefs(8), built for the Mac from usr.sbin/makefs (in the clone since 2026-10-03), not by a file system writer of our own (decided by the user on 2026-10-03). The on-disk format is then OpenBSD’s by construction. makefs runs as OpenBSD’s distrib/ runs it for its ramdisks (-t ffs -o disklabel=rdroot,minfree=0,density=4096). The rdroot entry is read by OpenBSD’s own getdiskbyname (lib/libc/gen/disklabel.c, built in) from a disktab xtask writes (target/userland/<arch>/host/disktab, OpenBSD’s /etc/disktab is not in the clone): one track of one cylinder spanning the image, partition a FFS at offset 0 with 4096/512 blocks/fragments. makefs’s own rdroot=1 label is not used: it leaves d_nsectors 0, which checkdisklabel rejects. The result is FFS1 in a and the label in sector 1; the size is twice the contents in whole MiB (at least 2 MiB), the timestamps fixed (-T). Its tree is root/ plus /etc, /dev and the directories below. M10c adds /root/images (images.rs): a FAT12 and an ISO 9660 image made by the same makefs (-t msdos, -t cd9660 -o rockridge) and a UDF image made by macOS’s own hdiutil makehybrid -udf (OpenBSD has no UDF writer; nothing is installed), for vnd(4) to attach in smoke-fs. Disk nodes follow MAKEDEV: UNITMULT 64 minors per unit (MAXPARTITIONSUNIT).

M10d adds /root/images/ntfs.img on amd64 only (ntfs is only in amd64’s kernel): a 4 MiB NTFS 3.1 volume made by a generator of our own, tools/xtask/src/ntfsgen.rs (ISC, written from the public description of the format, no code of ntfs-3g or any other implementation; cargo xtask ntfs-image OUT [--check] makes it alone). Why our own: OpenBSD has no NTFS writer, and the usual one, ntfs-3g’s mkntfs, does not build on macOS (decided by the user, 2026-10-04). It holds the 16 system files ($MFT .. $Extend, a full $UpCase, a $LogFile of 0xff bytes, which readers take as empty and clean), a root that is a large index (one INDX block), a resident m10d-ntfs.txt and a non-resident m10d-ntfs-big.txt. So that a generator bug shared with our kernel cannot pass unseen, every image is mounted read-only by an independent reader, macOS’s own NTFS driver (ntfs.fs, an FSKit module on macOS 26: diskutil mount readOnly over an hdiutil attach -nomount raw device, no root), which must list exactly the two files and read their bytes back (docs/SETUP.md, “NTFS check”).

/etc is our own minimal set (OpenBSD’s etc/ is not in the clone), text in ramdisk.rs: motd, shells, fstab (/dev/rd0a / ffs rw 1 1, which mount -uw / needs), ttys (a getty std.9600 on tty00, console off), gettytab, login.conf (a default class with auth=passwd, a daemon class), group, master.passwd (root, daemon, nobody; only root has a password, emibsd, docs/SETUP.md) and rc, a minimal script that runs mount -uw /, creates utmp, wtmp, lastlog and failedlogin and prints rc: multi-user. pwd.db, spwd.db and passwd are made by OpenBSD’s own pwd_mkdb(8) (-p -d <staging>/etc), built for the Mac like makefs from usr.sbin/pwd_mkdb (in the clone since 2026-10-03), over OpenBSD’s own db(3) (lib/libc/db, hash and btree) and pw_scan (lib/libutil/passwd.c), not macOS’s dbopen, so the databases have OpenBSD’s format by construction. The hash is OpenBSD’s bcrypt.c with blowfish.c, in a small helper that replaces arc4random_buf with a fixed salt before including the unmodified source, so the image is reproducible. For the network clients (M9+): resolv.conf (nameserver 10.0.2.3, QEMU’s user-network DNS), hosts (localhost, and emibsd-host for 10.0.2.2, QEMU’s alias of the Mac, where smoke --https-server runs test servers), and /etc/ssl: cert.pem, LibreSSL’s CA bundle (lib/libcrypto/cert.pem, as its distribution target installs it, 0444), and emibsd-test-ca.pem, the certificate of the test CA userland/testca.rs makes once with the Mac’s openssl (docs/SETUP.md, “The test CA”). With LibreSSL, ftp and nc the image is about 22 MiB, so the boot image (boot.rs, IMAGE_SECTORS) is 128 MiB. The directories are /home, /mnt, /root (0700), /tmp and /var/tmp (1777), /var/{log,mail,run}. /dev has console, tty, mem, kmem, null, zero, klog, tty00 (the console on both architectures: com0 on amd64, and on arm64 pluart0 takes com’s slot, major 8, in pluartcnattach), rd0{a,b,c} (block 17), rrd0{a,b,c} (47), sd{0,1}{a..p} (block 4) and rsd{0,1}{a..p} (character 13; minor unit * 16 + partition, 0640 root:operator, as MAKEDEV’s dodisk; the image has sd0 and sd1), fd/0..63 and stdin/stdout/stderr; the majors and minors, with their conf.c lines, are in DEVICES’s comment.

makefs is written for OpenBSD only; the Mac build takes host shims, all in tools/xtask/src/userland/ramdisk.rs and none in the sources: a force-included header (daddr_t is 64-bit, st_*tim, OpenBSD’s MAXBSIZE, no-op pledge/unveil, srandom_deterministic as srandom), OpenBSD headers macOS lacks taken from the clone (ufs/, msdosfs/, sys/disklabel.h, machine/disklabel.h, sys/uuid.h with uuid_t renamed), a sys/endian.h over <libkern/OSByteOrder.h>, scan_scaled from lib/libutil/fmt_scaled.c, cgetent pointed at $EMIBSD_DISKTAB, and an lstat wrapper for device nodes: macOS lets only root mknod and OpenBSD’s makefs has no mtree spec, so a staging file holding one emibsd-makefs-device c <major> <minor> <mode> line is reported to makefs as that device (with OpenBSD’s makedev() encoding). The same wrapper makes every file root:wheel and applies a table ($EMIBSD_OWNERS, mode uid gid path; paths relative to $EMIBSD_STAGING) for the exceptions (login_passwd setuid, spwd.db root:_shadow, master.passwd 0600, /tmp sticky, /dev modes); makefs would otherwise take owner and group from the host files, the building user’s. pwd_mkdb’s shims (passwd.rs) are likewise a force-included header (__BSD_VISIBLE, OpenBSD’s <pwd.h> before macOS’s, __dead, libc’s DEF_WEAK/PROTO_* macros as nothing, explicit_bzero, a check that the host is little-endian as the db(3) files are made in host order), OpenBSD’s <pwd.h>, <util.h>, <mpool.h> and hidden/db.h from the clone in a directory searched first, and a getgrnam("_shadow") that answers with the building user’s group (macOS has no such group, and pwd_mkdb insists on one).

EmiBSD’s own test programs (M10f) are the one thing in the userland build that is not OpenBSD’s C. OWN_PROGRAMS in userland.rs lists directories of this repository (tools/<name>/, paths relative to the workspace root), each with an OpenBSD-style Makefile (PROG, BINDIR, LDSTATIC= -static, NOMAN, .include <bsd.prog.mk>) and an ISC-licensed source naming the EmiBSD authors. They are built after PROGRAMS by the same build_prog, with the same clang, sysroot, libc and static-PIE link, and installed stripped under root/ where BINDIR says; only make_for differs, taking the Makefile’s directory from the workspace instead of the reference tree. The licence report lists reference-tree files only, so they do not appear in it. First user: tools/sr6create (/usr/sbin/sr6create), which creates a softraid(4) RAID 6 volume through BIOCCREATERAID; OpenBSD’s bioctl(8) refuses -c 6 (“unsupported RAID level”) although softraid_raid6.c is in the kernel, and the userland is compiled unmodified, so the test program does what bioctl’s bio_createraid() does for a non-crypto level, for level 6. Second user (M10d): tools/fusehello (/usr/sbin/fusehello), a read-only FUSE file system with a fixed tree (/hello.txt, /sub/deep.txt) over OpenBSD’s own libfuse (fuse_main with getattr, readdir, open, read and statfs), for smoke-fuse: OpenBSD has no FUSE file system of its own in base, only the library. Its Makefile adds -I${DESTDIR}/usr/include/fuse and -lfuse, what libfuse’s fuse.pc gives its users.

M10d adds lib/libfuse to LIBRARIES. Its includes rule makes /usr/include/fuse with install -d before installing its headers there; the install(1) stand-in records a -d dir line for that, and xtask makes the directory. Its sources include the kernel’s <sys/fusebuf.h> from the sysroot. ext2fs’s newfs_ext2fs(8), fsck_ext2fs(8) and mount_ext2fs(8) and mount_ntfs(8) are in PROGRAMS; mount_ntfs’s Makefile sets NOPROG= unless MACHINE is alpha, amd64 or i386, and build_prog then builds nothing, as bsd.prog.mk does, so arm64’s ramdisk has no mount_ntfs. The ramdisk has /dev/fuse0 (character 92, minor 0, 0600, as MAKEDEV makes it), the one node libfuse opens.

smoke-ext2fs checks the guest’s ext2 file system twice: with OpenBSD’s fsck_ext2fs(8) in the guest and with e2fsprogs on the Mac (cargo xtask e2fsck, tools/xtask/src/e2fs.rs; docs/SETUP.md, “e2fsprogs”), an independent implementation, so a bug shared by our kernel and OpenBSD’s tools cannot pass unseen. xtask finds partition a as readdoslabel does (the MBR’s 0xA6 partition, its label in sector 1) and hands e2fsck and debugfs image?offset=BYTES.

The comp set (M14)

The compiler is OpenBSD’s: clang and lld from gnu/llvm (LLVM 22.1.6, Apache-2.0 WITH LLVM-exception, compiled unmodified, not ported; the user’s decision of 2026-10-04), built by OpenBSD’s own build glue (gnu/usr.bin/clang, gnu/lib/libcxx, gnu/lib/libcxxabi, gnu/lib/libclang_rt, lib/librthread) through bsdmake.rs, with the userland’s toolchain. cargo xtask comp --arch A [--jobs N] (tools/xtask/src/userland/comp.rs, just comp) writes target/comp/:

Incremental: objects are remade only when a source, a header or the command changed (the .d files and .cmd stamps of userland), archives only when an object is newer, links only when an input is; installed headers keep their mtime when unchanged. just comp is not part of ci: a first build compiles about 2,850 C++ files per architecture (measured: 30 minutes for arm64 with 5 jobs, while other builds kept the Mac at a load average near 30), plus about 200 files of build tools for macOS, once; a run with nothing changed takes 15 to 20 seconds (most of it checking the objects’ .d files and the licence report). Its job count defaults to about half the Mac’s cores (COMP_JOBS): other builds share the machine.

What of OpenBSD’s comp set (distrib/sets/lists/comp/{mi,md.<arch>,clang.<arch>}) and of the base set’s compiler is built: clang (and links), ld.lld (ld), clang-scan-deps, llvm-config, llvm-objcopy (strip), llvm-objdump, llvm-readobj (llvm-readelf), llvm-symbolizer (llvm-addr2line), llvm-profdata, llvm-cov, ar (ranlib), libc++, libc++abi, libpthread, libclang_rt.*.a and the headers. Not built: lldb and lldb-server (BUILD_LLDB=no: the debugger needs ptrace(2) work and a host libLLVM for lldb-tblgen; the exit criterion is cc hello.c); the profiled *_p.a libraries (none are built anywhere here); /usr/include/llvm (libLLVM’s includes), binutils’ as, gdb, ldscripts (md.amd64; GPL, not in the clone); the mi list’s other libraries and tools that userland builds or not, as its lists say.

Workarounds, each printed by the build (flags only; no source is edited):

Deviations: the programs are static PIE executables, like the rest of the userland’s; OpenBSD links them dynamically and ships libLLVM as a shared library (NOLIBSTATIC), here libLLVM.a is linked into each program.

just smoke-cc (not in smokes: it needs just comp) boots the ramdisk kernel with comp.ffs as sd0, mounts it on /mnt, runs /mnt/usr/bin/clang --version (OpenBSD clang version 22.1.6, Target: amd64-unknown-openbsd8.0 or aarch64-unknown-openbsd8.0), compiles a hello world with cc --sysroot=/mnt -static and runs it. On the way the kernel reports one gap it works around: unported: amap_copy: chunking. Then the plain dynamic link of an installed system: chroot /mnt /usr/bin/cc -o /tmp/d /tmp/h.c (a dynamic PIE linked against libc.so.M.m with -dynamic-linker /usr/libexec/ld.so), run in the chroot, and ldd /mnt/tmp/d from the ramdisk, whose own ld.so and libc.so serve it.

Shared libraries and ld.so (M14)

A plain cc hello.c on OpenBSD makes a dynamic PIE, so the system needs the run-time link-editor and the shared libc. userland builds them (tools/xtask/src/userland/shlib.rs) after the static libraries, as OpenBSD’s Makefiles say:

Deviations, each written in shlib.rs: the host’s LLD 17 is given two defaults OpenBSD’s lld has built in (--undefined-version: libc’s Symbols.list names symbols that only other architectures define; --ignore-function-address-equality); ld.so is built with STACK_PROTECTOR= (its __guard_local and __stack_smash_handler stubs), the branch of the architectures OpenBSD’s clang has no retguard for, because the host’s clang has no retguard and emits the stack protector on amd64 and arm64 too; objects are linked in Makefile order, not shuffled (sort -R), and no relink kits (/usr/share/relink, ld.so.a) are made; test-ld.so is not built or run on the Mac (as in OpenBSD’s cross builds), smoke-cc is that test. The kernel side is uvm_mmap.c’s file half (uvm_mmapfile over uvn_attach; device mappings other than /dev/zero report the unported udv_attach). __thrsleep/__thrwakeup (kern_synch.c) are still sys_nosys: libc’s FILE locks and libpthread’s _rthread_dl_lock call them only when two threads contend.

The install media (M14c)

OpenBSD’s installer is run unmodified: distrib/miniroot/install.sub, the arch’s install.md, dot.profile, over a bsd.rd and signed sets that this build makes. Commands (all in tools/xtask/src/install.rs; recipes in the justfile, “the install media”):

bsd.rd. OpenBSD’s RAMDISK_CD kernel is a smaller GENERIC with option MINIROOTSIZE and rdsetroot(8) patches the file system into it. Here the kernel is our GENERIC MP kernel with cargo feature miniroot (sys/dev/rd.rs): rd_root_image[ROOTBYTES] and rd_root_size are compiled in under the C names, ROOTBYTES is $EMIBSD_MINIROOTSIZE sectors (the justfile’s miniroot_sectors, 65536 = 32 MiB, the miniroot is padded to it so one kernel serves every run), and rdattach roots on rd0a itself. cargo xtask rdsetroot (rdsetroot.rs) is usr.sbin/rdsetroot over our ELF; bsd.rd is that kernel with its debug information stripped. Deviation: the Limine module ramdisk.ffs path stays and wins. The kernel is built in target/bsdrd (in smoke-build only).

The miniroot (userland/miniroot.rs) is built from the arch’s list (distrib/amd64/ramdisk_cd/list, distrib/arm64/ramdisk/list) with list2sh.awk’s meaning (COPY SCRIPT LINK SYMLINK MKDIR REMOVE SPECIAL TZ TERMCAP), the directories of mtree.conf, and install.sub, install.md, dot.profile, group, master.passwd (pwd_mkdb), protocols, services, /dev/MAKEDEV. Deviations: no crunchgen instbin (each name is the static program userland built, from the normal Makefiles; distrib/special’s -DSMALL -Oz variants are not used, except init, which must be -DDEFAULT_STATE=single_user without DEBUGSHELL/SECURE, more, and doas: the media’s own is a root-only doas -u user command with no doas.conf, which install.sub’s unpriv runs ftp and signify through); /dev from the smoke ramdisk’s node table plus diskmap (MAKEDEV ramdisk makes it; opendev(3) opens a disk by its DUID through it, and install.sub names every partition by DUID); usr/mdec/mbr a 512-byte stub; no firmware, termcap or Raspberry Pi files; /etc/signify/openbsd-80-base.pub is a test key (userland/signify.rs: OpenBSD’s signify built for macOS with shims, key pair made once in target/install/test-signify), never OpenBSD’s. The entries not satisfied go to miniroot.txt.

The sets (userland/sets.rs): base80.tgz, comp80.tgz, bsd (our MP kernel as built, debug info kept; only bsd.rd is stripped), bsd.mp (the same kernel: OpenBSD ships a uniprocessor bsd beside it, and install.sub insists on bsd.mp when hw.ncpufound is above 1, installs it as /bsd and keeps bsd as /bsd.sp), bsd.rd, INSTALL.<arch> (host m4 on distrib/notes/INSTALL), SHA256 and SHA256.sig. A set is its list (distrib/sets/lists) applied to the tree this build has: userland’s and comp’s roots, etc/mtree/4.4BSD.dist’s directories, the etc distribution of etc/Makefile (as a table, with var/sysmerge/etc.tgz that install.sub extracts after base), zoneinfo from OpenBSD’s zic built for macOS, usr/mdec (efiboot; BOOTIA32.EFI is a zero sector because installboot copies it; biosboot is a placeholder i386 ELF whose one segment is a sector of text saying so, because installboot loads the BIOS boot record (md_loadboot) before it looks at the disk and never writes it on a GPT disk with an EFI system partition, while the BIOS boot programs are not built (stand/biosboot’s Makefile wants GNU as); BOOTX64.EFI and BOOTAA64.EFI come from target/efiboot/<arch>, and the set build fails without them). Archives are written by xtask (ustar with OpenBSD’s owner names and ids, hard links) and the host’s gzip. What the lists name and the build lacks is printed and written to MISSING-<set>.txt with a reason: perl, cvs, texinfo and GNU as, gdb, ld.bfd and the binutils tools are GPL and not in the clone (the user’s decision of 2026-10-05), and man pages, terminfo, locale, usr/share/misc, firmware, httpd/nsd/unbound files, the BIOS boot programs and the programs and libraries userland does not build are not built yet. Built files in no list are EXTRA-<set>.txt (our test programs). Every program is static, as in the rest of the userland.

The install run (install.rs): a fresh 3 GiB disk is sd0 (the install media’s own disk is sd1; on arm64 virt the virtio-mmio slots are probed top-down, so the disk QEMU is given last is sd0 there too). bsd.rd is booted by our efiboot (BOOTX64.EFI, BOOTAA64.EFI) from a disk laid out as OpenBSD’s miniroot80.img (efiboot-disk: the OpenBSD partition’s FFS holding /bsd, which is bsd.rd, and the EFI system partition; the run answers boot> ). Its ramdisk holds /auto_install.conf (autoinstall(8): .profile starts the install after five seconds when that file exists, the way OpenBSD does when no DHCP server names a response file; static 10.0.2.15, so dhcpleased is not needed), the sets come over HTTP from this machine (10.0.2.2:PORT, the server of diff-openbsd), the disk is set up by the arch’s md_prep_fdisk: on amd64 a GPT with an EFI system partition (fdisk -gy, answer G, so installboot(8) uses its EFI path and copies BOOTX64.EFI), on arm64 an MBR whose FAT partition (fdisk -iy -b 32768@32768:C, answer whole) installboot -p formats and installboot fills with BOOTAA64.EFI and startup.nsh; both are then partitioned by disklabel -T (a template: / 2400M first, so it is a, and 64M of swap), newfs, the sets (-all bsd bsd.mp base* comp*) verified by signify. The run ends at CONGRATULATIONS! and the installer’s reboot (boot(9)’s vfs_shutdown: “syncing disks… done”); a second boot of the plain bsd.rd makes the disk’s nodes (MAKEDEV sd0), prints its label, mounts it read-only and lists /bsd, /bsd.sp, /usr/bin/cc, /etc/rc, /usr/libexec/ld.so, amd64’s /etc/boot.conf (arm64’s installer writes none: its console is already the serial one), the EFI system partition (efi/BOOT/BOOTX64.EFI or bootaa64.efi, and the copy in efi/openbsd/) and runs fsck_ffs -n (five phases, clean).

install-boot copies the installed disk into a fresh VM’s boot image: the firmware (OVMF, EDK2 AArch64) starts \EFI\BOOT\BOOTX64.EFI (BOOTAA64.EFI) from its ESP, efiboot reads amd64’s boot.conf the installer wrote (stty com0 115200, set tty com0), loads /bsd and boots it; the kernel finds its root by boot(8)’s DUID, /etc/rc from the base set runs (fsck, pf, the network, rc.firsttime) to login:, root logs in and cc hello.c && ./a.out prints hello from cc 42 (the source is written with ksh’s print -r: base has no printf(1) yet).

On an ACPI arm64 machine (just smoke-install-arm64-acpi, smoke-install-boot-arm64-acpi: both runs with --acpi, QEMU virt,acpi=on, the disks and vio0 on PCI) the same media install and boot; there the media’s disk is probed first and is sd0, so the answers name the fresh disk sd1 (install.rs, target_sd, checked against the kernel’s sd1: 3072MB line), with the run’s disks in target/smoke/smoke-install-acpi and its logs in target/install/arm64/acpi.

Status (2026-10-05): both architectures pass both (smoke-install-amd64 about 3 minutes with the media already made, the installer itself 2.5; arm64 about 4.5, the installer 4; smoke-install-boot-<arch> under a minute each). What the installed system lacks shows in its /etc/rc run: the programs userland does not build (sort, head, cut, mktemp, find, install, printf, swapctl, ttyflags, kvm_mkdb, dev_mkdb, savecore, ssh-keygen, openssl, mail, …), so the rc.d daemons (syslogd, pflogd, ntpd, smtpd, sndiod, cron) say (failed): rc.subr accepts base’s /bin/ksh, the full build (install-boot fails on wrong shell); /dev/random (random was not a driver yet; rnd(4) and /dev/random came with M16d); /dev/ttyC* (wscons is not ported); installboot cannot add its UEFI boot entry (/dev/efi, efi(4), is not ported), so the firmware boots the ESP’s fallback \EFI\BOOT\BOOTX64.EFI (BOOTAA64.EFI). On arm64 the installed kernel has rd0 with no image; it once took the root disk’s DUID from a stale buffer and fsck failed (sys/dev/rd.rs, deviations: an empty rd0 reads no label now).

The release images (M16g)

OpenBSD ships its installer on two more images, which distrib/ makes from the install media: install80.img, a disk image for a USB stick with bsd.rd, efiboot and the sets on it (distrib/amd64/iso/Makefile, distrib/arm64/iso/Makefile), and amd64’s boot-only cd80.iso (distrib/amd64/ramdisk_cd/Makefile). xtask makes both from what install-media made (tools/xtask/src/distrib.rs; recipes install-img-<arch>, cd-iso-amd64), into target/install/<arch>:

Deviations: what the build does not make is not on the images (printed when they are made): the BIOS boot programs (the MBR’s code, biosboot, boot, cdbr, cdboot; stand/ wants GNU as), so the CD has no BIOS El Torito entry and its EFI image is the default entry; the game, man and X sets and BUILDINFO; arm64’s Raspberry Pi firmware and U-Boot (packages). The label’s d_uid is fixed per arch (I80AMD64, I80ARM64); makefs makes its own cylinder groups (newfs’s -c FSSIZE has no makefs option). The CD’s descriptors name EmiBSD and its author where mkhybrid’s command line names OpenBSD and its release engineer, as the kernel’s ostype does. makefs needs two workarounds (EXT-223 and EXT-224 in LZ’s external bugs log).

The smokes. just smoke-install80-<arch> (cargo xtask install80, part of ci-full, not smokes: two minutes on amd64, four on arm64) boots install80.img as a USB stick (boot.rs, BootMedium::UsbStick: a qemu-xhci of its own and usb-storage with bootindex=0, QEMU’s writes to an overlay) beside a fresh 2304 MB disk, small enough that disklabel’s automatic layout is one / and swap (alloc_small). The firmware boots efiboot from the stick (on amd64 the run types set tty com0 and boot at its prompt, as one does on a serial console; the kernel comes from boot.conf’s set image), and OpenBSD’s installer is answered over the console as a person answers it (Vm::respond_in_turn: a question asked twice gets its answers in turn, and one no answer covers fails the run at once): no response file, no network, no HTTP server; the sets come from disk, sd1, partition a, 8.0/<arch>, and the installer goes on without SHA256.sig. After CONGRATULATIONS! the installed disk boots to login: and compiles a program (install-boot’s run, install.rs::boot_installed). The fresh disk is sd0 and the stick sd1 (umass attaches after the virtio disk); the run checks both attach lines. At efiboot’s prompt each line is typed a character at a time, a character again when its echo does not come back (distrib.rs, type_echoed): after set tty com0 a boot typed at once under the load of four smokes once reached efiboot as b alone, and efiboot waited for the rest for ever. cargo xtask install80 --image FILE puts another image on the stick: with OpenBSD’s own install80.img (the diff-openbsd snapshot’s) the same answers install OpenBSD on both archs (2026-10-09), its installer asking the same questions in the same order, with the same automatic layout, but for one: EmiBSD’s amd64 installer also asks An EFI/GPT disk may not boot. Proceed?, because efi(4) is not ported, so efi0 at bios0 is not in the dmesg and install.md does not set MDEFI (M14’s known gap, as installboot’s /dev/efi). just smoke-cd80 (cargo xtask cd80, part of smoke, about a minute) boots cd80.iso as q35’s CD-ROM (ide-cd on AHCI’s first port): OVMF loads the El Torito EFI image, efiboot reads etc/boot.conf from the CD (cd0a:) and loads bsd.rd, the kernel attaches the drive (cd0 at scsibus1, ahci’s ATAPI) and the installer asks its first question; a shell then mounts the CD with mount_cd9660(8) and lists it by its Rock Ridge names.

Boot loaders (M14)

OpenBSD’s boot programs are ported as OpenBSD builds them (the user’s decision of 2026-10-03, M14; track A1 did amd64’s efiboot): three crates and one binary.

Building BOOTX64.EFI (just efiboot-amd64, part of just build). Deviations, each for a reason:

arm64’s efiboot, BOOTAA64.EFI (M14 track A3; just efiboot-arm64, part of just build, output target/efiboot/arm64/BOOTAA64.EFI, which installboot copies from /usr/mdec):

The kernel’s boot(8) entry (amd64, M14 track A2)

The same kernel ELF boots from Limine and from efiboot (the user’s decision: Limine retires only once boot(8) boots the kernel in QEMU).

The kernel’s boot(8) entry (arm64, M14 track A3)

The same arm64 kernel ELF boots from Limine and from arm64’s efiboot.

Deviations from OpenBSD (deliberate)

Every file-level deviation is in that file’s //! ## Deviations list and in ports.toml notes.

Testing architecture

See .claude/rules/testing.md. Host tests exist because of arch/host. QEMU smoke tests exist because custom_test_frameworks is unstable. Reference-backed tests exist because constants copied by hand drift.

Two-machine tests (the M9b/M9c tunnels) are cargo xtask smoke2 (tools/xtask/src/twovm.rs): two QEMUs of one arch run at once, each with its own image and EDK2 variable store, each driven by its own send-after/expect script; the run passes when both are done. The second virtio-net NIC of each VM (vio1) is on QEMU’s dgram netdev, a pair of UDP sockets on localhost: unlike socket,mcast= it does not depend on the host’s multicast routing, and unlike socket,listen=/connect= neither VM has to start first. No kernel change is needed: vio* at virtio? already attaches a second device on both archs. One harness detail follows from the kernel finding virtio-mmio slots bottom up while QEMU virt hands them out top down: on arm64 the link NIC is added to QEMU’s command line before the user-mode one, so that vio0 is still the user-mode NIC. just smoke-link is the first user and is not part of smoke.

QEMU’s command line makes EDK2 boot Limine at once: the boot image’s device has bootindex=0 (QEMU’s bootorder fw_cfg file puts it first in the firmware’s BootOrder, so the blank persistent disk is no longer tried first: BdsDxe: failed to load Boot0001 "UEFI Misc Device"), and -boot menu=on,splash-time=0 sets the boot manager’s timeout to 0 through etc/boot-menu-wait. ArmVirtQemu otherwise waits its platform default: about 5 s of every arm64 boot (firmware start to BdsDxe: starting went from 5.5 s to 0.5 s). OVMF’s default is already 0, so amd64 boots gain nothing measurable there.

Host tests of arch code

sys/arch/amd64 and sys/arch/arm64 are compiled for their bare targets only, so the tests in their files do not run under just test. Where an arch file holds plain logic worth testing that has no machine-independent home in OpenBSD, the host double compiles that file for its tests (M16e: amd64’s bios.c, the SMBIOS structure-table walk and its strings, which OpenBSD writes again in arm64’s dev/smbios.c): sys/arch/host/mod.rs has a #[cfg(test)] module whose #[path] is ../amd64, holding the file and the self-contained headers it reads in their own layout (include/biosvar.rs, include/smbiosvar.rs, amd64/bios.rs), so the file reaches its headers by relative paths (super::super::include) in both builds. What reaches the machine (bios0’s attach, its mappings, bios_efiinfo) is #[cfg(target_os = "none")] in the file. This adds no logic to the host double, which only names the files. hal does the same since N3.1: sys/hal/arch/host/mod.rs compiles, for their tests, each arch’s include/param.rs, include/pte.rs, include/vmparam.rs and the pure constants of its pmap (pmap/layout.rs) in their own layout (#[path = "../amd64"], "../arm64").

Parallel smokes

just smoke (and so just ci) runs its recipes several at a time, four by default, since a sequential run had grown to about 35 minutes of boots under TCG. Decided on 2026-10-04 (the user’s plan). The shape is one build phase, then one run phase:

JOBS=N just smoke (or just jobs=N smoke, or JOBS=N just ci) picks another N; JOBS=1 runs the recipes one after the other, still each in its own directory.

How many processors the VMs get (the user’s decision of 2026-10-07, replacing the -smp 4 of every recipe decided on 2026-10-03 for M11):

diff-openbsd

just diff-openbsd (cargo xtask diff-openbsd, tools/xtask/src/diffopenbsd.rs, M12+) runs the same scenarios on EmiBSD and on a real OpenBSD and compares them step by step. The smokes check that EmiBSD does what we expect; this checks that it does what OpenBSD does.

First results (2026-10-05): one stale stub fixed (amap_copy chunking, uvm_amap.c now ported); five expected differences on both archs: fifofs, exec_script.c, file mmap, core dumps (all visible EmiBSD stubs) and kern.ostype (branding). Since M14 exec_script.c and file mmap are ported, so their two entries are gone: three expected differences remain (fifofs, core dumps, branding).

Timings on the M-series Mac with the other milestone agents running: download about 1 minute for both images (1.5 GB); install and first boot 272 s (amd64) and 426 s (arm64), once. A run after that: about 35 to 65 s for amd64 and 80 s for arm64, image build included. It stays beside just ci, a documented recipe, not part of it: the first run needs the network and a snapshot that the mirrors drop within days, so ci could not stay reproducible on a fresh clone; and it adds no build step ci does not already do. testing.md says when to run it.

bench

just bench (cargo xtask bench, tools/xtask/src/bench.rs, N3) measures EmiBSD against OpenBSD 8.0 on the same emulated machine, so that a redesign of a hot path (N3’s uvm and pmap: page faults, fork, exec, mmap) has a before and an after. PHASE2’s goal 3 (“measured improvements only”) rests on it; a milestone close that moved a number carries Bench: trailers (git-commits.md), printed by cargo xtask bench compare BEFORE AFTER.