What it is
A consolidated field guide to the five harness-shaped efforts in this workspace — agentlab, sam-evals, doesitarm-harness, roof-frames, and harness-watch — each mapped to its directory, its mechanism, and its status.
Harness Atlas
One map of every test, benchmark, and eval harness in the workspace — what each is, which directory it lives in, and where they overlap.
Read the introduction Jump to the overlap map
Why it exists
These grew independently across two years and several domains. They share a spine — run a thing → capture evidence → score it deterministically — that’s easy to miss when each lives in its own folder. This is the map that makes the overlap visible.
The five harnesses
Section titled “The five harnesses” agentlab Overnight local-LLM coding-agent benchmark, sandboxed in Docker. Working.
sam-evals The planned public extraction of agentlab's engine. Planned.
doesitarm-harness Mac-app compatibility tester — drives apps, records evidence, classifies arch. Prototype.
roof-frames Radar-frame benchmark — renders frames, pixel-scores them vs TV/NOAA. Prototype.
harness-watch Weekly radar on the external agent-harness ecosystem. Active.
Start here
Section titled “Start here” Introduction What this atlas is and the question it answers.
The shared spine The run → capture → score shape all five share.
Comparison matrix All five, side by side, on one table.
Overlap & shared primitives Where they actually overlap — and what could merge.