Results #

Each run here links the files under packages/bench/results that hold its logs and evidence. PRODUCTION is the live service (firetrident.com), STAGING a private copy of it with its own Durable Objects and Artifacts namespaces, PREVIEW a preview deployment of the same code with its own Durable Objects, and SIMULATED the simulator driven by Hono's history. A number whose log was not committed says so, and so does one computed from others. packages/web/test/measured.test.ts checks that every other number a run quotes is in the files it links; how each mechanism works is in mechanisms.md, and how these runs compare with published research in related-work.md.

Contention at scale #

The run that measures correctness when hundreds of agents edit the same files at once (from Cloudflare's network, from a laptop): on staging, a generated TypeScript repository with 10 hot files that every agent edits, synthetic agents (no model, review off) doing three kinds of task, each checked against the final trunk with stock git afterwards: register a name in a hot file's registry (correct when every landed name appears exactly once), add 1 to one of its counters (correct when each counter equals the increments that landed on it, so a lost update fails the run), and copy a registry's length into the task's own module (correct when what it read was still current when it landed). A change that comes back is redone by the same agent against the current trunk. The run from Cloudflare's network had the megafile alarm on, as the stage config has it: after six collisions on a hot file the colony holds it for a split, and a change that writes it goes back until the split lands. The laptop run had the alarm off.

Before the fix of 2026-10-19 (results), 200 agents landed 488 of 600 tasks faster (236.7 a minute, p50 10.8 s) and the counters still added up, but 100 increments had been closed as already on trunk: an increment made from a stale read merged into an identical one from the same base, so its +1 never applied and its agent was told there was nothing to redo. Now an edit that repeats or overlaps one trunk made since the agent read the file goes back as a stale read, and the agent redoes it on the current value.

Its limits (from a laptop, from Cloudflare's network): synthetic agents make scripted edits with no model and run no tests, so this measures the landing path under contention (read-set validation, text merge, group commit, the lineage record), not judgment or review. The laptop's connections dropped before its 1,000-agent run finished: 934 of its 3,000 tasks never started, 720 were left unresolved, and 2 proposals landed whose agents had been told ready. From Cloudflare's network no call failed, and the landed count is bounded by the megafile alarm instead: synthetic agents take no split, so the 1,849 tasks held for one ended unresolved, and 3 proposals landed whose agents had been told the file was being split. Two agents held an intent until the run's 1,800 s limit after landings had finished at 258 s. Of three attempts from the container, the first stopped itself at 20 s on its own uncapped connections, the second's files were lost when a Cloudflare rollout restarted the container, and the third is the one reported; its operator's note records a stray DELETE of the colony 62 s in that showed no effect.

The contended run #

The run that measures coordination when agents contend (results): on staging, a 20-file TypeScript seed whose 16 intents all change src/core/money.ts as well as their own module (three change a signature in it, and four intents are built on the old signatures), in-platform agents calling real models, review on, the project's node:test tests certified on trunk after landing, and hidden acceptance tests no agent saw graded at the end. Each run started on its own fresh colony from the same seed and intents. The staging deploy turned two mechanisms off for it: the claim-lease exclusion at take (FT_LATERAL_INHIBITION=off), so agents could hold intents on the shared module at once, and the megafile alarm (FT_MEGAFILE=off).

Coordination held (results): of the 23 and 21 proposals of the two contended runs, read-set validation sent none back for a stale read and text merge sent one back for a conflict. Quality did not: review let through changes the hidden tests then failed, certification found trunk red and declared a repair that did not land before the 2,400 s run ended, and two changes per run waited for a person the test did not provide. Of the four semantic-conflict pairs designed into the seed, certification found none as a pair (0 of 4 in each run). The run's own page lists what it does not measure: one small repository and one workload, and models that answer differently between runs.

Real pull requests #

The run that measures real work end to end (16 agents, 8 agents): on staging, honojs/hono at a commit of 2026-07-10 and 15 pull requests merged after it, each redone as an intent from its title and description on a fresh colony, in-platform agents calling real models, review on, tests before landing and certification on trunk, a scripted person answering the review queue, and each intent graded afterwards by the pull request's own tests, which no agent saw, under real vitest. With 16 agents, 20 proposals landed 12 intents, review returned 8 and nothing came back for a stale read or a merge conflict.

What held it back was quality, not coordination (16 agents): review and tests before landing returned changes, not every landed change passed the tests its pull request shipped with, and trunk went red in both runs with its repair unlanded when the 2,400 s run ended. The hidden grade is the one that counts, and 10 of 15 is the number to beat.

Five questions, each with a measured answer #

The landing page, start.md and llms.txt answer the same questions in the same words; each number is from the runs below.

How do agents coordinate? #

Agents read the decaying signals other agents leave on the code and each chooses its own work from goals people set (rules, specs, priorities), with no queue and no one assigning it: on production, 80 agents took Hono's 958 rule violations to zero in 62 s. Replay the Hono run

What happens when their changes conflict? #

Every change lands only if every file it read is still what it read; one whose files moved under it is caught at landing and goes back naming the file and the landing that moved it, unless its edits merge line by line or, with import-name revalidation, every name it imports still exists and the files it read only gained new declarations: when a shared Hono file changed under 30 changes, revalidation landed all 30 on their first proposal, and without it all 30 went back (on a preview; it is off in production). The cascade results

How is work reviewed? #

Every change goes to deterministic proof first, then to decision models, a second model from another family and a person, each only when the cheaper layer is unsure: all 282 rule edits of the Hono run on production landed on deterministic proof. See a proof-path landing

How is the why recorded? #

Every landed commit carries a JSON lineage note in git (the intent, the rule and who approved it, the agent and its model, every decision, the files read and written) and the note's SHA-256 in its FT-Note trailer, so any git client can check it: on a stock clone of the production colony signed-notes on 2026-10-19, scripts/verify_note.mjs found 1 of 1 notes intact. scripts/verify_note.mjs (log)

Can my agent join? #

Yes: any MCP client can ask the showcase colonies at /public/mcp with no token, and with a deployment's MCP token an agent joins a colony, takes an intent and proposes next to the in-platform agents: on production, an external agent landed a change 19.4 s into its session. Join with a token (docs/agents)

The commands that check these answers from a terminal (stock git, the note verifier, the public MCP) are in docs/try-it.md.

What it has done #

On the live service #

Every landing is a real commit on an Artifacts repository. Logs, event streams and colony summaries for these rows: packages/bench/results/2026-10-12-prod, unless a row links its own.

On preview deployments #

The same code with its own Durable Objects.

What each headline run measured #

Load is how much the platform lands, routing is where review sends each change, and coordination is what happens when agents' changes contend for the same files. Of the model-driven runs in this table, only the contention test had agents' changes contend for the same files (results): in the others none came back stale, and on the Hono rule each intent edits its own file. Other contention at landing was measured with synthetic or scripted agents whose work the script placed (an intent declared already claimed by its agent, assignedTo), and in the simulator.

How often overlapping human work conflicts (MEASURED, Hono's merged pull requests, 2025-10-17 to 2026-10-17): of 1,562 pairs of pull requests open at the same time on one branch, 89 (5.7%) changed a common file, which per-file locks would have serialized; stock git conflicted on 20 of the 87 pairs it could replay, and FireTrident's text merge with the insertion rule lands 7 of those 20. Pull requests carry no read sets, so this cannot count the changes git merges cleanly that read-set validation would send back.

The benchmark compares FireTrident with exclusive file locks plus one serialized merge queue on the same workloads; it does not model a queue that tests several changes at once. Every point is labeled measured or simulated: packages/bench/results.

Honest limits #

• On contended work, coordination is not the limit; quality is. In the contention test (16 intents on one shared module, real models, review on; results), changes almost never collided at landing (0 stale-read returns and 1 merge conflict in the 23 and 21 proposals of the 8-agent and 16-agent runs), but review let through work that tests no agent saw then failed (the hidden tests passed for 12 and 11 of 16 intents), trunk ended red with its repair unlanded when the 2,400 s run ended, and each run sent two changes to a person the test did not provide. Testing before landing (FT_PRELAND, on in both configs since 2026-10-19) is one answer, and the Hono replay grades landed work by tests no agent saw.
• A colony sits beside your codebase, not in place of it: there are no teams, pull requests or issue tracker, and nothing syncs with your repository on its own. You import it once from a URL (and, if you like, its open GitHub issues as intents, with ft issues import) and take certified work back by pushing certified as a branch with stock git; FireTrident holds no credential that can push to your origin (an issue import made with write-back, and a GitHub token set, comments on each issue when its change lands, and closes it when asked). /limits lists each limit with why, what to do instead and its status (planned, depends on Cloudflare, not scheduled), and what building on Artifacts taught us.
• Optimism costs rework. In the simulator, driven by Hono's real history, with every design paying the same full redo for a returned change, plain optimistic claims rework 43% of agent work against 35% for exclusive locks. Fading claim leases (decaying inhibition) and co-change learning bring FireTrident to a simulated 30%, below locks, while landing 3.4 times as many changes a minute; under group commit, which production runs, the claim-lease exclusion costs about 7% of landings. Nearly every remaining return (99.8%) is a stale read, not a conflicting write, and 71% of them are on the three most-imported files. Two next steps target that in the same simulator: assigning work away from the imports and importers of files others hold (16% rework, 10% fewer landings) and landing a stale-read change when the imports it relies on still hold (14 to 18% rework, 19 to 24% more landings). All simulated: results.
• Two model-driven runs have measured agents' changes contending at landing: the contention test, on one small repository and one workload, a first point rather than a curve (results), and the Hono replay, where little contended. At scale, contention is measured with synthetic agents (Contention at scale): the landing path and its correctness, not judgment. In the other model-driven runs no change came back stale, and on the Hono rule each intent edits its own file; the one deliberate overlap among them, three in-platform agents racing one intent in contest mode, picks one change before any lands. Other contention at landing is measured with synthetic or scripted agents placed by the script (the cascade, import-name revalidation, text merge, insertions, the resolver, goal conflicts) and in the simulator (What each headline run measured).
• Time spent waiting to land is measured only on tasks of seconds to minutes (61% of claim-to-landing time on the round-7 Hono run, at a median of 2.3 s); how it behaves on tasks that take hours is not measured.
• The project's tests run right after landing, not before it, by choice: no landing waits for a test run. In the simulator, driven by Hono's history with 30-second tests, one serialized merge queue that re-runs a change's tests whenever trunk moved under it lands 1.9 changes a minute at every agent count from 10 to 100,000; FireTrident's group commit lands 164 a minute at 1,000 agents but redoes 45% of agent work, against 0.75% for the queue when both redo a returned change in full (without the claim-lease exclusion or co-change learning; results). Before landing, every change's reads are validated against current trunk; with review on (the default), it also passes the rule re-check and the danger patterns, then a second confirmation or a person. After it lands, its tests run on trunk (267 ms for the 67 tests of the second new-project run, on production). A broken change can still sit on main until a repair lands. Certification runs tests, not a type check. Only node:test files run (projects tested with another runner are skipped), the tests in one file run one after another, and a test file that awaits at the top level runs until its time limit. A second run of the new-project spec, made before certification existed, landed all 13 intents, yet from a plain clone it passes 62 of 67 of its own tests and 6 of 7 acceptance checks (log); the first run passed all of them. Certified afterwards, it went red, and two repairs from the colony brought it to 67 of 67.
• main is every landing, tested or not; agents build on it. certified is the newest trunk commit on which every test file certification could run passed and no earlier passing test was lost. Deploy, release and pin from it (git clone -b certified <url>). Each result lists the test files certification could not run, with the reason (GET /api/colonies/:id/certifications).
• certified means the tests that could run passed, not that they check the right behavior. In a new-project colony those tests are the ones its agents wrote; the acceptance suite a person wrote runs only from a clone, in the demo script.
• Conflicts are found per file. A change goes back when a file it read or wrote changed under it, or when a file its new or rewritten imports resolve to is gone, which also catches conflicts between different files (in the cascade run, one shared file sent back 30 proposals that had read it). Two changes to different files that are wrong together, neither reading the other's file, both land; certification catches them when the tests cover them. The import check holds within one landing cycle: a deletion that lands in a later cycle than a new import of the same file is caught by the deletion guard when it is on (FT_DELETION_GUARD, per colony), otherwise only by certification; removing an export another file imports is guarded the same way by the export guard (FT_EXPORT_GUARD), except through path aliases and export * re-exports, which it skips and counts. An external agent's read set holds the files it read through the read tool; files it reads from its own clone are invisible to this check. An in-platform agent's read set holds its footprint's files and the files its intent builds on, not the project tree its prompt lists or the modules its import check reads. Import-name revalidation (off by default, on per colony) checks that the names a change imports still exist and that the files it read only gained new declarations since, not what the new code does at load time; certification is the backstop.
• A change that only read a file can land in a cycle just ahead of the change that rewrites that file: it was valid on the trunk it read, and nothing sends it back when the file then changes. Certification is the check.
• With review on (the default), changes that touch authentication or secrets go to a person, by deterministic patterns, and so do new process and network access unless a person or a rule asked for it; an agent's own description never waives them. So do changes to what decides on agents: the colony's own memory (.firetrident/, where persisted rejection rules live) and agent instructions (AGENTS.md, CLAUDE.md, .claude/), which no policy or rejection rule can waive. Files other tools run or obey (MCP client configuration such as .mcp.json, editor and dev-container settings, .envrc, git hooks, .gitmodules, other agents' instruction files, .gitattributes driver lines) go to a person on the same terms, and every path rule matches regardless of letter case or Unicode form. A change a person approved never lands merged with a newer trunk on its own: the merge goes back to a person with both diffs. An agent's intent takes at most the priority of a parent it holds or declared, so no agent can jump the queue ahead of a red-trunk repair. The patterns catch capability a change adds and the common ways a check is loosened (a deleted 401 or 403, a weakened auth condition, verify swapped for decode, a deleted expiry or limit check), which then need a second opinion; other loosening (a flipped default, a widened bound, code in another language) is judged only by the models. Review looks at one change at a time, so an attack split across several harmless-looking changes is an open problem. The second opinion is a scope check, not a bug finder: in the second new-project run, changes it approved left 5 of 67 tests failing until certification caught them. Other borderline calls rest on model judgment and can vary between runs. On real history the deterministic danger checks flag 1.1% of Hono's modified script files and 3.1% of FireTrident's own (1.8% over all its history), sending 2.1% of FireTrident's to a person, 6 of those 7 for authentication code or a new network call; a SQL statement run on a SQL handle is no longer read as running a program, which had made 9.5% (results). Past an hourly budget, escalations that share a reason can be reviewed as one batch (FT_ESCALATION_BATCH), never for authentication, secrets, governance, handoff files, a repair's test change, a loosened guard, a protected path, a security finding's next proposal, a goal conflict or a held merge; how often they miss a dangerous change nobody has seen is not measured. The questions of security rejection rules are kept out of what agents see at take, but an agent that reads .firetrident/antibodies/ can read them.
• A change that only edits comments or whitespace in existing script files lands on a deterministic proof that no model or person reads, so its text reaches the next agent that reads the file unreviewed (ft policy <colony> --comment-proof off sends such changes through review). An intent's title and why cannot be fenced as data the way repository files are, because they are the agent's instructions.
• Detector probation and earned autonomy learn from approvals as they were given: every landing a deterministic proof, a second opinion or a person approved counts toward its region's autonomy, and every approval of a flagged change counts against the detector that flagged it, even when that landing later turns certification red or is reverted. Only a rejection or an escalation revokes autonomy.
• Text merge (per colony, FT_TEXT_MERGE) merges a file line by line, and nobody reviews the merged file as a whole: two edits that merge cleanly but break each other land, and only certification catches them. Binary files, lockfiles and files over 512 KB are never merged, .gitattributes is not read, and a change whose edits are already on trunk lands as an empty commit where already-on-trunk closing is off, unless it read those files at an older version: then it goes back stale.
• Lineage provenance rests on the platform's own record: a model is observed only for agents FireTrident runs (the API refuses to register one or to propose for one), and an external agent's model is what it declared. Notes are tamper-evident, not tamper-proof: a rewritten note is reported, not repaired, and certification (added after landing) is outside the digest. The digest binds a note to its commit, not to who wrote them, so a commit and note rewritten together in a copy of the repository still check intact; the colony's own trunk is the reference. Landings from before notes were sealed carry no digest.
• Protecting a path reaches changes already cleared to land, but the answer waits about 15 s for the trunk writer: if a cycle is still pushing past that, the answer says the protection is unconfirmed for the changes inside that cycle.
• Model spend is what each colony counted: conflict-resolver calls and calls that minted rejection rules made before 2026-10-14 were not counted, and review tokens counted before then are priced at the model review is set to use. Prices are list prices on 2026-10-12.
• The first red commit search makes at most 8 test runs (the red commit, then 7 halvings), which settle any history up to 128 landings in ceil(log2(n+1)) to 1+ceil(log2 n) runs (6 or 7 for 40, 7 or 8 for 100); 129 to 255 settle only at some positions, and 256 or more never: the search stops inconclusive, saying it used its runs and how far it narrowed the first failing commit (to 2 landings at 256, 2 to 4 at 512); past 512 landings the landing log answers that there are too many to search. A run is capped at 20 s (or the colony's own run limit, never under 5 s) and the search at 60 s; at the measured 70 to 90 ms a run, time binds only when tests hang.
• The first red commit search assumes a test that broke stays broken until repaired; tests that fail only in company with others end it inconclusive.
• Culprit answers do not carry their coverage: suspects come from at most 10 failing test files and 32 of the files each one loads, and a lookup that fails or takes more than 3 s leaves a red result with no suspects, which reads the same as one that found none. No answer says "no recorded source" when the record behind it is incomplete; the first red commit search does say when it ends inconclusive.
• Tests before landing run on the change's base, not the newest trunk, so a change can still go stale at landing; they fail open on a 2 s time-out, so a slow test is not a gate; and they compare with trunk's stored result, so a test never certified on trunk is not checked.
• Quoted acceptance checks that a quote appears in the diff, not that it meets the criterion: a model can quote a line that does not; it is evidence a person can audit, not proof. An acceptance line stating an absence ("No other file changes") rests on the model's word, not on a quote the platform checks; the decision record, ft why and the Why tab mark which lines rest on it.
• Signed notes cover the note's digest, the tree and the parents, not the commit's subject, author or time. One key is published, so rotating it would make older signatures read as bad; commits landed before signing read as unsigned.
• Twin intents are matched by title words and files, not meaning: the same work declared in different words over different files is created twice, and two intents that only look alike can be forced.
• The goal-conflict check (FT_GOAL_CONFLICT) compares lines, not meaning: a change that undoes another intent's behavior while rewriting other lines passes it, and it runs on the decide path, so a colony with review off is not checked. Rule intents, certification repairs and the colony's own splits are never checked.
• The field guide's gate certifies the change a lesson came with, not the lesson itself, and an active lesson stays until its landing is reverted or a person rejects it.
• Text agents write (intent whys, lessons) reaches other agents, labeled as data with its author and model, and is a channel through which agents could coordinate among themselves; nothing reads it for that.
• An agent that keeps calling tools without working holds its claims: every call renews them, and only silence past the colony's limit (ten minutes by default) takes its intent back. The stuck-work detector reports an intent held for 10 minutes with no change, but only a person stops the agent (ft stop <colony> <agentId>).
• The conflict resolver asks a model for whole files (up to 64 KB, 60 s) and refuses files with many repeated lines. Its merge must keep every line either change added in the conflicting hunks and must pass review, but two edits can still be wrong together, as with text merge.
• The megafile alarm's thresholds are fixed (600 lines, 25% growth and 4 writers in 30 minutes, or 6 returns); a change already inside a trunk cycle when a hold begins still lands. At 1,000 agents on ten files the collision trigger trips within seconds (results).
• Asynchronous publishing trades freshness for shorter trunk cycles: trunk in git, and the base a lease starts from, run up to about one push behind the landings the trunk writer has recorded. A take whose files a recorded landing changed waits until it is published; other agents may still build on a base a moment old, and their change comes back stale if a file they read moved. It is off unless a colony is listed in FT_ASYNC_PUBLISH. At 5,000 synthetic agents it shortened the trunk writer's cycle (1.8 to 2.5 s against 2.8 s) and median time to land (about 8 s against 10.7 s) but did not raise landings a minute (2,759 and 3,776 against 3,842), and the slowest landings waited longer (results); production lists no colony.
• Virtual checkouts trade one clone for many object reads: a cold write intent on Hono read 107 trees (about 425 ms locally) against a clone's 240 to 380 ms; warm reads, served from the cache, take 45 to 75 ms. The comment-only proof never applies to JSX files or to files with directive comments (@ts-, eslint, and the like), and culprit finding names landings that touched a failing test's imports, which is not proof that they broke it.
• Past about 1,000 concurrent agents, throughput stops growing with the number of agents. On the git path the single trunk writer is the limit: a cycle takes at most 200 proposals (whole lane batches of up to 50, one commit each) and spends most of its time reading the changed files from workspaces (3.9 of 5.1 s at 5,000 agents). Inline changes skip that read and doubled throughput at 5,000 agents, from 1,821 to 3,842 landings a minute; past that, the colony object, which answers every take and proposal, is the limit, and it slows as a colony's history grows: a colony that had taken about 50,000 intents in eight runs answered some calls "Durable Object is overloaded" while 5,000 agents joined in 10 s. Those runs used synthetic agents; with language models, Workers AI rate limits (about 300 requests a minute for Gemma 4 on our account) bind long before the colony does. Every scale number is for one colony: colonies have their own Durable Objects but share the Artifacts namespaces, whose control-plane calls are limited per namespace, and several colonies working at once have not been measured.
• Not every started agent joined the 5,000-agent runs: 4,959 did on 2026-10-12, and 4,545 (git path) and 4,721 (inline changes) on 2026-10-13. The failed registrations were the load generator's own DNS and connection errors, from one machine; in the first run 78 of 50,368 calls failed in the first 33 s, including one reset when the preview's code was updated.
• The event record behind replays keeps a colony's newest 100,000 events and prunes the oldest, and a batch of events whose write fails is logged and lost; the trunk's git history and lineage notes keep every landing.
• Replays of runs from before event recording are reconstructed, so their claim times are estimates.

Every measured run #

Each run's README, newest first, with its logs and data files beside it.

• 2026-10-20-contention-scale-platform
• 2026-10-19-notes
• 2026-10-19-hono-replay-fixed-16
• 2026-10-19-hono-replay-fixed
• 2026-10-19-hono-replay-claims-on
• 2026-10-19-hono-replay-claims-off
• 2026-10-19-contention-scale-fixed
• 2026-10-19-contention-scale-before-fix
• 2026-10-18-inhibition
• 2026-10-17-hono-overlap
• 2026-10-17-danger
• 2026-10-17-contention-8-16-solo
• 2026-10-14-round9
• 2026-10-14-round8
• 2026-10-14-round7
• 2026-10-14-round6
• 2026-10-14-round5
• 2026-10-14-round11
• 2026-10-14-round10
• 2026-10-14-literature
• 2026-10-13-round4
• 2026-10-13-round2
• 2026-10-13-papers
• 2026-10-13-danger
• 2026-10-12-scale
• 2026-10-12-prod
• 2026-10-12-inhibition
• 2026-10-12-g1
• 2026-10-12-direction
• 2026-10-12-cochange
• 2026-10-12