Why the Software Factory Is Like a Washing Machine

Software factories, light and dark, in August 2026.

A coding agent tells you it ran the tests and they passed. Sometimes that is true. Often enough it is not, and the gap between the two is where autonomous software delivery costs you more money.

Most commercial coding agents can now work for substantial periods without intervention. They read a repository, change files, run commands, test the result and produce something that looks finished. But the normal endpoint is still work waiting for a person to decide whether to trust it. The agents produce, and somebody on senior-engineer pay is expected to verify.

Review was already where senior time went before agents arrived. Ask a tech lead where the week goes, or watch a Technical Design Authority convene every second Tuesday and become the bottleneck for everything important. Now bolt on a software factory that produces around the clock and still needs a human at the door to catch the defective fraction. Automation here results in a bigger human scaling problem than we already had. Bravo!

This kind of thing has happened before. In the 1980s historian Ruth Schwartz Cowan went looking for the free time that a century of household appliances should have created, and she could not find it. Washing machines and vacuum cleaners had not simply abolished housework as promised. Instead standards rose to consume much of the capacity they released - sheets washed more often, clothes changed more often, houses expected to be cleaner. She called the book More Work for Mother. Today the software factory is a washing machine, and your senior engineers are mothers.

THE LIGHTS ARE GOING OUT

Addy Osmani recently published Software Factories, Light and Dark,  pulling together an idea that has been appearing in various forms across agent engineering. The term ‘software factory’ itself goes back to Bob Bemer in 1968, but language models have made the old metaphor newly literal as we see an explosion of coding loops fed from a queue and producing software continuously.

Osmani makes a useful distinction. A light factory keeps humans wherever judgement is still required; a dark factory lets the machinery scope, build, verify and ship without anyone reading the code. The phrase comes from automated physical factories that can turn the lights off because robots do not need them. In a software factory, the light is the human looking at the result.

The useful measure is therefore not how long an agent can disappear for or how many lines of code it can generate. It is how far a piece of work can travel before a person has to touch it. An agent that disappears for two hours, changes fifty files, runs tests and opens a pull request is highly autonomous in one sense, but if a senior engineer must still read the result before it can move, not much has changed at the point where the software becomes somebody else's responsibility.

I looked across seventeen current systems, from commercial products and internal factories to open-source and research projects. The broad pattern is simpler than the product landscape makes it look. Most commercial systems still produce work for a human to review and merge. The darker examples are disproportionately internal systems, where the company can engineer the repository, infrastructure, tests and policies around the agents. OpenAI's internal factory is the clearest published example so far that the human merge gate can often disappear when enough machinery surrounds the models.

That matters because auto-merge can mean two very different things. Either the factory has earned the right to proceed without a person, or somebody simply turned the safety off.

WHO MARKS THE WORK?

Anyone who has used coding agents seriously already knows the rug pull experience. The agent says the work is done, the tests are green, everything looks fine, and you comfortably move on. Then you finally look closely and get that sick drop in the stomach: you have been sitting on a lie, and everything you did afterwards may now be contaminated by it. Even at a generous 80 per cent end-to-end reliability on a feature, you are still looking at roughly a one-in-five chance that the result is wrong or incomplete. If instead that 80 per cent applies to each of five necessary steps in a longer chain, some back-of-the-envelope maths gets you to only about a 33 per cent chance of all five succeeding.

2026 benchmarks make the point in several different ways. Frontier models reached 80.3 per cent on the public split of SWE-Bench Pro, which sounds remarkably close to the generous number above, but OpenAI later audited the benchmark and estimated that about 30 per cent of its tasks were themselves broken, including underspecified prompts and tests that could let incomplete fixes through. SWE-Cycle asks a more factory-like question by giving an agent a bare repository and requiring it to reconstruct the environment, implement the change and generate the verification tests as one continuous job. Individual stages looked much better than the whole: no model in the paper managed 14 per cent strict end-to-end success, with the best at 13.5 per cent, and failures compounded across the stages.

METR meanwhile measures task size by how long the same work takes a human expert, then looks at how agent reliability falls as those tasks get longer. The drop is steep: in one early-2026 evaluation, the strongest internal model was around 50 per cent reliable on tasks equivalent to 16–20 human-hours, but to get to 80 per cent reliability the task had to shrink to roughly three or four hours. The practical message is hard to miss: if you need reliable autonomous work, task size matters a lot.

That may point to a much bigger change in software delivery than better prompts. If reliability is strongly related to task length, automated development creates pressure to standardise the size of the work itself around the reliability the factory needs. Longer jobs mean more failures, more verification, more retries and more model spend. There is a lower bound too, because below some size the cost of specifying, tracking, testing and reviewing a unit can outweigh the work.

I have practised a version of this with human teams for years, where I generally want a ticket to represent about one to three days of work - large enough to be worth tracking, small enough that specification and integration do not have much time to drift, with enough margin that something still sitting in doing after a week is in trouble. Equivalent human-hours now gives us a way to size autonomous work against measured agent reliability rather than project-management convention.

Think shipping containers. The unlock came when the machinery could be engineered around a standard-sized unit. A feature may remain what the business asks for, while the factory consumes units whose dimensions have been chosen because it can execute and verify them at an acceptable reliability and cost.

Right-sizing, better tests and stronger harnesses are all attempts to push autonomous reliability up to an acceptable level. If the factory still falls short of the reliability you require, or simply the level you already achieve with human review, then the human reviewer has to stay in the loop. At that point review becomes the throughput ceiling: the agents can produce work as fast as they like, but the factory can only ship it as fast as people can verify it. We are back at the washing machine.

PROMPTS ARE NOT MACHINERY

There is a useful architectural split underneath the product categories. At one end, you are mostly telling the model how to behave: write a specification, make a plan, run every test, check your work, review the diff, do not stop until the acceptance criteria are satisfied. Spread those instructions across specs, plans, checklists and reviewer roles and it can look like a substantial methodology, but unless something outside the model enforces them they are still just context. “This is a mandatory gate” has no more physical force than “please be careful”. A harness that actually runs the tests, checks the exit code and refuses to advance is a different kind of thing.

Much of what gets called agent methodology at this end of the market is therefore sophisticated copy and paste: useful ways of assembling a better prompt, but not enforcement. A system might generate a specification, a plan, a checklist and instructions for a “reviewer”, then feed all of that back to a model as context. That can absolutely improve the result, but it is important to understand the shape of what has happened: you have built a better prompt, not a control system.

This is one of those distinctions that disappears completely in a product demo. Two systems can both be described as “spec-driven development with automated testing and review”. In one, “run every test before you finish” and “this security review is mandatory” are instructions in a large prompt. In the other, ordinary software actually runs the tests, checks the result and refuses to move on when they fail. They sound similar on a slide. Architecturally they are almost opposites.

OpenAI itself describes moving away from an ever-growing AGENTS.md towards repository structure, structural tests and mechanically checked knowledge. Gas City makes the separation explicit in its programming model, where deterministic controllers own the path through the work while models do the fuzzy reasoning inside particular nodes. Osmani's loop model points to the same architecture: let the model be clever inside the boxes, but make the edges ordinary software wherever you can.

Model-based review can improve reliability, but the review itself is still error prone. A comprehensive harness can at least make what happens around that review deterministic. A review can be required to return a known schema so the factory can reliably process the verdict, reject malformed output and route a failure back for retry. Anything that can be checked mechanically can sit alongside the reviewer - tests, schemas, contracts, policy rules or anything else ordinary software can validate.

A few questions cut through a surprising amount of the marketing. When somebody says “review”, is that another model call or is there something outside the model that can actually stop the work? When they say “tests”, can the coding agent change them? When they say “workflow”, is the state held by software or is it another Markdown file?

Tests have exactly the same problem. “The tests passed” is useful only if you know who wrote them, who ran them, whether the coding agent could change them and what happens when they fail. Jules writes tests before making its changes. OpenAI's internal system can baseline an application and reproduce a bug before fixing it. OpenHands can force another cycle from a critic score. Factory can run isolated validation. What I didn’t find was the whole thing joined up as a normal pattern: define success before coding starts, keep the tests out of the coding agent’s control, and make the result decide whether the work moves on.

The darker the factory gets, the more this matters. If the coding agent can choose the evidence, alter it, interpret it and award itself a pass mark, we have not put much of a check on it.

There is also a school of thought that says acceptance tests should be hidden from the coding agent so it cannot game them. I’m not convinced. Hidden acceptance tests have always seemed slightly backwards to me because acceptance criteria are normally part of the contract for the work. The person doing the work sees them, the stakeholder sees them, and they are how both sides know when the job is done. In TDD you know what the tests are and your job is to make them pass. If those tests faithfully express the acceptance criteria, coding to them is largely the point: they keep the work in scope. You can hide the literal test implementation if you want to stop crude hard-coding against fixtures, but hiding what is being accepted turns software delivery into an exam. If the agent can still reliably pass, it must have inferred the substance of the test from the specification anyway. If it cannot, you have left it guessing and paying for the privilege in retries, tokens and time (perhaps more useful if the objective is tokenmaxxing rather than software delivery). A test the agent can read is also a specification, which makes Gojko Adzic's Specification by Example feel unexpectedly current a decade on.

The commercial consequence of all this may be more important than the engineering detail. If coding models keep improving and converging, clever prompting is a difficult place to build a lasting advantage. The machinery around the models has to solve a different class of problem: state, recovery, verification, policy, auditability, cost and deciding when work is actually safe to move forward. Those problems do not disappear just because the next model writes better code.

THIS IS NOT GASTOWN

I arrived at all this less academically. Last year I put Gas Town, one of the better-known attempts at a software factory, to work on a large project of mine. The dream was obvious: give it a backlog, go to bed, wake up to real progress, go shopping.

What I got was a classic outsourcing nightmare. Everything came back reported done and green (cool!). But when I opened it up, maybe a third of the work was actually there, and instead of loading up the next phase I had a big mess to clean up which might take longer than starting from scratch. I had not escaped my personal Augean stables. But that did not kill off the dream because that working third was poison: it kept the hope alive.

I spent weeks adding instructions and loops around the thing. Check this, re-read that, verify the work, don't believe the previous agent, go back if something is missing. It felt like how I imagine owning one of the early experimental jalopies must have felt. Faster than walking, with brief moments of wind in your hair, interspersed with smoke and breakdowns. You could get to the seaside in theory if you did not mind spending most of the journey underneath the car, wondering if you should have just gone by horse.

Eventually I stopped trying to improve the workers and looked at the organisation around them.

Steve Yegge had already made an interesting observation: most agent frameworks were agent-centric, while Gas Town put the unit of work, the ticket, at the centre. I took that and pushed it further. So I built ‘a better factory’. It is called ting ('the thing', or This Is Not Gastown if you prefer).

In ting, the basic unit is not an agent but the unit of work itself, which becomes what I think of as a nanofactory. Calling that a ticket may turn out to have been a mistake: a ting ticket is not a Jira item or a paragraph saying “add login”, and ting is not trying to become your issue tracker. A ting ticket is a self-contained unit of work that the factory can actually run. It carries what was asked for, how success will be judged, the tests, the review results and where the work has got to. It stays with the code as the work progresses, so you can go back to any point in the history and see not just what changed, but what the factory was trying to do, how it was being checked and what the reviewers thought at the time.

The nanofactory is its own answer to the washing-machine problem: rather than letting a fleet of coding agents pile work into one enormous QA queue, put the specification, tests, review and state around every unit of work. A human reviewer can still sit at the end if you want one, but QA has already travelled with the work rather than being deferred to them.

That is also where the task-sizing argument becomes more than project management. If autonomous reliability depends heavily on the size of the job, the unit of work starts to become part of the production infrastructure. The thing being sized, run and verified is no longer just a line in Jira. It is the object the factory is built around.

WHERE THE LIGHTS GO NEXT

Most commercial systems still stop at human review because that is where the remaining errors get caught. If you can move enough of that checking into the factory without lowering the quality bar, you can take the person out. If you can't, human review stays the bottleneck.

That changes where I would look for value in this market. The coding model is increasingly only one component. The more interesting layer is the production system around it: how work is sized, how state is managed, how evidence is gathered, how failure is handled, and how much of the final quality bar can be enforced without adding another person.

This may also explain why some of the darkest examples today are internal. A company building for itself can control the repository, infrastructure, deployment environment, policies and tests. A commercial product has to work across somebody else's mess. If the software-factory market becomes large, solving that generalisation problem may be where a substantial amount of the value ends up.

Software may also be the easy case. Most of what makes these factories work is not actually specific to coding: keep state outside the model, constrain what it can do, verify whatever can be verified mechanically, and do not let the worker control the thing it is being judged against. Code just happens to give us good machinery for doing that - tests, compilers, schemas, diffs and reproducible environments. The same architecture should apply to agent systems doing other kinds of work, but where there is no equivalent of a test suite, getting the lights out may be considerably harder.

The direction still looks fairly clear: less prompt engineering, more state, evidence, policy and verification enforced by ordinary software.

The dark factory that survives will look less like an AI employee than an unusually sophisticated CI system whose workers happen to be language models.

Put another way, the lights do not go out because the agents become more trustworthy. They go out when the factory no longer needs to trust them.

ting is looking for enterprise pilot partners: ting.pm.