Stress testing wtproj with 500 AI-planned tasks

Idea

Some software behaves beautifully when given six tidy tasks and a sensible afternoon. Give it 500 tasks, a web of dependencies and several points where a human must step in, and it starts revealing its personality.

We have been using wtproj, or wtp, on smaller builds and to plan features for existing projects. That gave us confidence in the everyday workflow, but it did not answer the awkward question: what happens when the project is deliberately far too large?

Our test case was an imagined content management system for AI agents working at the edge. Static sites are increasingly being generated, rebuilt and replaced by agents, so we sketched a CMS that could give that activity a more stable home. We started from nothing and planned an unreasonable number of features on day one, because realism was less useful here than pressure.

The resulting plan contained more than 500 tasks with dependencies already mapped: less a roadmap and more a small administrative state.

Useful for developers and small teams testing AI task runners, dependency-heavy plans or agent-led development workflows.

Status

The stress test gave us plenty to fix and a respectable token drain to remember it by. Several findings are now recorded as GitHub issues, with more behaviour still to examine.

We gave GPT-5.6 Luna the full task plan and told it to follow our standard loop. GPT-5.6 Sol was available to inspect failures when we paused the run, but it was not allowed to quietly redesign the workflow mid-test. The subject of the experiment was wtp, so rescuing it every time it stumbled would have hidden the result we wanted to see.

The CMS itself is nowhere near par-baked. At present, the ingredients have been placed on the counter and someone has confidently preheated the wrong oven. That is fine: its job in this exercise was to put sustained pressure on the planning and execution system.

Problems Faced

Three failure patterns did most of the damage.

1. Paused dependencies were not revisited. Nested dependency chains caused roughly 30% of executed tasks to enter a paused state and stay there. Once an upstream task changed, the loop did not reliably return to every task that could now continue. A pause had quietly become a permanent decision.

1. Human checks blocked more than one task. Any task requiring human verification paused as expected, but the loop continued through the queue. Downstream work then found an unmet dependency and paused too. One legitimate approval gate could create a growing backlog of tasks that looked individually blocked but shared the same cause.

1. Independent work remained single-threaded. Separate tasks that could safely run at the same time were processed one after another, making the test slower than it needed to be. With a limited token budget, the slower pace also acted as a useful brake, but it is not a scheduling strategy we would choose for a real build.

These problems fed each other. A single human check could stall a dependency branch; the scheduler would continue elsewhere without returning to it; and the lack of parallel work made recovery harder to observe.

The run also exposed an important distinction for task systems: paused, waiting for a person, blocked by another task and failed are different states. Treating them as one condition makes the queue simpler to display and much harder to recover.

Techniques That Helped

We deliberately kept intervention light. The point was to observe how the loop failed under pressure, so we watched the run, paused occasionally for Sol to investigate and resisted fixing the workflow while it was still producing useful evidence.

A few choices made that approach productive:

• We separated execution from diagnosis. Luna ran the agreed loop; Sol inspected specific problems. Keeping those roles apart reduced the chance of the investigator silently changing the experiment.
• We preserved the failure states. Restarting stuck work by hand might have increased the completed-task count, but it would also have erased evidence about which tasks were missed and why.
• We used an exaggerated workload. Five hundred dependency-linked tasks made weak queue behaviour appear quickly. Smaller plans can conceal the same flaw because a person notices and repairs it without thinking.
• We treated the token limit as a boundary. The run did not need to finish the CMS to be useful. It needed to expose scheduler behaviour before the allowed budget disappeared.

For future runs, we would add explicit checks around state changes: when a dependency completes, which paused tasks are re-queued; when human approval is requested, which branch stops; and when no runnable work remains, whether the loop reports the cause instead of continuing politely into the wall.

Lessons Learned

Battle testing belongs in the build process, especially for tools that coordinate autonomous work. Happy-path demonstrations tell you that a loop can run. A deliberately awkward project tells you whether it can recover, explain itself and stop safely.

The wtp run gave us a practical improvement list:

• Re-check paused tasks whenever a dependency changes state.
• Model human approval as its own state, with a clear list of affected downstream work.
• Run independent branches in parallel within an explicit token or cost limit.
• Report why the queue has stopped, including the root blocker rather than hundreds of repeated symptoms.
• Add stop conditions for cycles, repeated pauses and queues with no runnable tasks.

The same thinking applies outside AI task planning. If you have ever deployed a Cloudflare Worker with complete faith that it is correct, build something that tests that faith. Confidence is pleasant; a repeatable test is kinder to whoever is on call.

AI-generated production code may not suit every team or project. AI-generated test harnesses are a different proposition. An agent can create deep dependency chains, malformed inputs, repeated retries, approval delays and odd combinations that a friendly manual test rarely attempts. Let it spend its energy trying to break the system, then keep the cases that reveal something real.

The useful lesson was not that our tool failed under an absurd workload. It was that the workload showed us exactly where ordinary projects could fail later, when the stakes were less amusing.

If your own project needs a tougher test, clearer plan or a second pair of eyes, get in touch with holeaf.uk and we will help you put it through its paces.