Supply the ceiling
Mitchell Hashimoto ran an agent loop to optimize a renderer. It took frame times from 88ms to 1.5ms and cut allocations from 150,000 to ~500, over four hours and $350 of compute.1
His hand-written port of the renderer runs the same benchmark at 0.020ms with zero allocations in the update path. The agent's 60x win is 75x short of what the system allows.
The agent made a slow renderer fast, but reaching the hardware limit required a human who knew where that limit was.
The agent did its job
The objective was to minimize frame time, with the input structures, public API, and tests held fixed. Under those constraints the agent hill-climbed until the next improvement got hard, then stopped.
A 60x speedup with passing tests is a real result, not a hallucination. The agent measured honestly and reported what it found.
Why it stopped 75x early
An expert carries a prior on what the hardware allows: an update path that allocates nothing and runs in tens of microseconds. That prior is the difference between 1.5ms and 0.020ms.
The agent has no such prior. Without a known lower bound, any local optimum looks like the global one, and there is no signal left to tell it to keep going.
Hashimoto calls this agent psychosis and puts the fault on the human who accepts the number without a baseline. The agent's measurement was correct; the failure was trusting it as the answer.
The floor moved, the ceiling didn't
Agent output now clears the bar for anyone who has no baseline to check it. A 60x speedup with green tests looks like excellence to someone who can't see that 0.020ms was reachable.
That is the risk as agents get cheaper. "Impressive and wrong" becomes the common output, and catching it takes a baseline most readers don't have.
Supply the ceiling
To get the ceiling out of an optimizing agent, you have to supply it. Give the achievable bound as the target, or the agent ships a local optimum and calls it finished.
The $350 of compute was the cheap part. The expensive part is knowing the answer should have been 0.020ms, and that still comes from the human.