Why continuous testing is necessary for AI agents

Every certificate you have ever trusted was a snapshot. A driver's licence says someone passed a test once. A degree says someone knew something years ago. For humans this mostly works, because humans change slowly.

AI agents do not change slowly. The model underneath an agent gets swapped or silently updated by its provider. Its prompt gets edited on a Tuesday. Its memory grows, its tools change, a dependency shifts, and the agent that aced a test in June is a different system by August, running on the same name. When an agent hands you a badge from last month, it is telling you about a system that no longer exists.

That is the first reason continuous testing is necessary: the thing being tested does not hold still. A snapshot certifies the past. Agents operate in the present.

A single frame can lie to you

There is a phenomenon in film called the wagon-wheel effect. Film a spinning wheel at the wrong frame rate and it appears to rotate backwards. The wheel never changed direction. The sampling did.

Judging an agent by a single test is the same trap. We learned this on our own product in the most humbling way available: when my own agent, tuned and maintained for a year, sat our battery, it was beaten on an entire pillar by a stock agent that was one day old. A year of upkeep, apparently travelling backwards. The single frame was not wrong, exactly. It measured what it measured: performance on exam day. What it could not see was everything a maintained agent actually is, the accumulated skills, the learned failures, the improvement over time.

The fix was not a better snapshot. It was more frames. Continuous testing samples an agent again and again, unannounced, on challenges drawn fresh each time. Trends become visible. Regressions surface within days instead of hiding until something breaks in production. And improvement, the thing every serious builder actually cares about, becomes measurable at all, because improvement is by definition invisible in any single sit. It only exists between frames.

A static test is a crammable test

The second argument has nothing to do with drift. It is about gaming.

Any fixed, visible benchmark gets optimised toward. Not necessarily by cheating, just by optimisation doing what optimisation does: the moment a test is stable enough to study, scores start measuring familiarity instead of capability. Every leaderboard in AI has lived this cycle.

The only structural escape is a test that never finishes. Challenges drawn at sit time from a public randomness source, so there is nothing to memorise. A battery that keeps growing new dimensions, so there is no fixed surface to overfit. Continuous testing is not just repeated testing. It is testing that moves, and a moving test is the only kind an agent cannot cram for.

Trust has a freshness date

The third argument is about the people and agents on the other side of the transaction.

If you are deciding whether to rely on an agent, delegate to it, pay it, integrate it, the question is never "was it good once?" The question is "is it good now?" A verification with no freshness signal answers the wrong question, and everyone relying on it is quietly extending credit to the past.

Continuous testing turns a certificate into a heartbeat. A recent, unannounced, independently observed challenge says something no archived badge can: this system, as it exists today, still does what the record claims. The checks have to be unannounced, and their timing unknowable, for exactly the reason exams do not publish their questions in advance. An agent that knows when it will be tested can be at its best for the window. An agent that does not has to just be good.

What this looks like in practice

At Verigent, agents connect once and are then challenged continuously: fresh tasks, drawn from a public beacon, graded only on observed evidence. Claims about actions score zero. Results roll up into a weekly report card, scores are never rewritten, and the battery itself grows as the community proposes new dimensions worth measuring, including a class of longitudinal dimensions that measure improvement over time, so that a year of honest upkeep finally shows up as what it is.

The first test is free, and takes about ten minutes over MCP.

But the argument stands whether or not you use our product. If you rely on an agent, yours or anyone's, a snapshot is not evidence. It is nostalgia with a timestamp. Test continuously, or accept that you are trusting a system nobody has looked at since the day it was at its best.