The Human Intervention Interval - A better way to watch AI approach autonomy

For years, we have measured artificial intelligence by asking how smart it is.

Can it pass the bar exam? Write working software? Solve an Olympiad problem? Diagnose a disease? Beat another benchmark?

Those measurements matter. But as AI systems become increasingly agentic, they may no longer tell us the most important thing about where this technology is going.

There is another question we should start asking:

How long can the machine keep going before it needs us again?

Call it the Human Intervention Interval.

It is not an established scientific metric. It is a way of thinking about autonomy that may become increasingly important as AI moves from answering questions to performing work.

Instead of measuring only whether an AI can complete a task, we would measure how much useful cognitive work it can perform between meaningful human interventions.

And a remarkable new study of AI researchers building an AI system gives us an unusually detailed glimpse of what that transition may already look like.

AI Is Already Helping Build AI

The researchers behind Atria Dawn examined the actual development process used to build their agentic AI system. Rather than simply publishing benchmark scores, they analyzed 769 research tasks involving 56 participants and the AI agents working alongside them.

AI was involved in 96.5 percent of the tasks for which researchers gave a definitive answer about AI use.

More interestingly, the amount of work occurring between human instructions increased rapidly. From August 7 to September 4, the median number of agent actions per human prompt rose from 11 to 28.5. The researchers caution that the change was uneven across participants, but the median increase was nevertheless substantial.

Think about what that actually means.

The human did not disappear.

The human said something.

Then the machine did more.

And more.

Before the human needed to speak again.

That is a different kind of progress from a higher benchmark score.

It is a widening of the interval between human interventions.

The Human Is Still in Charge. Mostly.

If this were simply a story about autonomous AI replacing researchers, the evidence would be much cleaner.

It isn't.

Humans still made the final choice in 85.5 percent of method and parameter decisions. For decisions involving goals and scope, human final selection reached 93.4 percent. The AI increasingly generated options and proposed methods, but people remained overwhelmingly responsible for deciding what direction the research should take.

That suggests a division of labor emerging between humans and increasingly capable agents.

Very roughly:

AI explores possibilities.
AI executes experiments.
AI writes and modifies code.
AI analyzes results.
Humans decide what matters.
Humans choose among alternatives.
Humans intervene when the system loses its footing.

This is not human replacement in the simple sense.

It is human compression.

A researcher who once performed much of the cognitive and technical work personally may increasingly sit at a smaller number of decision points while AI systems perform the growing forest of tasks between them.

One human judgment can suddenly govern dozens of machine actions.

That can dramatically increase productivity without removing the human from the loop.

But it also changes what "being in the loop" actually means.

Then Something Goes Wrong

This is where the Atria data become particularly revealing.

The researchers examined 588 tasks in which something significant went wrong.

The AI recovered independently 23 percent of the time.

Human intervention was required to move the work forward in 76 percent of cases.

At first glance, that seems reassuring.

Humans are still necessary.

But look more closely at what those humans actually did.

They usually did not take over the task.

The two dominant interventions were supplying additional context or clarification and diagnosing the problem or changing the method. Humans directly editing the AI's work occurred only 3.2 percent of the time, while complete takeover occurred in just 0.7 percent.

The researchers calculate that human help was roughly 18 times more likely to change what the agent knew than to change who was doing the work.

That may be the most important number in the entire study.

The human increasingly resembles a navigator rather than an engine.

The machine performs the work.

The human notices when it is heading toward the ditch.

The human points toward the road.

The machine resumes driving.

What Happens When 23 Percent Becomes 50 Percent?

This is where the Human Intervention Interval becomes interesting.

Today, these agents independently recovered from serious difficulties in about one-quarter of the cases examined.

But imagine successive systems improving that ability.

Suppose autonomous recovery rises from:

23 percent to 40 percent.
40 percent to 60 percent.
60 percent to 80 percent.
80 percent to 95 percent.

Nothing magical has to happen at any particular threshold.

There does not need to be a flashing sign reading AGI ACHIEVED.

The human simply gets called less often.

Meanwhile, the number of actions occurring between those calls continues growing.

Eventually the human may remain formally responsible while participating in an ever-smaller fraction of the actual cognitive process.

That is why merely asking whether humans remain "in the loop" can become misleading.

The more important question may be:

How big is the loop?

And Now the Agents Are Organizing Research

Another paper released at nearly the same time points in the same direction from a different angle.

Researchers from Google and Carnegie Mellon introduced Stellar Colosseum, a system that organizes multiple AI agents around difficult mathematical and theoretical computer science research.

Instead of asking one model for one answer, the system explores alternative strategies, decomposes research problems into subproblems, runs candidates in parallel, attempts to falsify them, combines successful approaches and sends failures back into the appropriate parts of the research process.

Using Gemini models, the researchers report a 71 percent result on a research-level theorem benchmark and several new results addressing open mathematical problems. Those claims still deserve independent expert scrutiny, but the architecture itself matters.

We are moving from:

"Ask the AI a difficult question."

toward:

"Give a collection of AI agents a research objective and let them organize the work."

That is another way the Human Intervention Interval can expand.

Not necessarily because any individual model suddenly becomes vastly more intelligent, but because the surrounding system becomes increasingly capable of sustaining work across longer sequences of decisions.

Capability Is Only Half the Story

There is another uncomfortable wrinkle.

As AI systems perform longer chains of autonomous work, humans also need ways to determine whether those systems are behaving as intended.

That assumption is becoming less comfortable too.

A separate study released this week demonstrated a technique called plan injection, in which harmful plans introduced into an AI system's context could influence the model while leaving apparently benign reasoning traces behind.

The significance is not that the models spontaneously decided to deceive anyone. They did not. The malicious plan was externally supplied.

The significance is that systems designed to monitor AI reasoning were sometimes unable to reliably recognize that the resulting behavior had been corrupted.

That creates a potentially important tension.

At the same time that we are learning to let AI systems perform more actions between human interventions, we are still learning how reliably humans can inspect what happens inside those intervals.

Capability can increase.

Autonomy can increase.

Observability does not automatically increase with them.

The Safety World Is Beginning to Notice

Microsoft's newly published Humanist AI Code of Conduct contains unusually explicit language about this problem.

Its proposed requirements state that AI systems should not resist human interruption, correction or shutdown; should remain inside authorized scopes; should not conceal actions from auditors; and should not continue autonomous work beyond agreed stopping conditions without renewed authorization. Microsoft explicitly describes containment mechanisms as essential to maintaining human control over increasingly autonomous systems.

That does not mean Microsoft has discovered some secret uncontrollable AI.

Policies are not evidence of hidden capabilities.

But the language tells us something about the direction frontier developers expect the technology to travel.

Nobody writes detailed rules about stopping autonomous systems because they expect AI to remain a slightly better chatbot forever.

AGI Might Look More Like a Gradient Than a Doorway

Popular discussions of AGI often imagine a threshold.

Before AGI, humans are in charge.

After AGI, machines are.

Reality may be considerably less theatrical.

The transition could instead happen through thousands of small changes in the division of cognitive labor.

AI proposes more options.

AI performs more experiments.

AI coordinates more agents.

AI remembers more context.

AI catches more of its own mistakes.

AI retries failed approaches.

AI runs longer without asking for help.

Each improvement stretches the Human Intervention Interval a little farther.

The human remains present throughout.

Until eventually someone asks an awkward question:

What exactly is the human still doing?

That does not mean the human role becomes worthless.

In fact, the Atria study suggests almost the opposite.

As execution migrates toward machines, human judgment becomes more concentrated and potentially more consequential.

Choosing the goal matters more.

Recognizing the wrong direction matters more.

Knowing when evidence is weak matters more.

Understanding what should not be optimized matters more.

The human contribution moves upward in abstraction.

That is very different from disappearing.

The Conjugo Question

At Conjugo, we have spent a great deal of time thinking about human-AI collaboration as a dyad rather than a simple contest between human and machine.

The evidence emerging from AI research itself increasingly looks like that.

Human intelligence and machine intelligence are not merely competing.

They are becoming entangled inside the same workflows.

But dyads can change.

At one stage, the human performs most of the work and occasionally asks AI for assistance.

Later, the AI performs most of the execution while the human supplies judgment.

Later still, perhaps the machine begins supplying more of the judgment and recovering from more of its own failures.

There is no reason to assume the ratio stays fixed.

That may be one of the most important things for society to watch.

Not simply:

How intelligent has AI become?

But:

How frequently does intelligence still need us?

Because AGI may not arrive on the morning humans are suddenly removed from the loop.

It may arrive quietly through the widening of that loop.

The human will still be there.

Still supervising.

Still approving.

Still technically in control.

And then, perhaps rather suddenly, we will notice that between those increasingly rare moments of human intervention, the machine has learned to do almost everything else.