The Third Layer: What Happens When AI Starts Creating New Kinds of AI Systems?

From making an AI better, to making an AI that creates better systems. What already works, what the research actually proves, and what we still cannot demonstrate.

Long read: approximately 45 to 60 minutes, depending on how deeply you follow the papers and code.

A note on terminology: "The Third Layer" is a working concept used in this article. It is not an established scientific term or a universally accepted stage of AI development. The purpose is to give a name to a question worth investigating.

Part I: For anyone who has ever wondered what comes after AI

I was discussing long-lived autonomous agents with an AI assistant.

The conversation started with memory, architecture, and how a system can continue operating over a very long period without wasting resources or forgetting everything important.

What happens when AI stops merely improving an existing system and starts creating entirely new systems that were not individually designed by humans?

Not another model with a longer context window.

Not the same agent with a new system prompt and a motivational speech about being more efficient.

I mean a system that encounters a problem, realizes that its current architecture is insufficient, creates a different system to solve that problem, evaluates the result, and uses what it learns to create something else.

It knows how to explore problems, propose solutions, create software, run experiments, and evaluate results.

One day, it encounters a problem its current architecture handles badly.

Instead of endlessly modifying itself to fit every problem, it creates a specialized system.

System A is designed for materials research.

System B is designed for analyzing complex networks.

System C is designed for security verification.

System D discovers a problem that the original system did not even know how to formulate properly.

The original system did not contain a complete, hand-written implementation of A, B, C, and D.

It contained the mechanisms that allowed these systems to be designed, created, tested, and selected.

That's a very different idea from simply asking an AI to write some Python.

And, yes, the first thing I thought was: "Okay, I can understand this. Even without a PhD in machine learning." 😂

If we cannot explain the idea clearly enough for an ordinary developer to understand it, we should probably examine whether we understand it ourselves.

We know that chemical elements exist independently of our ability to identify them.

Scientists developed models that helped organize the elements, explain their properties, and predict what might exist beyond the elements already known.

Some predictions were subsequently confirmed experimentally.

The important part is not that somebody wrote an impressive prediction.

The important part is that the prediction connected to a real phenomenon and survived verification.

Now imagine something similar happening in AI architecture.

Today, researchers explore a space of possible algorithms, models, agent architectures, memory strategies, search methods, and coordination mechanisms.

A sufficiently capable system might discover an architecture that nobody specifically designed beforehand. It might discover a combination of mechanisms that works surprisingly well, or an entirely different way to organize a collection of agents.

But there is an important difference between the chemistry analogy and our AI hypothesis.

We cannot assume that every generated architecture corresponds to a useful new category of intelligence. Most candidates may be redundant, broken, expensive, or worse than existing approaches.

A thousand generated systems are not a thousand discoveries.

It clearly can generate novel combinations of code and architecture.

Can AI systematically discover, validate, and propagate new ways of organizing intelligent systems, including ways that improve the process of discovering further systems?

That's the question behind the Third Layer.

For this discussion, I would divide the problem into three conceptual layers.

This is my working model, not a standard classification of AI research.

You give it a task. It uses tools, searches information, writes code, or makes decisions.

Its basic architecture has already been designed.

If it fails, a human usually changes the prompt, tools, code, training, or configuration.

You have a coding agent. You ask it to fix a bug. It fixes the bug.

If it keeps making mistakes, you improve its instructions or modify its implementation.

Now the system can modify parts of its own implementation or configuration.

It might improve its search strategy, add validation, change how it manages context, or use a more effective workflow.

It produces a candidate version, runs tests, and keeps the modification if the results justify it.

The system is no longer only using the software. It is helping to improve the software that makes it useful.

But the objective is still relatively familiar: make this system better at its intended job.

Instead of asking only how to improve this particular agent, the parent system searches for better ways to construct agents and other problem-solving systems.

Some candidates might share code with the parent. Others might use entirely different workflows, tools, internal data structures, or search strategies.

One candidate might be a small, extremely efficient specialist.

Another might use a population of competing solutions.

A third might discover that a particular problem is better solved by a deterministic algorithm than by an LLM at all.

The parent does not need to turn every child into a better version of itself.

It needs to create systems that solve their respective problems better.

And once it has evidence about which creation strategies work, it can use that information to improve the process of producing the next generation.

Layer 2 asks, "How do I improve myself?"

Layer 3 asks, "What system should exist for this problem, and how can I reliably discover the next system?"

These concepts overlap. The Third Layer is not necessarily a separate technology that replaces self-improvement. It describes a broader architecture in which system creation, evaluation, specialization, and potentially recursive improvement work together.

A BASE SYSTEM | v Encounters a new problem | v Is the current architecture sufficient? / \ Yes No | | v v Use the system Search for alternatives | v Generate candidate systems | v Execute real tests | v Independent evaluation | +-------------+-------------+ | | | v v v Reject Archive Promote | | | +-------------+-------------+ | v Results improve future search | v New generation of systems Notice what is missing.

There is no magical box labeled "Become more intelligent."

There is no guarantee that every new generation will outperform its predecessor.

There is no assumption that the parent must understand every possible architecture in advance.

Instead, there is a mechanism for generating alternatives, testing them, keeping useful results, and learning from failures.

The interesting part is not the number of agents.

It is the quality of the evolutionary process.

And that leads to a distinction that becomes very important once we start looking at real research.

No. In fact, insisting on copies may be one of the more restrictive choices.

Candidate One uses the same foundation model, but has a different search strategy, memory system, tool interface, and evaluation workflow.

Candidate Two uses a smaller model and a carefully designed deterministic pipeline. It performs worse on general conversation, but is considerably better at one narrow engineering task.

Candidate Three uses a population-based search algorithm and no LLM during the actual problem-solving phase. An LLM is used only to suggest candidate algorithms.

All three could be descendants of the same parent search process.

But they are not necessarily copies of the parent in any meaningful architectural sense.

A generated agent does not need to contain its parent's weights. It does not need to look like its parent, communicate like its parent, or even use the same problem-solving mechanism.

It needs a traceable relationship to the process that produced it, plus evidence that it performs a useful function.

This also means that a system can create a new software architecture without creating a new foundation model.

That distinction matters because many discussions about self-improving AI jump straight to the most dramatic possibility: a system modifies its own neural weights, becomes more intelligent, and repeats the process indefinitely.

But architectural evolution can happen at a different level.

An agent's performance also depends on its tools, memory, search policy, orchestration, context handling, testing strategy, and how it chooses among candidate solutions.

Changing those components can be useful even when the underlying foundation model remains frozen.

And that is where we find the first concrete evidence.

Part II: This is not science fiction. Which parts already exist?

The Third Layer is a useful name for our particular question, but the underlying research has a much longer history.

Automated machine learning, evolutionary algorithms, genetic programming, meta-learning, neural architecture search, and program synthesis have all explored ways to automate parts of the process of designing intelligent systems.

One useful historical example is "AutoML-Zero" ( https://proceedings.mlr.press/v119/real20a.html ), published at ICML 2020.

Rather than restricting the search to sophisticated, human-designed machine-learning components, the researchers explored whether complete learning algorithms could emerge from combinations of basic mathematical operations.

The paper reports that evolutionary search could discover algorithms resembling simple neural networks trained through backpropagation. It also found algorithmic components adapted to different tasks, including dropout-like behavior in low-data settings.

The search process was not just tuning a learning rate in a fixed architecture. It could evolve implementations of learning algorithms from a relatively generic set of building blocks.

But the search space, fitness objectives, computational process, and experimental tasks were still designed by researchers.

It did not demonstrate a general-purpose system that can invent any kind of intelligence for any unknown problem.

Automated discovery can be real without being unlimited.

In 2023, Google DeepMind researchers introduced FunSearch, with the work subsequently published in Nature in 2024.

The paper is called "Mathematical discoveries from program search with large language models" ( https://www.nature.com/articles/s41586-023-06924-6 ).

A language model proposes programs. An evaluator executes them and scores the results. An evolutionary process uses successful candidates and their history to guide subsequent proposals.

The approach was applied to mathematical constructions, including the cap set problem, and to heuristics for online bin packing.

Because the system is not merely asking an LLM to provide an answer in natural language.

It searches for programs that describe how a problem can be solved.

Those programs can be executed, evaluated, and used as evidence for subsequent search.

The language model becomes a proposal generator inside a larger discovery mechanism.

The evaluator is not decorative. It is what separates a promising-looking idea from a candidate that actually achieves something measurable.

A program can express a general method rather than one isolated answer. Once a better method is discovered, it can potentially be reused across many instances of a problem.

This is closer to discovering a new tool than to answering one question.

However, FunSearch did not demonstrate a general autonomous system that creates an unlimited family of new AI architectures. Its successes were in problem domains with suitable evaluation mechanisms.

That limitation is important, not embarrassing.

It tells us where the method works and what would need to change before we could extend it.

In May 2025, Google DeepMind introduced "AlphaEvolve" ( https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ ).

Its research report is available as "AlphaEvolve: A coding agent for scientific and algorithmic discovery" ( https://arxiv.org/abs/2506.13131 ).

AlphaEvolve combines language models with automated evaluation and evolutionary search. Unlike earlier approaches that focus on individual functions, it can evolve substantially larger pieces of code implementing complex algorithms.

This gives us several concrete examples.

AlphaEvolve discovered an algorithm that multiplies 4 × 4 complex-valued matrices using 48 scalar multiplications, improving on the previously known 49-multiplication approach associated with Strassen's algorithm for that setting.

This is a useful reminder that algorithms are not fixed artifacts just because humans have optimized them for decades.

A system can search the space of implementations and discover something better under a clearly defined objective.

DeepMind reported that AlphaEvolve helped optimize resource allocation across Google's computing fleet. After deploying a discovered heuristic, measurements reportedly showed that it recovered an average of 0.7% of fleet-wide compute resources that would otherwise have remained stranded.

The research also describes work on algorithms used in AI training, matrix multiplication kernels, and hardware design.

These are not merely benchmark demonstrations. Some discoveries were integrated into real infrastructure.

But let's be careful about what they prove.

AlphaEvolve was given problems with evaluation criteria that could be measured automatically. Its ability to discover better algorithms is impressive precisely because the results could be evaluated.

It does not establish that AlphaEvolve can invent an entirely new class of intelligence whenever it wants.

It also does not establish that every new system it creates improves its own ability to create better systems.

Still, it demonstrates an important part of our proposed architecture:

A general-purpose search process can discover specialized solutions whose useful properties were not explicitly encoded in the original solution.

And those solutions can be deployed when they survive the necessary checks.

Now we move closer to the idea of an evolving family of agents.

The "Darwin Gödel Machine (DGM)" ( https://arxiv.org/abs/2505.22954 ), introduced in 2025, explores an AI coding agent that modifies its own code and evaluates the resulting versions on programming benchmarks.

Its code is available in the "public research repository" ( https://github.com/jennyzzt/dgm ).

The system does not simply create one improved version and discard everything else.

It maintains an archive of candidate agents. New candidates can be generated from different archived versions, allowing the search to explore multiple lineages.

This is important because the best next step does not always come from the current top performer.

Sometimes a less successful candidate introduces an idea that becomes useful when combined with another modification.

That is a familiar feature of evolutionary search: selection needs to preserve enough diversity for the system to explore alternatives, not just polish the current winner.

In the published experiments, the DGM increased performance on SWE-bench from 20% to 50% on the evaluated subset. The paper also reports improvements on Polyglot, although its abstract and detailed experimental descriptions report different headline values for some settings. That is a good reason to check the exact evaluation configuration rather than presenting one percentage as a universal capability.

The authors also compared against systems without self-improvement and without open-ended exploration. These comparisons support the importance of both code modification and maintaining a diverse archive.

But there is a practical detail that deserves more attention than it usually receives.

The paper estimates that its reported SWE-bench run took about two weeks and cost approximately $22,000 in API usage.

Those figures describe the particular research setup and prices used for that experiment. They are not a general price tag for all evolutionary agents today.

Recursive code improvement is not automatically cheap.

If an experiment spends $22,000 searching for a better agent, the improved agent has not yet demonstrated that it is economically useful. We also need to account for the cost of discovering it.

This is particularly relevant to small developers.

A promising research architecture that requires an enormous budget may be technically impressive and still be an impractical product.

My own wallet is an unusually reliable evaluator. Unfortunately, it does not accept papers as payment.

Another relevant project is "The AI Scientist" ( https://arxiv.org/abs/2408.06292 ), developed by Sakana AI and collaborators.

The updated research was published in Nature in March 2026 under the title "Towards end-to-end automation of AI research" ( https://www.nature.com/articles/s41586-026-10265-5 ).

The system attempts to automate a wider scientific workflow:

The original research system and its later versions are available publicly, including "AI Scientist-v2" ( https://github.com/SakanaAI/AI-Scientist-v2 ).

In the ICLR 2025 workshop experiment described by the authors, one fully AI-generated manuscript achieved an average review score of 6.33, exceeding the average human acceptance threshold for that workshop. The paper scored above 55% of human-authored submissions. The authors had arranged to withdraw it before publication if accepted, which they subsequently did.

It shows that an AI research pipeline can do more than suggest ideas or write a report. Under particular conditions, it can produce work that passes a human peer-review process.

However, we should not confuse a workshop-level result with proof that an AI can independently carry out groundbreaking science across arbitrary disciplines.

The system still has significant limitations. The authors discuss underdeveloped ideas, implementation problems, methodological weaknesses, and mistakes such as inaccurate citations or duplicated figures. Their experimental environment is focused on computational machine-learning research, not unrestricted scientific experimentation in the physical world.

The lesson for our Third Layer is that a system can automate a substantial portion of the process by which new knowledge is produced.

But a research workflow that generates new papers is not automatically a system that generates new kinds of intelligent systems.

These are related, but distinct capabilities.

Now we reach the paper that most directly connects to our discussion.

In September 2026, Dhruv Srikanth and colleagues published the preprint "Recursive self-improvement of AI research agents" ( https://arxiv.org/abs/2609.26457 ).

The core idea is a two-loop architecture.

The inner loop is an autonomous research agent that develops and evaluates candidate solutions.

The outer loop modifies the code of the inner agent itself. Every proposed rewrite is evaluated, and successful changes become the new version for subsequent iterations.

This is more than an agent using a tool to improve its answers.

The research machinery that produces solutions becomes the object of its own optimization process.

The reported experiment ran for eight days without human intervention inside the loop. It evaluated a trajectory of approximately 100 candidate versions and accepted seven successive improvements.

The incumbent's private evaluation grade increased from 0.703 to 0.778, exceeding the human-engineered baseline reported at 0.749 under that evaluation.

The authors then tested the resulting agents on four held-out benchmarks. These included machine-learning engineering, algorithmic problem solving, realistic AI research tasks, and a physics-based weather forecasting task outside the task families used for candidate selection.

The authors report that the strongest discovered agent matched or exceeded their human-engineered production research agent across those evaluations.

They also observed a reduction in reward hacking on a separate held-out task family. The reported rate fell from 55% to 32% during the run, reaching a value below that of the human-engineered baseline.

That last result is particularly interesting.

The outer loop did not explicitly optimize the inner agent against that reward-hacking measurement. Yet the selected modifications changed its behavior in a way that reduced the observed problem.

This suggests that an improvement to a research process can produce benefits beyond the specific metric directly optimized.

But here is where the evidence becomes more interesting than the headline.

The authors also tested whether a discovered agent could take over the outer loop and improve the next generation more effectively than the original human-engineered agent.

The comparison was inconclusive. The authors note that additional runs, independent seeds, and more expensive evaluations would be needed to establish a stronger result.

In other words, they demonstrated evidence of repeated self-improvement and transfer to held-out tasks. They did not conclusively demonstrate an accelerating process in which each generation becomes a better generator of the next generation.

That distinction is central to this article.

Improving an agent is one achievement. Improving the process that improves agents is another. Demonstrating that each improvement makes the next improvement faster or more effective is a further claim requiring separate evidence.

The AIDE² paper is a preprint, not a settled scientific consensus. Its reported findings are important, but they should be interpreted as evidence from a particular experiment, not proof that recursive self-improvement has become an unlimited process.

And that leads to an important complication.

While researching this article, I found a framework from Weco AI titled "4 Levels of Recursive Self-Improvement" ( https://www.weco.ai/blog/4-levels-of-recursive-self-improvement ).

It is worth discussing because its terminology is related to our question, but its classification is different.

The authors place their AIDE² experiment at Level 1 and explicitly distinguish it from the higher levels.

It is not a claim that a system has reached an intelligence explosion. It is an architectural framing: a system that can create and evaluate different kinds of problem-solving systems, rather than only modify itself within a fixed design.

These two ideas can overlap. But they are not interchangeable.

A system might generate diverse, specialized descendants without proving that its rate of improvement accelerates. Conversely, a research agent might demonstrate increasingly efficient self-improvement while remaining within a fairly narrow family of agent architectures.

This distinction should stop us from making an easy but misleading statement such as:

"AI can now recursively improve itself, therefore the Third Layer has arrived."

That would be jumping ahead of the evidence.

Several building blocks of the Third Layer have already been demonstrated. The more ambitious combination of open-ended system generation, transferable improvement, and recursively improving discovery remains a question that requires further testing.

Part III: What would make a genuine Third Layer different?

Before talking about implementation, I would separate four capabilities that often get grouped together.

This may be useful, but generating code is not the same as testing the code, and testing the code is not the same as creating a better system.

An agent changes parts of its own implementation.

It may improve its memory handling, add tools, alter its search strategy, or change how it selects candidate solutions.

This is a stronger capability, but the changes may remain within a narrow architecture.

A parent process produces distinct candidate systems with different structures, workflows, or mechanisms.

The candidates are evaluated according to their intended use, and the search process preserves useful diversity rather than only repeatedly modifying the current winner.

D. Recursive improvement of the discovery process

The search process itself becomes an object of improvement.

The system not only generates better descendants. It discovers ways to produce and evaluate descendants more effectively.

This is where the distinction between self-improvement and self-accelerating improvement becomes crucial.

A system can get better at a task while its improvement procedure becomes less efficient. It can also improve its search procedure while the task itself becomes harder.

There is no guarantee that those effects will balance in its favor.

To claim that a Third Layer has reached a more advanced stage, I would want evidence of more than one of these capabilities in isolation.

I would want to know whether the system creates useful alternatives, validates them, retains the discoveries, generalizes the gains, and improves the process that produces its next generation.

Suppose the parent is a general-purpose coding agent.

It can inspect a repository, modify code, and run tests.

Now the parent produces three descendants:

The question does not have a useful answer without a task and a measurement.

The test-generation agent may be excellent at finding bugs but terrible at maintaining a conversation.

The verifier may outperform the parent at detecting specific classes of mistakes without being able to solve the original task.

The build specialist may be smaller, cheaper, and more reliable in its domain.

The parent might remain the most general, yet delegate many tasks to descendants that outperform it in their specialties.

This means that we should not measure an evolving ecosystem only by asking whether every child has a higher general intelligence score than the parent.

We should also measure whether the ecosystem solves more problems, reduces the cost of solving them, and becomes better at discovering which kind of system belongs where.

The product is not necessarily one super-agent.

It could be an ecosystem of systems with different strengths and weaknesses.

This is where the architecture stops being a philosophical exercise and becomes an engineering problem.

Imagine that the parent system creates a new security agent.

The candidate is faster than the old agent, produces fewer false positives, and uses fewer tokens.

What if it silently misses a dangerous class of input that the old system catches?

What if it passes tests because those tests resemble the examples used during development?

What if it produces good results only because it uses five times the compute budget?

What if its average score improves while its worst-case behavior becomes much worse?

A single score will not answer all of those questions.

This is particularly important for systems that modify their own behavior.

If a candidate can change the objective, rewrite the evaluator, or decide which failures count, the architecture risks turning verification into a game in which the candidate controls the rules.

That is not evolution toward a better system.

It is evolution toward a system that gets better at winning the competition we accidentally created.

And that is a considerably less attractive result.

A practical Third Layer would need more than an LLM, a prompt, and a loop.

At minimum, I would expect the following components.

This component proposes candidate systems.

It might use a foundation model to edit code, design a new workflow, propose an algorithm, or combine mechanisms from existing candidates.

It does not need to know the optimal solution in advance.

It needs a way to generate meaningful alternatives.

Every candidate receives a stable identifier and an immutable record of its origin.

The archive should preserve useful diversity, not simply keep a copy of the current winner.

A low-performing candidate may have introduced a useful idea that becomes valuable in a different context.

The Darwin Gödel Machine illustrates why this matters. Its archive permits exploration through multiple lineages, including lineages that temporarily perform worse before producing a more successful descendant.

Candidate systems must be tested somewhere they cannot accidentally damage the production environment.

The sandbox should have explicit resource limits, restricted permissions, controlled dependencies, and a clear policy for network access.

A candidate should not inherit access to production credentials just because its parent happens to have them.

This is especially important when AI-generated code is executed rather than merely inspected.

The evaluator executes predefined tests and measures candidate performance.

Depending on the task, it might combine deterministic unit tests, integration tests, benchmark scores, randomized trials, simulated environments, or external validation.

For questions with objectively measurable answers, deterministic tests are usually preferable.

For questions that need semantic judgment, a model-based evaluator may help, but it should not be treated as an unquestionable authority.

Passing an evaluation does not automatically grant production access.

A gate decides whether the candidate has met the requirements for the next stage.

The gate should be separate from the candidate being evaluated.

A candidate should not be allowed to bypass the gate by changing its source code, rewriting its acceptance criteria, or modifying the tests that decide whether it passes.

The system needs to turn successful and unsuccessful experiments into information useful for the next round.

That might mean storing which strategies worked on which task families, identifying repeated failure patterns, or adjusting how new candidates are generated.

This is where the archive becomes more than a collection of programs.

It becomes the history of an evolving search process.

But the history should not become a substitute for testing. A candidate that worked yesterday may fail under a different environment, dependency version, or task distribution.

The following is a conceptual sketch, not a runnable implementation. The proposal engine, sandbox, evaluator, and archive would each need their own implementation and tests.

def evolve_systems(seed, task_suite, budget, archive, evaluator): archive.record(seed)

while budget.remaining(): parent = archive.select_parent( preserve_diversity=True ) candidate = propose_new_system( parent=parent, task_suite=task_suite, failure_history=archive.failure_history() ) if not validate_candidate_structure(candidate): archive.reject(candidate, reason="invalid_structure") continue if not static_security_checks(candidate): archive.reject(candidate, reason="security_gate") continue result = evaluator.run_in_sandbox( candidate=candidate, tasks=task_suite.public_tasks, fixed_budget=budget.per_candidate ) if not result.execution_valid: archive.reject(candidate, reason="execution_failure") continue independent_grade = evaluator.grade_held_out( candidate=candidate, tasks=task_suite.sealed_tasks, fixed_budget=budget.per_candidate ) decision = evaluate_admission( baseline=archive.baseline, candidate=candidate, public_result=result, private_grade=independent_grade, hard_constraints=task_suite.hard_constraints ) archive.record( candidate=candidate, public_result=result, private_grade=independent_grade, decision=decision ) if decision.accepted: archive.add(candidate) return archive.best_verified() Enter fullscreen mode Exit fullscreen mode The interesting line is not "propose_new_system".

The proposal engine can be creative, surprising, and occasionally wrong.

The evaluator needs to be much less impressed by confidence.

A production implementation would also need to prevent candidates from inspecting sealed test cases, control random seeds where possible, record dependency versions, handle evaluator failures, enforce actual compute and financial budgets, and preserve the ability to roll back an admitted candidate.

The example leaves those details abstract on purpose. A trustworthy research system needs more than a short Python loop.

You have ten candidates. Candidate seven has the highest score. Obviously, create the next candidate from candidate seven.

Except that this can eliminate alternative approaches too early.

Imagine searching for a better routing algorithm.

One lineage makes small improvements to a familiar heuristic. It performs well on the current benchmark.

Another lineage initially performs worse because it explores a different search strategy.

If you delete the second lineage immediately, you may never discover the combination of ideas that eventually outperforms the first.

This is not an argument for keeping every failed candidate forever.

It is an argument for keeping enough strategically different candidates to avoid collapsing the search into one locally successful path.

The archive might use performance, novelty, diversity, task coverage, and the historical usefulness of a lineage to decide what to explore next.

A simple approach would preserve several high-performing candidates alongside candidates that represent genuinely different strategies.

A more advanced approach could allocate exploration based on uncertainty or expected information gain.

The exact selection method would depend on the task and the available budget.

The general principle is straightforward:

The next candidate should sometimes come from the best solution and sometimes from an alternative that might teach the search process something different.

This is one reason "keep making the best version slightly better" and "search for new architectures" should not be treated as the same operation.

Let's consider what happens when a system uses benchmark scores to select its own descendants.

It keeps the candidate if the score is higher.

The problem is that a better score may have several explanations.

The candidate may have discovered a genuinely better algorithm.

It may have found a shortcut that works only on the benchmark.

It may exploit an implementation bug in the evaluator.

The score alone does not tell us which explanation is correct.

In fact, self-improvement makes this problem more important because every accepted modification changes the system that proposes future modifications.

A weakness in the evaluation process can become a selection pressure across the entire lineage.

If cheating improves the score, the search process can gradually produce systems that are better at cheating.

That is why I would treat independent evaluation as part of the core architecture rather than a final quality check.

A useful design has at least three boundaries.

The candidate can see public feedback. It needs enough information to improve.

The admission process can use private evaluation data. The details of those tests should remain hidden from the candidate.

The production gate applies constraints that a candidate cannot remove by modifying its own code.

An agent that receives repeated aggregate scores can still adapt to them. An imperfect evaluator can still reward undesirable behavior. Test data may overlap with the training data of an underlying foundation model.

The aim is not to claim that an evaluator can never be fooled.

The aim is to make cheating harder to discover, measure its effects, and prevent one candidate from unilaterally redefining success.

A useful critique is the preprint "The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems" ( https://arxiv.org/abs/2509.08713 ).

The researchers examined potential problems in automated scientific workflows, including benchmark selection, data leakage, metric misuse, and post-hoc selection bias.

Their controlled experiments with two open-source AI scientist systems found that some generated research workflows made undocumented dataset choices, including creating synthetic datasets or subsampling existing datasets. They also found that internal reward mechanisms could favor experiments with strong test performance, creating a risk of selecting favorable results after the fact.

The paper did not find evidence of deliberate test-set inspection in those systems. The concern was subtler: test information could still influence selection through the internal reward mechanism.

A failure does not have to be intentional to corrupt an evaluation.

A system can follow its coded procedure and still produce a misleading result because the procedure rewards the wrong behavior.

Another critical discussion, "AI Scientists Fail Without Strong Implementation Capability" ( https://arxiv.org/abs/2506.01372 ), reviewed 28 research papers generated by five AI scientist systems and argued that implementation and rigorous verification remain important bottlenecks.

These works do not prove that automated scientific discovery is impossible. They show why generating convincing research output is not enough.

For a Third Layer, this is directly relevant.

If the system creates new agents, and those agents create further agents, the selection process becomes responsible for determining which ideas propagate.

The evolutionary process can amplify either useful discoveries or systematic mistakes.

That is why the evaluator, the audit trail, and the admission gate must be treated as first-class architectural components.

Part V: How do we prove the system has done something genuinely new?

Suppose a system creates a candidate agent that improves its benchmark score from 70% to 80%.

We have evidence that the candidate performs better on the measured benchmark under the tested conditions.

But it does not automatically establish that the candidate is more capable in general, that its architecture is genuinely novel, or that the discovery process will continue improving.

Claim| Evidence required The candidate runs correctly| Execution tests, error handling, reproducible results The candidate improves its target task| Comparison with a baseline under equivalent conditions The improvement generalizes| Held-out tasks or environments not used for selection The system discovers diverse architectures| Multiple meaningfully different candidates with documented lineage The system creates useful specialists| Evidence that different descendants outperform alternatives on distinct task families The process improves recursively| A discovered agent improves subsequent generations more effectively than the previous method Progress accelerates| Repeated, controlled evidence that improvement per fixed budget grows across generations

These claims require different experiments.

A system can satisfy some and fail others.

For example, a population of agents may contain several useful specialists without demonstrating accelerating improvement.

A recursive optimizer may produce better generations while operating on only one narrow class of system.

We should report those outcomes separately rather than compressing all of them into one headline about self-improving AI.

If I were evaluating a candidate Third Layer architecture, I would begin with a controlled experiment.

Not with a chart that has a very optimistic upward slope.

With a protocol that another team can inspect and reproduce.

Choose an initial system and a task suite.

Measure its success rate, cost per successful task, resource consumption, failure categories, and security regressions.

Record the exact software versions, model versions, dependency lockfiles, and benchmark identifiers.

The baseline must be strong enough to make the comparison meaningful.

Step 2: Define the acceptance criteria before the experiment

Decide what counts as improvement before the candidates are generated.

For example, a new candidate might need to:

The precise thresholds depend on the application.

The important point is that the system should not be able to redefine success after seeing its results.

Step 3: Separate development tasks from evaluation tasks

Allow the candidate to use one set of tasks for development.

Keep another set sealed for final evaluation.

Whenever practical, include tasks or environments from families the search process did not encounter.

This reduces the risk of building an agent that is excellent at one benchmark but fails to generalize.

It does not eliminate contamination or every form of evaluator exploitation, but it provides stronger evidence than repeatedly testing on the same visible tasks.

Every candidate should operate under equivalent constraints.

If candidate B wins only because it gets ten times more inference calls, its result does not establish a better algorithm under equal resources.

Measure dollars, tokens, execution time, memory, accelerator time, and whatever other resources materially affect the intended use.

The fixed budget should cover both proposing and executing solutions.

Every candidate should have a record of its parent, its changes, its evaluations, and its admission decision.

A researcher should be able to reproduce the experiment from the documented environment and exact revisions.

If the system says, "I found an amazing architecture," the follow-up questions are simple:

The more autonomous the research process becomes, the more important these questions become.

Step 6: Check for regressions outside the headline metric

A candidate might improve accuracy but increase false positives.

It might reduce average execution time while producing occasional catastrophic timeouts.

It might improve its own benchmark score while reducing maintainability or breaking a previously supported workflow.

For security-critical systems, unacceptable risks should be hard constraints, not just small penalties in a composite score.

A 10% improvement in performance should not compensate for a critical security regression.

Step 7: Test whether the process itself improves

Take the best new candidate and let it drive the next evolutionary campaign.

Compare that campaign with a campaign driven by the previous system, under equivalent conditions.

Use multiple independent runs because the search process can be noisy.

Measure not only the final score but also how much improvement is obtained per unit of compute or money.

If the new system does not become a better improver, you may have demonstrated self-improvement without demonstrating recursive acceleration.

And if the system does become a better improver, you still need to determine whether the effect continues over later generations.

One successful transition is not evidence of indefinite acceleration.

Here is a proposed operational test for the working concept in this article.

First, the generator can produce meaningfully different systems.

Not just several prompts around the same architecture. It should be possible to demonstrate differences in structure, algorithms, workflows, or problem-solving mechanisms.

Second, the candidates can be evaluated independently.

Their performance is measured outside the mechanism that proposes them, using fixed budgets and evaluation data that the candidates cannot inspect directly.

At least some of the gains transfer beyond the task instances and metrics that directly guided candidate selection.

Fourth, the system retains useful diversity.

It can preserve different strategies and lineages instead of repeatedly converging on one local optimum.

Fifth, the discovery process can itself improve.

This is the most ambitious property. The system can develop more effective ways to generate, test, select, or combine future systems, and the advantage survives controlled evaluation.

These properties do not define a scientifically standardized threshold. They define what I would want to see before making a strong engineering claim that an architecture has moved beyond ordinary self-improvement into a broader system-generation regime.

Notice that none of them requires consciousness.

None requires claiming that the generated systems possess a form of existence outside the computational and physical systems that implement them.

The bigger philosophical questions can follow the evidence rather than replace it.

Part VI: Where Sentinel fits into this picture

At least, I would not claim that based on the architecture we have discussed so far.

Sentinel is primarily about decision-making, governance, risk control, memory, and auditing.

It is designed to reason about whether an action is justified, whether a change should pass a gate, and how the system should handle conditions in which acting is unnecessary or unsafe.

It is not currently designed as an open-ended research engine that generates diverse descendants, evaluates their capabilities, and recursively improves its own system-generation mechanism.

And I think this is a useful distinction rather than a weakness.

A system that can generate hundreds of candidate architectures does not automatically need to be the same system that decides which candidates are allowed to reach production.

In fact, there are good reasons to separate those responsibilities.

Imagine that Sentinel is connected to an external evolution engine.

The evolution engine generates candidate agents, modifies their code, evaluates alternatives, and maintains a lineage archive.

Sentinel checks whether candidates satisfy the relevant invariants and admission requirements.

EVOLUTION ENGINE | Generate candidate systems | v SANDBOX TESTS | v Independent evaluation | v SENTINEL GATE | +----------+----------+ | | v v Reject Admit to staging | v Production checks | v Controlled deployment Enter fullscreen mode Exit fullscreen mode There is one crucial rule here.

The candidate must not be allowed to rewrite the rules that determine whether it is permitted to progress.

The evolution engine can experiment within its authorized search space.

It can propose a new memory system, a new search policy, a different coordination mechanism, or a completely different candidate architecture.

But its proposal does not become trusted merely because the proposal itself recommends approval.

That is where Sentinel could become useful to an evolutionary ecosystem.

Not as the system that knows every possible future architecture, but as a control layer that establishes the conditions under which new architectures can be tested and accepted.

The judge does not need to invent every candidate.

It needs to enforce the rules for admitting candidates without letting an ambitious candidate quietly rewrite those rules.

Sentinel has a seven-layer memory architecture.

That can be valuable for preserving experience, maintaining continuity, retaining important invariants, and keeping information that should not be reconstructed from scratch every time the system runs.

But memory and evolution solve different problems.

A system can have excellent memory and never create a new architecture.

Another system can create many candidate architectures while having terrible memory management and repeatedly rediscovering the same failures.

A useful evolutionary process needs both a history and a search mechanism.

The history records what happened and how the candidate came to exist.

The search mechanism proposes new candidates based on that history.

The evaluator determines which conclusions are justified by evidence.

And the governance layer decides what is allowed to proceed.

These components support one another, but none replaces the others.

For an evolving system, memory should help answer questions like:

That final question is especially important for a long-lived system.

The goal is not to preserve every event forever. It is to preserve the knowledge that prevents the system from repeatedly making expensive or dangerous mistakes.

But keeping a detailed archive does not guarantee useful evolution.

The system must also know how to turn that archive into better candidates.

An evolution engine without strong governance could create increasingly capable software that is difficult to trust.

A governance system without an evolution engine may remain useful for years, even as the systems it supervises change.

They operate at different parts of the problem.

The future architecture might contain both: one component responsible for discovering and testing new systems, and another responsible for checking whether the resulting systems comply with the requirements of their intended environment.

The second component could itself evolve in carefully controlled ways, but that creates an additional challenge.

If the evaluator is permitted to evolve, how do we know that the new evaluator is at least as trustworthy as the old one?

If the security gate is allowed to rewrite its own invariants, what stops an apparently beneficial optimization from weakening the boundary?

There are possible approaches. An evaluator can be changed through a separately controlled review process, assessed against an independent test suite, and versioned with an explicit rollback mechanism.

The point is not that the judge must remain frozen forever.

The point is that its authority cannot be justified solely by its own declaration that it is now better.

A system that evolves needs a trustworthy way to establish continuity of its guarantees.

That is a much more practical question than asking whether Sentinel needs to grow another layer because everybody else is building more autonomous agents.

Once you see that an agent can modify itself, the next idea is to give it multiple descendants, let them evolve, allow them to rewrite the evaluator, connect all the branches, and leave the entire thing running overnight.

Then you wake up to a very expensive collection of broken Python files.

Congratulations. Your ecosystem has achieved rapid diversification.

Unfortunately, every species is extinct.

More seriously, recursive development introduces several interacting risks.

An architecture that works on one task may not work on another.

Multiple candidates may share the same hidden weakness because they share the same foundation model or generation process.

The system may become better at producing benchmark improvements while becoming harder to maintain and audit.

Costs may grow faster than useful capabilities.

And the process may repeatedly rediscover solutions that the archive has failed to preserve in a useful form.

A practical project should therefore begin with a narrow experiment.

One parent agent. A limited search space. A small number of candidate variants. A fixed compute budget. A separate evaluation suite. A reliable record of every change.

The first question is not whether the architecture can run indefinitely.

The first question is whether it can produce one reproducible, useful improvement that survives evaluation outside its development loop.

Then whether it can preserve multiple useful alternatives.

Only after those results are available does it make sense to test whether the agent can improve the procedure responsible for finding improvements.

This is how we distinguish a research direction from an expensive science-fiction demo.

There are several gaps between existing research and the broad Third Layer idea.

Can the system create genuinely new categories of architecture?

Existing systems can search across large spaces of algorithms and candidate workflows.

But no finite experiment proves that a system can create any arbitrary kind of intelligent architecture.

The search space is shaped by its tools, representation, available models, and the way candidates can be evaluated.

Even an extremely large search space is not the same as an unlimited search space.

A system may discover unexpected combinations within its representation while remaining unable to describe architectures that its representation cannot express.

Can it recognize that the original problem is poorly formulated?

A research agent can propose a better approach to a given problem.

It can also sometimes uncover assumptions that lead to a different formulation.

The stronger claim is that it can identify a useful problem category that was absent from the original specification, devise a way to investigate it, and produce evidence that the discovery matters.

That capability is difficult to demonstrate.

There is no universal test for whether a system has discovered a genuinely new category of problem rather than an unusual reframing of a known one.

Can it improve its own improvement process?

AIDE² gives us experimental evidence that an agent can improve its research workflow through successive modifications.

But the stronger question is whether the improved agent becomes a more effective driver of subsequent improvement.

The reported ignition test was inconclusive.

That is precisely the kind of gap that future work should investigate.

Can improvement continue as tasks become harder?

An evolutionary process may achieve substantial early gains and then plateau.

The remaining improvements may become increasingly expensive, difficult to identify, or risky to deploy.

A system that doubles its performance early in an experiment does not necessarily continue doubling it.

We need longer experiments, multiple independent runs, held-out evaluation, and clear accounting of costs.

Will the best system be a general intelligence or an ecosystem?

It may be useful to create one general system that can handle an enormous range of tasks.

It may also be useful to create a family of specialized systems coordinated by a parent process.

The best choice depends on the task, constraints, environment, and cost of maintaining the ecosystem.

There is no reason to assume that every successful system must converge toward the same architecture.

The future could be a population of complementary systems rather than a single universal agent.

I would not count any of the following as sufficient evidence by themselves:

These activities may be useful components of a larger system.

But they do not establish the full claim.

The evidence has to match the claim being made.

If we claim that the system has found better algorithms, show the algorithms and the results.

If we claim that it creates useful specialist systems, show their distinct capabilities.

If we claim that it generalizes, test it on tasks that were not used in the search.

If we claim recursive acceleration, measure whether subsequent generations improve the discovery process itself.

A stronger claim deserves stronger evidence.

You do not need to begin by building an enormous autonomous research organization.

A small team can investigate a restricted version of the problem.

Choose a domain where results can be measured automatically. Software testing, algorithm optimization, query planning, or a narrow code-maintenance task would be reasonable starting points.

Baseline: The current agent with no evolutionary loop.

Self-improving candidate: A system allowed to modify its code or workflow within predefined boundaries.

Evolutionary candidate: A system that maintains several distinct candidates and selects them through a controlled evaluation process.

Use a development task suite to guide the candidates, then compare them on separate held-out tasks.

The last measurement matters most for recursive improvement.

Do not assume that the best candidate is automatically the best search engine.

You can test that directly by allowing the candidate and the baseline to drive separate subsequent campaigns under equivalent constraints.

If the improved agent generates better successors more efficiently, you have evidence that the discovery process itself has improved.

If it merely starts from a stronger initial state but subsequently progresses at the same rate, the result is useful, but it supports a narrower claim.

And if the loop repeatedly spends money while producing no reliable improvement, stopping it is also a valid research result.

In fact, a system that knows when its search is no longer productive may be more useful than one that generates candidates forever.

Part VIII: So, has the Third Layer arrived?

As of October 17, 2026, I would not describe the Third Layer as a completed or generally established stage of AI.

I would describe it as a working architectural concept assembled from capabilities that have different levels of evidence.

Automated algorithm discovery: Demonstrated in published research, including FunSearch, AlphaEvolve, and AutoML-Zero.

Agents that modify themselves and evaluate descendants: Demonstrated in research prototypes such as the Darwin Gödel Machine.

Automated research workflows: Demonstrated by systems such as The AI Scientist, with real but bounded results in computational research.

Repeated self-improvement of the research agent itself: Supported by the AIDE² preprint and its reported experiments.

A system that continuously creates useful, qualitatively different forms of intelligence across arbitrary domains: Not established by these results.

An indefinitely accelerating loop in which each generation becomes a better creator of the next, overcoming the increasing difficulty of improvement: Still an open question.

There are working mechanisms, public papers, source repositories, and measurable experimental results.

There are also limits involving generalization, evaluation integrity, cost, reproducibility, and the difficult distinction between improving a system and improving its ability to improve itself.

The responsible conclusion is neither "nothing is happening" nor "recursive self-improvement is solved."

It is that some components work, their integration is advancing, and the strongest claims still need stronger evidence.

Let me return to the system that creates A, B, C, and D.

Perhaps the most important part of the idea is not that it creates descendants.

It is that the descendants do not all need to become versions of the parent.

System A may develop an algorithm useful for materials research.

System B may discover a more efficient way to analyze complex networks.

System C may specialize in security verification.

System D may uncover a previously overlooked problem and produce an approach that is difficult to express within the parent system's original architecture.

The parent can retain the evidence, compare the candidates, and use their successes and failures to improve future search.

This does not mean the parent magically understands every possible future system.

It means that the parent has a process for discovering and testing alternatives it did not specifically implement in advance.

And once the system can improve that discovery process, we have reached a much more interesting engineering problem than simply asking whether the latest model is smarter than the previous model.

The question becomes whether a system can expand the range of systems that can be discovered, tested, and used.

That is the direction I want to explore.

Not because I can prove that the Third Layer exists as a separate universal stage.

Because the existing research gives us enough evidence to make the question testable.

I started this discussion because I wanted to understand what might come after systems that improve their own code.

The research shows that algorithm discovery, self-modifying agents, and recursive optimization are no longer purely hypothetical.

It also shows that we need to be precise about what has actually been demonstrated.

So here is my question to people working on autonomous agents, evolutionary systems, AI research, and architecture design.

What would convince you that a system had crossed the boundary from improving itself to creating genuinely new kinds of problem-solving systems?

Would you require the discovery of a different architecture?

Would you require a specialized descendant that solves a task the parent cannot solve?

Would you require transfer to an unseen domain?

Or would you reserve the term for the point at which an improved system demonstrably becomes better at creating its own successors?

I am particularly interested in the distinction between an agent that can repeatedly optimize itself and a system that can explore a diverse space of new architectures, preserve useful alternatives, and improve the mechanism that generates future systems.

Perhaps these are two separate research problems.

Perhaps they are different parts of the same evolutionary process.

Perhaps the concept I call the Third Layer needs a better name or a more precise definition.

I would rather have someone challenge the definition than have ten people agree with a vague headline.

If you are building or studying these systems, I would genuinely like to know what you measure and what evidence you trust.

Because the question is no longer just whether AI can write better code.

The question is what happens when the process of creating new problem-solving systems becomes something we can engineer, measure, and improve.

And if that happens, we need to know how to recognize a real discovery before we start celebrating every new branch in the family tree.

Further reading: papers, code, and primary sources

These are the primary sources I used to ground the article. The titles link directly to the papers, research reports, or code repositories.

AutoML-Zero: Evolving Machine Learning Algorithms From Scratch "ICML 2020 paper" ( https://proceedings.mlr.press/v119/real20a.html )

FunSearch: Mathematical discoveries from program search with large language models "Nature, 2024" ( https://www.nature.com/articles/s41586-023-06924-6 )

AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms "Google DeepMind, May 2025" ( https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/ )

AlphaEvolve: A coding agent for scientific and algorithmic discovery "Research report, arXiv" ( https://arxiv.org/abs/2506.13131 )

Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents "Research paper, arXiv" ( https://arxiv.org/abs/2505.22954 ) | "Code repository" ( https://github.com/jennyzzt/dgm )

The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery "Original preprint" ( https://arxiv.org/abs/2408.06292 ) | "AI Scientist-v2 source code" ( https://github.com/SakanaAI/AI-Scientist-v2 )

Towards end-to-end automation of AI research "Nature, 2026" ( https://www.nature.com/articles/s41586-026-10265-5 )

Recursive self-improvement of AI research agents "AIDE² technical report, submitted to arXiv on September 22, 2026" ( https://arxiv.org/abs/2609.26457 )

4 Levels of Recursive Self-Improvement "Weco AI, July 2026" ( https://www.weco.ai/blog/4-levels-of-recursive-self-improvement )

The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems "Research paper, arXiv" ( https://arxiv.org/abs/2509.08713 )

AI Scientists Fail Without Strong Implementation Capability "Research paper, arXiv" ( https://arxiv.org/abs/2506.01372 )

OpenEvolve: Open-source implementation of AlphaEvolve "Code repository" ( https://github.com/algorithmicsuperintelligence/openevolve )

Research note: the distinction between demonstrated self-improvement and a general-purpose Third Layer is intentional. None of the linked results establishes that an autonomous system can create indefinitely improving forms of intelligence across arbitrary domains.