This page requires JavaScript. Please enable it to view the website.

The Agentic ScientistOrchestrating Coding Agents for End-to-End Scientific Co-Discovery

TL;DR
Our previous work on The AI Scientist (v1 and v2 ), including our recent Nature publication , showed that AI systems can automate substantial parts of the scientific research process, while also revealing important limitations: generated ideas can be underdeveloped and implementations can contain errors.
The Agentic Scientist is our next-generation AI Scientist, designed to address these limitations by orchestrating modern coding agents in a more rigorous, evidence-grounded research workflow. We apply the system across several machine learning research problems, including long-context model adaptation, RL fine-tuning, and learned optimizers. Notably, research on biologically inspired alternatives to backpropagation (called Error Diffusion) co-discovered through this process resulted in a paper accepted as an ALIFE 2026 oral presentation .

Resources

Authors

Robert Tjarko Lange*

Sakana AI

Luca Grillotti*

Sakana AI

Yutaro Yamada*

Sakana AI

Rujikorn Charakorn

Sakana AI

David Ha

Sakana AI

* Equal contribution.

Published

October 2026

Modern coding agents are solving day-to-day software engineering tasks with ever-increasing time horizons. This workflow has fundamentally changed how humans write software and optimize code. Scientific research, however, is more than the sum of engineering optimizations. A useful autonomous researcher has to define tasks, propose and evaluate ideas, turn them into an experiment, check whether the implementation really tests the idea, remember failures, compare variants fairly, and finally explain what happened based on grounded evidence. If we ignore those pieces, an “AI scientist” can easily become solely a strong hyperparameter optimizer in code space (aka. autoresearch ).

Similar to our previous work on The AI Scientist (v1 and v2), The Agentic Scientist is our attempt to make the research loop itself explicit. Instead of simply asking an agent to “do science” in a single conversation, we wrap coding agents in a scaffold that controls what they see, what they write, how they are evaluated, and how their results are passed to the next stage. The Agentic Scientist is a general framework that can coordinate different coding agents across different research problems, as long as each task provides a clear, verifiable goal through a scripted evaluator.

The Agentic Scientist improves upon our prior works in many ways, including incorporating agentic loops into many aspects of the scaffold, program optimization with parallel agents, rubric-based evaluation, evidence generation, a novel diagram generation method, and robust scientific paper writing. These improvements collectively allow computational scientific processes to be automated end-to-end while allowing human scientists to co-discover interesting research questions and hypotheses. Notably, the development version of The Agentic Scientist and our human scientists have co-produced a machine learning paper which has been accepted as an ALIFE 2026 oral presentation .


Successful Co-Discovered Paper:
from Error Diffusion experiments to an ALIFE paper

Our Error Diffusion project shows how the proposed research loop can support a co-discovery investigation between machines and humans. We used an early version of The Agentic Scientist to explore whether Error Diffusion, a biologically motivated learning rule, could extend beyond its earlier binary-classification demonstrations. We set the scientific question and the constraints a valid implementation had to preserve. The system proposed hypotheses, implemented experiments, and investigated failures. Our human scientists then used those results to choose promising directions and request further tests. This back-and-forth brought the work from an initial hypothesis toward a broader study of how the learning rule behaves, resulting in a coherent scientific paper.

Human–machine co-discovery pipeline: human direction, agent experiments, human-guided transfer to reinforcement learning, result verification, and review revisions
Human researchers defined the initial direction, baseline, and evaluation rubric; supported the transition from classification to reinforcement-learning experiments; verified results, references, and claims before submission; and incorporated reviewer feedback into the camera-ready paper. Orange boxes indicate stages involving human input.

In an early run, human inspection caught a high-scoring candidate that violated the intended learning rule, prompting us to tighten the rubrics. Within those constraints, the system developed a way to route error signals that enabled multiclass learning, then used ablations to identify which training adjustments mattered on each task. We took the strongest classification implementation into reinforcement learning and prepared the test environments. The system then adapted the method, tested variants, and analyzed where learning became less reliable. Each stage produced optimized solutions and experimental evidence, allowing human-in-the-loop to make decisions about what to try next.

The system organized the experiments and analyses into a manuscript. Before submission, we independently checked the implementations, results, and references, and revised claims that exceeded the evidence. The resulting study was accepted as an ALIFE 2026 oral presentation after external peer review. This is the human-steerable, end-to-end research process we want The Agentic Scientist to support: developing ideas, testing their consequences, and turning the findings into a scientific contribution. Agents expand the experimental work researchers can undertake, while human scientists steer the investigation and remain responsible for its conclusions.

Additional Case Studies: Putting The Agentic Scientist to Work

We first showcase two sampled papers below by putting the system to the test in practice: starting from a scientific goal and an existing codebase, proposing ideas, testing them, and investigating why the strongest approach works. We asked The Agentic Scientist to (i) improve long-context adaptation with Doc-to-LoRA and (ii) to explore learned evolution strategies built on CMA-ES . Together, these studies show how the same scaffold can support open-ended method development across different research problems, with experiments and ablations grounding the conclusions.


What's Inside The Agentic Scientist?

Overview of The Agentic Scientist
The Agentic Scientist workflow. Given a task description, parallel coding agents conduct rubric-based empirical hypothesis testing coordinated via a shared memory layer. The resulting winner program is used to determine which baselines, ablations, and post-hoc analyses to run; the resulting evidence then grounds the manuscript.

Task Setup & Drafting: The workflow begins with an executable task specification and evaluation procedure. Researchers can supply these directly or ask a drafting agent to build them from a research brief. Agent-drafted tasks undergo execution checks and an independent review of whether they match the intended scientific question. Once validated, a central controller distributes a fixed budget of research cycles across parallel workers, each exploring hypotheses in its own workspace.

Parallel Investigation: Inside each worker, an ideation agent proposes a hypothesis, an implementation plan, and rubrics specifying what the experiment must demonstrate. An execution agent then edits candidate programs, runs evaluations, and records metrics. All workers share memory entries and distilled conclusions at cycle boundaries, so successful ideas and failed attempts can inform subsequent experiments. Periodic context resets through agentic hand-off encourage fresh hypotheses while preserving each worker’s best program. This gives the search both continuity and room to change direction, without relying on a single ever-growing conversation.

Evidence Generation and Paper Writing: After optimization, the system selects the best correct program across workers and ablates its components. Ablation agents propose and check controlled baselines and ablations. An analysis agent then post-hoc reasons based on the logged metrics and produces the winner’s analysis. These results, together with the underlying code, logs, and artifacts, become the evidence for manuscript writing. The writer produces a LaTeX paper, overview plots via AutoIllustrate, comparison plots, and an explicit evidence map linking scientific claims to their sources. Automated checks and an independent audit look for unsupported numbers, inconsistent comparisons, and overclaiming. Finally, a reviewer agent assesses presentation and framing. Their feedback drives revisions, keeping the manuscript tied to what the experiments actually support.


Evaluating The Agentic Scientist Across Tasks, Harnesses, and LLMs

Pre-revision reviewer scores by model release date averaged across five tasks (left). Within-task percentile ranks of final performance and reviewer scores, colored by task (right).

Evaluation Setup: We evaluated The Agentic Scientist on five selected diverse research topics to assess the quality of its automated research. The evaluation covered nine frontier LLMs—four GPT models, four Claude models, and one DeepSeek model—and four agentic harnesses: Codex, Claude Code, OpenCode, and Pi. This totals 14 model–harness combinations and 70 research campaigns across all research tasks. Because the tasks require different kinds and amounts of computational resources, we used a common budget of 20 idea iterations per campaign to make comparisons easier. All runs for a given task instance used the same scoring function. We also assessed the generated manuscripts using AI Reviewer , treating its scores as a proxy for paper quality rather than ground-truth evidence of scientific correctness.

Main results and takeaways: Newer models generally received higher reviewer scores, particularly within the Claude model family, as shown in the left plot. In these evaluations, the choice of agentic harness had a smaller effect on performance than the underlying model. To compare numerical performance with manuscript reviewer scores across tasks, we converted both measures to within-task percentile ranks and pooled the results from all 70 research campaigns. The resulting positive association (r = 0.44; right plot) is consistent with reviewer scores tracking measured numerical performance.


AutoIllustrate: Editable Scientific Diagram Generation Pipeline

AutoIllustrate pipeline
The AutoIllustrate pipeline, generated by AutoIllustrate.

Darwin Gödel Machine overview figure before enhancement: the plain draw.io render
Before: deterministic draw.io render
Darwin Gödel Machine overview figure after enhancement by the image model
After: enhanced by the image model
The Darwin Gödel Machine overview figure before and after the enhancement step.
The editable vector draw.io diagram remains the source of truth; the image model only restyles it.

We also built AutoIllustrate, a companion pipeline for automatic scientific figure generation. Prior approaches either produce scientific diagrams as pixel-based images, making fine-grained corrections hard, or generate editable files whose layout is only checked visually. AutoIllustrate is vector-first: an editable, vector draw.io diagram is the source of truth, and every pixel-based output is derived from it, so human scientists can adjust any diagram by hand. Since the diagram is a structured file, a deterministic checker can verify its geometry, rejecting overlapping shapes and badly routed arrows before any figure is emitted. An image generation model then enhances the plain draw.io figures, adding more details and restyling them. In our evaluation on PaperBananaBench, AutoIllustrate improved faithfulness and aesthetics over PaperBanana , a strong raster baseline on that benchmark, while preserving an editable source for every diagram.

The gallery below shows overview figures that AutoIllustrate generated for several Sakana AI papers. Click a figure to open its editable draw.io source next to the enhanced version.

  • The AI Scientist
    Overview of The AI Scientist and the Automated Reviewer. (1) Idea Generation: an LLM mutation operator iteratively grows an archive of structured research ideas (hypothesis, experimental plan, self-assessed interestingness/novelty/feasibility); each idea is checked for novelty against the literature via Semantic Scholar and discarded if it duplicates existing work. In the template-based mode, ideation starts from a human-provided code template. (2) Experimentation: in the template-free mode, an experiment progress manager runs a parallelized agentic tree search over four stages (preliminary investigation, hyperparameter tuning, research agenda execution, ablation studies); each node writes and executes code, plots its results and has the plots critiqued by a vision-language model, being marked buggy (then debugged) or non-buggy (then refined), and the best node of each stage seeds the next stage. The template-based mode instead executes a sequential experimental plan with Aider. (3) Manuscript Writing: results are aggregated into figures and written into a LaTeX paper that is refined through reflection with VLM and linter feedback, then compiled to PDF. The Automated Reviewer ensembles five independent NeurIPS-style reviews into an Area-Chair meta-review that produces scores and an accept/reject decision.
  • DiscoPOP
    LLM-driven objective discovery. An LLM, seeded in context with established preference-optimization losses and their scores, proposes a candidate objective f_i as PyTorch code over policy and reference log-probabilities. Candidates failing unit tests are resampled with the error message as feedback; valid ones replace DPO to fine-tune an LLM on preference pairs, and the fine-tuned model is scored on MT-Bench (\eta). Appending (f_i, \eta) to the context closes the loop, which repeats until convergence or a generation budget. One discovered objective, DiscoPOP (LRML), blends the logistic (DPO) and exponential losses with a weight \sigma(\beta\rho/\tau) modulated by the log ratio.
  • Darwin Gödel Machine
    The DGM iteratively builds a growing archive of agents by interleaving self-modification with downstream task evaluation. Agents in the archive are selected for self-modification through open-ended exploration; each selected parent analyzes its own evaluation logs, proposes a feature, and implements it in its own codebase, and only children that still compile and can edit code are added back to the archive.
  • Continuous Thought Machine
    The CTM unfolds an internal sequence of ticks t = 1,\ldots,T, decoupled from the data. At each tick: (1) a synapse model f_{\theta_{\mathrm{syn}}} maps the concatenated post-activations \mathbf{z}^t and attention output \mathbf{o}^t to pre-activations \mathbf{a}^t; (2) the M most recent pre-activations form a history \mathbf{A}^t; (3) each neuron d has a privately parameterized neuron-level model (NLM) g_{\theta_d} that processes its own history \mathbf{A}^t_d to produce (4) post-activations \mathbf{z}^{t+1}_d; (5) post-activations accumulate in a growing history \mathbf{Z}^t; (6) neural synchronization \mathbf{S}^t = \mathbf{Z}^t (\mathbf{Z}^t)^\intercal, with learnable per-pair exponential decay r_{ij}, measures how neuron activity co-varies over time; (7) randomly sampled neuron pairs from \mathbf{S}^t form (8) synchronization representations \mathbf{S}^t_{\text{out}} and \mathbf{S}^t_{\text{action}}, projected into (9) outputs \mathbf{y}^t and attention queries \mathbf{q}^t; cross-attention over data features yields \mathbf{o}^t, which is (10) concatenated with \mathbf{z}^{t+1} for the next tick. Right: a loss is computed at every tick; the ticks of minimum loss (t_1) and maximum certainty (t_2) are selected per input and their losses averaged, giving native adaptive computation.
  • ShinkaEvolve
    Each generation (1) samples an island from the program archive and draws a parent (weighted by performance and novelty) together with top-K and random inspiration programs; (2) builds a mutation prompt and queries an LLM from a bandit-selected ensemble to propose a diff edit, full rewrite, or crossover, rejecting proposals whose mutable code is too similar to existing programs (embedding similarity, then an LLM novelty judge); and (3) executes accepted programs, storing fitness, public metrics and textual feedback in the archive. Relative-improvement rewards update the LLM selection probabilities (UCB1), and a periodic meta-scratchpad distils insights into future mutation prompts.
  • The RL Conductor which, alongside Trinity , powers Sakana Fugu ! 🐡
    Given a question q_i, a small trainable Conductor LM reasons in a chain-of-thought and then outputs an agentic workflow as three Python lists: the worker assigned to each step, a natural-language subtask for each step, and an access list selecting which previous answers enter each worker's context. Large frozen worker LLMs, drawn from a sampled k-of-n pool, execute the steps sequentially, and the final output o_i is rewarded (0 if the lists cannot be parsed, 0.5 if incorrect, 1 if it matches s_i) to train the Conductor with GRPO. Recursive calls let the Conductor re-plan from its own parent output and the last response, up to a maximum depth, giving a tunable test-time compute axis.
  • Error Diffusion
    Overview of the dual-stream Error Diffusion framework. Left: The excitatory/inhibitory architecture maintains separate streams (\mathbf{p}, \mathbf{n}) with four non-negative weight matrices per layer, enforcing Dale's principle structurally. Center: The Error Diffusion update broadcasts output error directly to hidden layers without weight transport or random feedback matrices. Right: The shared architecture is applied to classification (with domain-specific sigmoid widths, mean-centered error, and asymmetric initialization) and reinforcement learning (with ReLU activations and PPO integration).

Drag to browse. Click a figure to open its editable draw.io source and the enhanced version.


Lessons Learned and Limitations

Across our case studies, a recurring lesson is that the research task, evaluator, and verification process shape what the agent discovers. A high score can still come from a shortcut that violates the intended method (i.e., reward hacking), especially when important constraints are left implicit (e.g., the Error Diffusion experiment). Rubrics can make those constraints explicit, but they are also imperfect because of the agentic nature of rubric-based evaluation. Human researchers still need to choose meaningful questions, define what counts as a valid experiment, and check whether the resulting claims are ready for publication.

The scope for discovery also depends on the task. On a mature, heavily optimized benchmark, search may concentrate on hyperparameter tuning and small engineering changes, turning a science project into an engineering one. These gains can be useful, while their scientific significance depends on what we learn from them. More open-ended questions give agents room to explore alternative mechanisms, test competing hypotheses, and investigate when an idea generalizes. The Agentic Scientist can support both kinds of work. However, it leaves room for human researchers to judge and decide whether the optimized solution contains significant scientific discovery.

The current system also has gaps in how it uses prior work. Unlike our earlier AI Scientist systems, The Agentic Scientist searches the literature during manuscript writing rather than during ideation. This limits how directly new hypotheses build on existing research. It also lacks a reliable pipeline for turning published methods into working state-of-the-art baselines. Most generated comparisons instead modify the program selected during optimization. These comparisons help isolate which changes matter, but leave open how the method compares with the strongest alternatives from the literature.

Our evaluations have limits of their own. AI reviewer scores provide a proxy for paper quality, rather than independent evidence of scientific correctness; only the Error Diffusion paper underwent external peer review. All of our research tasks are in computational machine learning, so transfer to fields requiring physical experiments, specialized instruments, or human-subject studies remains untested. AutoIllustrate also needs broader human assessment: its benchmark uses a single vision-language model as judge, and its gains in faithfulness and aesthetics come with trade-offs in readability and conciseness.

Ethical considerations: AI-assisted research raises risks for attribution, research integrity, and an already strained peer-review process. We therefore submitted only the Error Diffusion paper for external review, and only after human researchers verified its implementations, results, claims, and references. The other generated manuscripts served only as internal evaluation artifacts. Responsible use of systems like The Agentic Scientist also calls for provenance for generated code and text and disclosure of agent contributions. Researchers remain accountable for safety, authorship, final claims, and deciding whether the work is suitable for submission and external review.


Outlook

We see The Agentic Scientist as an opportunity to give researchers greater experimental reach with faster iteration loops. The system allows exploring more hypotheses within a question and testing connections across research directions that might otherwise go unexplored. The scaffold serves as a reusable workflow for both types of exploration. A longer-term possibility is to use the system to improve its own research workflow, provided we can reliably measure whether those changes lead to better science.

We open-source both The Agentic Scientist and AutoIllustrate for the community to build on top of. We are excited to see The Agentic Scientist co-generate impactful scientific discovery in the wild. Please check out our code and try it for yourself!

Sakana AI

Want to make AI that improves AI? Please see our (RSI) Careers page for more information.

Citation

@article{agentic_scientist,
  title   = {The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery},
  author  = {Lange, Robert Tjarko and Grillotti, Luca and Yamada, Yutaro and Charakorn, Rujikorn and Ha, David},
  journal = {arXiv preprint arXiv:2610.18933},
  year    = {2026}
}