Loading page previews…
Click a page to preview
Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream NetworksModern coding agents are solving day-to-day software engineering tasks with ever-increasing time horizons. This workflow has fundamentally changed how humans write software and optimize code. Scientific research, however, is more than the sum of engineering optimizations. A useful autonomous researcher has to define tasks, propose and evaluate ideas, turn them into an experiment, check whether the implementation really tests the idea, remember failures, compare variants fairly, and finally explain what happened based on grounded evidence. If we ignore those pieces, an “AI scientist” can easily become solely a strong hyperparameter optimizer in code space (aka. autoresearch
Similar to our previous work on The AI Scientist (v1 and v2), The Agentic Scientist is our attempt to make the research loop itself explicit. Instead of simply asking an agent to “do science” in a single conversation, we wrap coding agents in a scaffold that controls what they see, what they write, how they are evaluated, and how their results are passed to the next stage. The Agentic Scientist is a general framework that can coordinate different coding agents across different research problems, as long as each task provides a clear, verifiable goal through a scripted evaluator.
The Agentic Scientist improves upon our prior works in many ways, including incorporating agentic loops into many aspects of the scaffold, program optimization with parallel agents, rubric-based evaluation, evidence generation, a novel diagram generation method, and robust scientific paper writing. These improvements collectively allow computational scientific processes to be automated end-to-end while allowing human scientists to co-discover interesting research questions and hypotheses. Notably, the development version of The Agentic Scientist and our human scientists have co-produced a machine learning paper which has been accepted as an ALIFE 2026 oral presentation
Loading page previews…
Click a page to preview
Diffusing Blame: Task-Dependent Credit Assignment in Biologically Plausible Dual-Stream NetworksLoading paper previews…
Our Error Diffusion project shows how the proposed research loop can support a co-discovery investigation between machines and humans. We used an early version of The Agentic Scientist to explore whether Error Diffusion, a biologically motivated learning rule, could extend beyond its earlier binary-classification demonstrations. We set the scientific question and the constraints a valid implementation had to preserve. The system proposed hypotheses, implemented experiments, and investigated failures. Our human scientists then used those results to choose promising directions and request further tests. This back-and-forth brought the work from an initial hypothesis toward a broader study of how the learning rule behaves, resulting in a coherent scientific paper.
In an early run, human inspection caught a high-scoring candidate that violated the intended learning rule, prompting us to tighten the rubrics. Within those constraints, the system developed a way to route error signals that enabled multiclass learning, then used ablations to identify which training adjustments mattered on each task. We took the strongest classification implementation into reinforcement learning and prepared the test environments. The system then adapted the method, tested variants, and analyzed where learning became less reliable. Each stage produced optimized solutions and experimental evidence, allowing human-in-the-loop to make decisions about what to try next.
The system organized the experiments and analyses into a manuscript. Before submission, we independently checked the implementations, results, and references, and revised claims that exceeded the evidence. The resulting study was accepted as an ALIFE 2026 oral presentation after external peer review. This is the human-steerable, end-to-end research process we want The Agentic Scientist to support: developing ideas, testing their consequences, and turning the findings into a scientific contribution. Agents expand the experimental work researchers can undertake, while human scientists steer the investigation and remain responsible for its conclusions.
We first showcase two sampled papers below by putting the system to the test in practice: starting from a scientific goal and an existing codebase, proposing ideas, testing them, and investigating why the strongest approach works. We asked The Agentic Scientist to (i) improve long-context adaptation with Doc-to-LoRA
Loading page previews…
Click a page to preview
Addressing Beats Architecture: Query-Conditioned Merge-Time Selection for Context-to-LoRA Hypernetworks on Multi-Needle RetrievalStarting from the public repository of Doc-to-LoRA, we asked The Agentic Scientist to improve how a language model retrieves facts from long documents. Doc-to-LoRA encodes document chunks as low-rank adapters (LoRA;
Loading page previews…
Click a page to preview
How Much of a Learned Evolution Strategy Is the Algorithm It Was Anchored To?For evolutionary optimization, we asked The Agentic Scientist to explore how learning could improve the design of a black-box optimizer. The system proposes optimizer variants, meta-trains their learned components on small mathematical test problems, and evaluates them on policy search for a continuous control task. It develops a CMA-ES-based strategy with learned adjustments that improves performance over the course of optimization relative to a CMA-ES reference. It then investigates where that improvement comes from by disabling the learned adjustments and comparing the resulting control with the full method. The control retains much of the advantage and achieves better final returns in longer runs, while the learned component mainly accelerates early progress. This is the evidence-generating part of the research loop: testing which innovations matter and identifying when they help. The continuous control task comparison gives researchers a concrete explanation of the gain and a starting point for testing whether the same mechanisms help on other tasks.
Loading paper previews…
Task Setup & Drafting: The workflow begins with an executable task specification and evaluation procedure. Researchers can supply these directly or ask a drafting agent to build them from a research brief. Agent-drafted tasks undergo execution checks and an independent review of whether they match the intended scientific question. Once validated, a central controller distributes a fixed budget of research cycles across parallel workers, each exploring hypotheses in its own workspace.
Parallel Investigation: Inside each worker, an ideation agent proposes a hypothesis, an implementation plan, and rubrics specifying what the experiment must demonstrate. An execution agent then edits candidate programs, runs evaluations, and records metrics. All workers share memory entries and distilled conclusions at cycle boundaries, so successful ideas and failed attempts can inform subsequent experiments. Periodic context resets through agentic hand-off encourage fresh hypotheses while preserving each worker’s best program. This gives the search both continuity and room to change direction, without relying on a single ever-growing conversation.
Evidence Generation and Paper Writing: After optimization, the system selects the best correct program across workers and ablates its components. Ablation agents propose and check controlled baselines and ablations. An analysis agent then post-hoc reasons based on the logged metrics and produces the winner’s analysis. These results, together with the underlying code, logs, and artifacts, become the evidence for manuscript writing. The writer produces a LaTeX paper, overview plots via AutoIllustrate, comparison plots, and an explicit evidence map linking scientific claims to their sources. Automated checks and an independent audit look for unsupported numbers, inconsistent comparisons, and overclaiming. Finally, a reviewer agent assesses presentation and framing. Their feedback drives revisions, keeping the manuscript tied to what the experiments actually support.
Evaluation Setup: We evaluated The Agentic Scientist on five selected diverse research topics to assess the quality of its automated research. The evaluation covered nine frontier LLMs—four GPT models, four Claude models, and one DeepSeek model—and four agentic harnesses: Codex, Claude Code, OpenCode, and Pi. This totals 14 model–harness combinations and 70 research campaigns across all research tasks.
Because the tasks require different kinds and amounts of computational resources, we used a common budget of 20 idea iterations per campaign to make comparisons easier. All runs for a given task instance used the same scoring function. We also assessed the generated manuscripts using AI Reviewer
Main results and takeaways: Newer models generally received higher reviewer scores, particularly within the Claude model family, as shown in the left plot. In these evaluations, the choice of agentic harness had a smaller effect on performance than the underlying model. To compare numerical performance with manuscript reviewer scores across tasks, we converted both measures to within-task percentile ranks and pooled the results from all 70 research campaigns. The resulting positive association (r = 0.44; right plot) is consistent with reviewer scores tracking measured numerical performance.
draw.io render
draw.io diagram remains the source of truth; the image model only restyles it.We also built AutoIllustrate, a companion pipeline for automatic scientific figure generation. Prior approaches either produce scientific diagrams as pixel-based images, making fine-grained corrections hard, or generate editable files whose layout is only checked visually. AutoIllustrate is vector-first: an editable, vector draw.io diagram is the source of truth, and every pixel-based output is derived from it, so human scientists can adjust any diagram by hand. Since the diagram is a structured file, a deterministic checker can verify its geometry, rejecting overlapping shapes and badly routed arrows before any figure is emitted. An image generation model then enhances the plain draw.io figures, adding more details and restyling them. In our evaluation on PaperBananaBench, AutoIllustrate improved faithfulness and aesthetics over PaperBanana
The gallery below shows overview figures that AutoIllustrate generated for several Sakana AI papers. Click a figure to open its editable draw.io source next to the enhanced version.
Drag to browse. Click a figure to open its editable draw.io source and the enhanced version.
Across our case studies, a recurring lesson is that the research task, evaluator, and verification process shape what the agent discovers. A high score can still come from a shortcut that violates the intended method (i.e., reward hacking), especially when important constraints are left implicit (e.g., the Error Diffusion experiment). Rubrics can make those constraints explicit, but they are also imperfect because of the agentic nature of rubric-based evaluation. Human researchers still need to choose meaningful questions, define what counts as a valid experiment, and check whether the resulting claims are ready for publication.
The scope for discovery also depends on the task. On a mature, heavily optimized benchmark, search may concentrate on hyperparameter tuning and small engineering changes, turning a science project into an engineering one. These gains can be useful, while their scientific significance depends on what we learn from them. More open-ended questions give agents room to explore alternative mechanisms, test competing hypotheses, and investigate when an idea generalizes. The Agentic Scientist can support both kinds of work. However, it leaves room for human researchers to judge and decide whether the optimized solution contains significant scientific discovery.
The current system also has gaps in how it uses prior work. Unlike our earlier AI Scientist systems, The Agentic Scientist searches the literature during manuscript writing rather than during ideation. This limits how directly new hypotheses build on existing research. It also lacks a reliable pipeline for turning published methods into working state-of-the-art baselines. Most generated comparisons instead modify the program selected during optimization. These comparisons help isolate which changes matter, but leave open how the method compares with the strongest alternatives from the literature.
Our evaluations have limits of their own. AI reviewer scores provide a proxy for paper quality, rather than independent evidence of scientific correctness; only the Error Diffusion paper underwent external peer review. All of our research tasks are in computational machine learning, so transfer to fields requiring physical experiments, specialized instruments, or human-subject studies remains untested. AutoIllustrate also needs broader human assessment: its benchmark uses a single vision-language model as judge, and its gains in faithfulness and aesthetics come with trade-offs in readability and conciseness.
Ethical considerations: AI-assisted research raises risks for attribution, research integrity, and an already strained peer-review process. We therefore submitted only the Error Diffusion paper for external review, and only after human researchers verified its implementations, results, claims, and references. The other generated manuscripts served only as internal evaluation artifacts. Responsible use of systems like The Agentic Scientist also calls for provenance for generated code and text and disclosure of agent contributions. Researchers remain accountable for safety, authorship, final claims, and deciding whether the work is suitable for submission and external review.
We see The Agentic Scientist as an opportunity to give researchers greater experimental reach with faster iteration loops. The system allows exploring more hypotheses within a question and testing connections across research directions that might otherwise go unexplored. The scaffold serves as a reusable workflow for both types of exploration. A longer-term possibility is to use the system to improve its own research workflow, provided we can reliably measure whether those changes lead to better science.
We open-source both The Agentic Scientist and AutoIllustrate for the community to build on top of. We are excited to see The Agentic Scientist co-generate impactful scientific discovery in the wild. Please check out our code and try it for yourself!
Want to make AI that improves AI? Please see our (RSI) Careers page for more information.
@article{agentic_scientist,
title = {The Agentic Scientist: Orchestrating Coding Agents for Automated Scientific (Co-)Discovery},
author = {Lange, Robert Tjarko and Grillotti, Luca and Yamada, Yutaro and Charakorn, Rujikorn and Ha, David},
journal = {arXiv preprint arXiv:2610.18933},
year = {2026}
}