| System type | What it automates | Example |
|---|---|---|
| Neural architecture search (NAS) | Designing model architecture (layers, connections) | Automated search over convolutional or transformer block designs |
| Automated hyperparameter optimization | Tuning learning rate, batch size, regularization | Bayesian optimization and population-based training |
| AutoML platforms | Feature engineering plus architecture and tuning end-to-end | Google Cloud AutoML, H2O.ai, open-source AutoML frameworks |
| AI Scientist systems | Idea generation, experiment code, writing, and review | Sakana AI’s AI Scientist |
| AI co-scientist systems | Hypothesis generation and experimental design via multi-agent debate | Google DeepMind’s AI co-scientist |
Machine learning research has always been a slow, iterative loop: form a hypothesis about what might improve a model, design an experiment, run it, interpret the results, and repeat. For most of the field’s history, every step of that loop required a human researcher. That is no longer entirely true. Neural architecture search and automated hyperparameter optimization have quietly become standard parts of the ML toolkit, automating the “which architecture” and “which settings” questions that used to consume enormous researcher time. More recently, a new generation of systems — Sakana AI’s AI Scientist and Google DeepMind’s AI co-scientist among them — has started automating a level up: generating research hypotheses, designing experiments to test them, and in some cases writing up the results, with a human reviewing rather than driving each step. This piece covers both generations of automation, what each can and cannot do today, and how teams are actually using them.
Neural Architecture Search: Automating the “Which Model” Question
Neural architecture search treats model architecture itself as a search problem: instead of a researcher hand-designing a network’s layers and connections, a search algorithm explores a defined space of possible architectures, evaluating candidates against a validation objective and converging on designs that perform well. Early NAS approaches were extraordinarily compute-hungry, requiring thousands of GPU-hours to search a single architecture space, which limited practical use to organizations with very large compute budgets. Subsequent techniques — weight-sharing across candidate architectures, differentiable architecture search, and more efficient search strategies — have brought that cost down substantially, making NAS a realistic option for a much broader set of teams by the mid-2020s.
In practice, NAS today is most often used not to invent an entirely novel architecture from scratch, but to adapt and specialize an existing architecture family to a specific deployment constraint — for example, searching for the best trade-off between accuracy and inference latency on a specific hardware target, which is a narrower and more tractable search problem than open-ended architecture design.
Automated Hyperparameter Optimization
Hyperparameter optimization automates the tuning of settings that control training but aren’t learned by the model itself: learning rate, batch size, regularization strength, and similar values. Manual grid search and random search remain common baselines, but Bayesian optimization, which builds a probabilistic model of how hyperparameters affect performance and uses it to choose the next configuration to try, and population-based training, which evolves a population of models with different hyperparameters during training itself rather than in separate sequential runs, have both become standard tools in mature ML pipelines. These techniques matter because hyperparameter choices can affect final model performance by a wide margin, and automated search consistently finds better configurations faster than manual tuning, freeing researcher time for higher-level decisions.
Two layers of automated ML research
The lower layer (NAS, hyperparameter optimization, AutoML) automates decisions within a single, human-defined research question; the upper layer (AI Scientist and AI co-scientist systems) automates the generation of the research question itself, proposing and testing hypotheses a human did not necessarily specify in advance.
The New Wave: AI Scientist and AI Co-Scientist Systems
Sakana AI’s AI Scientist
Sakana AI’s AI Scientist demonstrated a fuller loop of automated research: generating a research idea, implementing the code needed to test it, running the experiment, analyzing results, and drafting a paper describing the work, including a simulated peer review step. This end-to-end demonstration was notable less for the quality of any single output and more for showing that the entire research pipeline, not just isolated pieces of it, could be chained together with limited human intervention at each stage.
Google DeepMind’s AI Co-Scientist
Google DeepMind’s AI co-scientist takes a different architectural approach, built as a multi-agent system on top of Gemini, where multiple agent roles generate, critique, rank, and evolve competing scientific hypotheses through a process resembling iterative tournament rounds rather than a single linear pipeline. Rather than aiming to write a finished paper, the system is oriented toward generating and refining testable hypotheses that human researchers then pursue experimentally. It has already been applied to real open research questions across fields including fibrosis, antimicrobial resistance, and cellular aging, with one reported case identifying a fibrosis drug candidate that blocked the majority of a scarring-linked response in laboratory tests — a result that still required human-led laboratory validation to confirm, underlining that these systems currently function as hypothesis generators and research accelerants rather than fully autonomous discovery engines.
| System | Architecture approach | Primary output |
|---|---|---|
| Sakana AI’s AI Scientist | End-to-end pipeline: ideation through paper drafting and review | A complete draft research paper with experiments |
| Google DeepMind’s AI co-scientist | Multi-agent debate and tournament-style hypothesis ranking on Gemini | Ranked, refined testable scientific hypotheses |
| AlphaFold / GNoME-style systems | Specialized prediction models for a narrow scientific domain | Structure or materials predictions accelerating human-led discovery |
What These Systems Actually Change About Research Workflow
The practical effect of both generations of automation is a shift in where human researcher time goes, not a removal of human researchers from the loop. NAS and hyperparameter optimization free researchers from manually exploring huge configuration spaces, letting them spend more time on problem framing, evaluation design, and interpreting results. AI Scientist and AI co-scientist systems push this further up the stack, automating the generation and initial triage of hypotheses so that human researchers spend more of their time deciding which automatically generated hypotheses are worth pursuing experimentally, and validating the ones that look promising, rather than generating every candidate hypothesis themselves from scratch.
This changes what a research team’s bottleneck looks like. Where compute and human hypothesis-generation time used to be the binding constraints, teams using these systems increasingly find that experimental validation capacity — wet-lab time, real-world data collection, rigorous statistical testing — becomes the new bottleneck, since automated systems can generate more plausible hypotheses than any team can practically validate.
Where the Limits Are
- Validation still requires human and often physical-world effort. A ranked hypothesis from an AI co-scientist system is not a confirmed result; laboratory or real-world validation remains a human-led step, and automated systems can generate hypotheses faster than any team can test them.
- Novelty and correctness are not the same thing. Automated research systems can generate ideas that sound plausible and are stated with confidence but are subtly wrong or untestable, requiring domain expert review before resources are committed to validation.
- Evaluation of “good research” is itself hard to automate. Peer review simulated by an AI system is not equivalent to genuine expert peer review, and treating an AI Scientist’s self-assessment as a substitute for real scrutiny is a known failure mode.
- Compute and data access still gate what’s searchable. NAS and hyperparameter optimization are only as good as the search space and compute budget defined for them; a poorly scoped search space wastes compute without finding meaningfully better configurations.
Common mistake
Teams sometimes treat an AI Scientist or AI co-scientist system’s confident-sounding output as equivalent to a validated finding, skipping the expert review step that would normally happen before committing real experimental resources to a hypothesis. Since these systems can produce plausible-sounding but incorrect or untestable ideas at a much higher rate than a human researcher would independently propose them, a mandatory expert triage step before any resource-intensive validation is essential.
What worked
A research team used an AI co-scientist-style system purely as a hypothesis generation and ranking front end, feeding its top-ranked candidates into their existing, unchanged experimental validation pipeline rather than trying to automate validation itself. This kept the human expertise bottleneck exactly where it already existed while meaningfully expanding the number of plausible hypotheses considered before committing to expensive experiments.
Frequently Overlooked Details
- Search space design matters more than search algorithmA well-scoped, domain-informed search space for NAS or hyperparameter optimization often matters more to final results than which specific search algorithm is used within it.
- Simulated peer review is not real peer reviewAn AI Scientist’s self-generated review step can catch some obvious errors but should not be treated as equivalent to genuine domain-expert scrutiny before resources are committed.
- Multi-agent debate can still converge on shared blind spotsSystems like AI co-scientist that use multiple agents to critique each other can still share systematic biases inherited from their underlying base model, since the “debate” isn’t a fully independent check.
- Automated research shifts the bottleneck, it doesn’t remove itExperimental and validation capacity, not idea generation, increasingly becomes the limiting factor once hypothesis generation is automated.
- Reproducibility still applies to automated experimentsExperiments designed and run by an automated system still need the same reproducibility discipline as human-designed experiments, including fixed seeds and documented configurations.
Glossary
- Neural architecture search (NAS)
- An automated method for discovering effective neural network architectures by searching over a defined space of possible designs.
- Hyperparameter optimization
- The automated tuning of training settings, such as learning rate or batch size, that are not learned directly by the model.
- AutoML
- Platforms and tools that automate multiple stages of the machine learning pipeline, including feature engineering, architecture selection, and hyperparameter tuning.
- AI Scientist system
- An automated system that generates research ideas, implements and runs experiments, and drafts findings with limited human involvement at each step.
- AI co-scientist system
- A multi-agent system designed to generate, critique, and rank scientific hypotheses for human researchers to pursue experimentally.
Key Takeaways
- Neural architecture search and hyperparameter optimization have matured into standard, widely used ML tooling.
- AI Scientist and AI co-scientist systems represent a newer wave automating hypothesis generation itself, not just model configuration.
- Sakana AI’s AI Scientist demonstrates a full pipeline from idea to draft paper with simulated peer review.
- Google DeepMind’s AI co-scientist uses multi-agent tournament-style debate to generate and rank testable hypotheses on real research problems.
- These systems shift researcher bottlenecks toward experimental validation capacity rather than idea generation.
- Confident-sounding automated output still requires expert human triage before committing real experimental resources.
- Automated research systems accelerate hypothesis generation but do not yet replace human-led experimental validation.
FAQs
What is automated machine learning research?
Automated machine learning research refers to AI systems that automate parts of the ML research process itself, ranging from architecture and hyperparameter search up to generating and testing entire research hypotheses with limited human involvement.
What is neural architecture search?
Neural architecture search is an automated method for finding effective neural network designs by searching over a defined space of possible architectures rather than having a human hand-design the network structure.
What is Sakana AI’s AI Scientist?
Sakana AI’s AI Scientist is a system that demonstrated an end-to-end automated research pipeline, generating research ideas, implementing and running experiments, and drafting papers including a simulated peer review step, with limited human intervention.
What is Google DeepMind’s AI co-scientist?
Google DeepMind’s AI co-scientist is a multi-agent system built on Gemini that generates, critiques, and ranks scientific hypotheses through iterative tournament-style debate, and has been applied to real research questions in fields like fibrosis and antimicrobial resistance.
Can these AI research systems replace human scientists?
Not currently; they function as hypothesis generators and research accelerants, but experimental and laboratory validation, along with expert review of proposed ideas, still require human researchers to confirm findings are correct and worth pursuing.
How does automated hyperparameter optimization work?
Common approaches include Bayesian optimization, which builds a probabilistic model to choose promising configurations to try next, and population-based training, which evolves a population of models with different hyperparameters during training rather than in separate sequential runs.
What is the main risk of using AI Scientist-style systems?
The main risk is treating confident-sounding but unvalidated hypotheses as equivalent to genuine findings, since these systems can generate plausible-seeming but incorrect or untestable ideas faster than a human researcher would, making expert triage before resource commitment essential.
What becomes the bottleneck once research automation is adopted?
Experimental and validation capacity, such as laboratory time or rigorous statistical testing, typically becomes the new bottleneck, since automated systems can generate more plausible hypotheses than any team can practically validate.
Teams interested in the infrastructure needed to support this kind of research automation may find our guides on building an ML platform team and reproducible ML useful, since automated experiments still require the same rigor as human-designed ones. Our piece on reinforcement learning from human and AI feedback compared covers a related automation trend in how models themselves are trained and evaluated, and our guide to LLM evaluation frameworks is directly relevant to judging the outputs these systems generate. For a look at how labelling strategy interacts with automated pipelines, see our article on active learning.
- Turing Post, “12 AI Co-Scientists of 2026: From Biology to Math”
- Medium, “Top AI Systems for Science and Engineering Discovery: Mid-2026”
- Springer Nature, “Systematic review on neural architecture search,” Artificial Intelligence Review
- MLCommons, “Introducing the 2026 MLCommons Rising Stars”
- Medium, “Machine Learning in 2026: The Trends Reshaping the Future of AI”
