Tain Research

We are building the infrastructure to enable large-scale autonomous research. Our goal is to accelerate the advent of recursive self-improvement, because we believe it is the shortest path to solving the world's most interesting and consequential scientific problems.

Our roots are in evolutionary and meta-learning research. In working to close the loop for autonomous AI research, we found that the infrastructure for training large language models is still in an early phase, and that there are many ways to make it more efficient. Some are trivial, and some are very difficult. Removing that inefficiency is a prerequisite for running research at enormous scale and cost.

One of our early goals is to standardize and commoditize open-source LLM inference. We are also developing algorithms that let agents make efficient use of large quantities of highly variable compute.

On the pure research side, we have developed the state-of-the-art efficiency for AutoML-Zero, the search for machine learning algorithms from scratch. Existing methods can discover nontrivial update rules and algorithmic components, but the search is expensive and heavily constrained by the choice of representation, mutation operators, and evaluation procedure. We study how to make it materially more sample-efficient while preserving the ability to discover genuine algorithmic structure rather than only tuning within a narrow hand-designed family.

We are also working on self-referential neural networks: models that can modify parts of their own update process or parameterization. In principle such a model can represent richer forms of adaptation than a fixed-weight system. In practice these systems are difficult to initialize and train, because useful self-modifying behavior has to emerge before it can be exploited. A central question is how to construct training procedures and inductive biases that make this class of models experimentally workable without collapsing back into ordinary, externally specified optimization.

If these problems excite you, contact chris@tainresearch.com. You can find some recommended reading below.

Recommended reading

  1. E. Real et al. (2020), AutoML-Zero: Evolving Machine Learning Algorithms From Scratch
  2. K. O. Stanley, R. Miikkulainen (2002), Evolving Neural Networks Through Augmenting Topologies
  3. M. Andrychowicz et al. (2016), Learning to learn by gradient descent by gradient descent
  4. C. Finn, P. Abbeel, S. Levine (2017), Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
  5. K. Irie, I. Schlag, R. Csordás, J. Schmidhuber (2022), A Modern Self-Referential Weight Matrix That Learns to Modify Itself
  6. L. Kirsch, J. Schmidhuber (2022), Eliminating Meta Optimization Through Self-Referential Meta Learning
  7. T. Dao, D. Y. Fu, S. Ermon, A. Rudra, C. Ré (2022), FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
  8. W. Kwon et al. (2023), Efficient Memory Management for Large Language Model Serving with PagedAttention
  9. C. Lu, C. Lu, R. T. Lange, J. Foerster, J. Clune, D. Ha (2024), The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
  10. A. Novikov et al. (2025), AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery
  11. J. Zhang, S. Hu, C. Lu, R. T. Lange, J. Clune (2025), Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents