Past battle · 2024-11-29 UTC
Research AI Agents Showdown — November 29, 2024
From the Research AI Agents category. 2 marks placed across 2 fighters. Autoresearch took the crown.
Final standings
THEUS (Aigora)
AutoresearchThe line-up
The fighters
Profiles of every tool that competed in this battle, ranked by their final score.
Autoresearch
An open-source project that lets AI agents autonomously run LLM training experiments and keep the best model changes.
Autoresearch is an open-source project that enables AI agents to autonomously run LLM training experiments and retain the best model changes. The project allows users to set up a small but real LLM training environment and let an AI agent experiment with it overnight, modifying the code, training for a short period, and checking if the results improve. The goal is to automate the research process, letting the AI agent explore different model architectures, hyperparameters, and optimization strategies without human intervention. The project includes a simplified single-GPU implementation of nanochat and provides a basic structure for programming the AI agent's research process using Markdown files. The project is designed to be extensible, allowing users to add more agents and improve the research process over time.
Criteria breakdown
- Autonomous LLM training experiments
- AI agent-driven research process
- Single-GPU implementation of nanochat
- Markdown-based programming for the research process
- 5-minute training time budget with evaluation metric (val_bpb)
THEUS (Aigora)
Enterprise AI research system that turns proprietary studies into traceable insights with fact IDs, citations, and expert avatars for synthesis.
THEUS is an enterprise AI research system that transforms proprietary studies into traceable insights with fact IDs, citations, and expert avatars. It's built on Google's Agent Development Kit (ADK) and uses bidirectional modeling for data-grounded knowledge agents. Users can extract, synthesize, and explore their research data, and every claim is backed by a specific Fact ID with page-level citations. THEUS offers two ways to explore: generating new insights with Dr. Reed or analyzing existing knowledge with Dr. Sinclair. This platform is designed for VPs of Insights, Global Sensory Leads, and Innovation leaders evaluating strategic capability build-out. It uses a purpose-built research methodology, not generic enterprise AI, and creates data-grounded knowledge agents trained on actual institutional data. With 85% of new products failing within two years (Nielsen), evidence-grounded exploration dramatically improves decision quality. THEUS offers a solo seat, which includes features such as up to 30 documents ingested per month, up to 1,500 pages parsed per month, and up to 500 questions to Dr. Sinclair per month. One of the key benefits of THEUS is its ability to turn decades of proprietary data into traceable, decision-ready intelligence with AI-powered knowledge agents.
Criteria breakdown
- Extract, synthesize, and explore research data
- Generate new insights with Dr. Reed or analyze existing knowledge with Dr. Sinclair
- Bidirectional modeling for data-grounded knowledge agents
- Connect product and consumer knowledge for bidirectional insight
- Supports cross-study synthesis and research gap analysis