Past battle · 2025-03-28 UTC
Research AI Agents Showdown — March 28, 2025
From the Research AI Agents category. 12 marks placed across 5 fighters. Cell2Sentence took the crown.
Final standings
The line-up
The fighters
Profiles of every tool that competed in this battle, ranked by their final score.
Cell2Sentence
Open-source framework that turns single-cell gene expression into 'cell sentences' so LLMs can analyze and generate biology insights.
Cell2Sentence is an open-source framework that transforms single-cell gene expression data into 'cell sentences' for analysis and insight generation by Large Language Models (LLMs). It proposes a rank-ordering transformation of expression vectors into cell sentences, which are space-separated gene names ordered by descending expression. This allows LLMs to natively model single-cell RNA sequencing (scRNA-seq) data using natural language. The framework includes the C2S-Scale models, which unify transcriptomic and textual data and enable advanced single-cell tasks such as perturbation prediction, dataset summarization, cluster captioning, and biological question answering. The C2S-Scale models are available on Hugging Face and are based on architectures like Pythia and Gemma-2. Cell2Sentence is aimed at researchers and scientists working with single-cell transcriptomics data. The framework has been updated with new models and features, including support for fine-tuning on custom prompt templates and multi-cell prompt formatting. It also includes a suite of Pythia models for cell type prediction, cell type conditioned generation, and a diverse multi-cell multi-task model trained on over 57 million human and mouse cells. The Cell2Sentence framework is documented and has tutorials for use, including examples of fine-tuning and multi-cell prompt formatting. The development of Cell2Sentence involves the van Dijk Lab and has been published in a preprint on bioRxiv. Cell2Sentence enables next-generation single-cell discovery with LLMs.
Criteria breakdown
- Transformation of expression vectors into cell sentences
- C2S-Scale models for advanced single-cell tasks
- Support for fine-tuning on custom prompt templates
- Multi-cell prompt formatting
- Pre-trained models based on Pythia and Gemma-2 architectures
AlphaSense
An AI-powered market intelligence platform that autonomously sources expert insights and delivers analyst-level research at scale.
AlphaSense is an AI-powered market intelligence platform that provides trusted insights to guide business decisions. It offers a comprehensive collection of curated sources, including Tegus expert transcripts, broker research, and company filings. The platform uses advanced AI models to generate high-quality insights with sentence-level citations. AlphaSense integrates workflows, eliminating fragmented sources and providing streamlined, connected insights. Its GenAI capabilities provide real-time insights that ground decisions in reliable expertise. The platform is designed for investment banking, hedge funds, private equity, asset management, consulting, life sciences, and other industries. AlphaSense differentiates itself with its highly synthesized insights, synthesized from an unrivaled content set. The platform also features customizable agents to automate repeatable research tasks, transforming findings into executive-ready reports and slides. Its Generative Search provides a 360-degree view of markets and companies, connecting qualitative insights with structured financial data. One of AlphaSense's key strengths is its ability to empower users to make bold moves with confidence and speed. The platform gives teams the precise answers, unique perspectives, and critical updates they need to stay ahead of the competition.,
Criteria breakdown
- Curated sources
- Tegus expert transcripts
- Broker research
- Company filings
- Private and public financial data
- GenAI for real-time insights
AMIE
A multimodal AI diagnostic agent that conducts clinical conversations and interprets medical images for accurate diagnoses.
AMIE (Articulate Medical Intelligence Explorer) is a research AI diagnostic agent developed by Google DeepMind and Google Research. It is designed to conduct clinical conversations and interpret medical images for accurate diagnoses. Initially, AMIE was a text-based medical diagnostic conversational AI agent published in Nature. The latest advancement, multimodal AMIE, integrates the ability to intelligently request, interpret, and reason about visual medical information during clinical conversations. This development aims to improve diagnostic accuracy by incorporating multimodal data, such as images and documents, into the diagnostic process. AMIE uses a state-aware reasoning framework and is built on multimodal Gemini models. It has been evaluated through expert assessments, including Objective Structured Clinical Examinations (OSCEs), comparing its performance to primary care physicians (PCPs) in various patient scenarios.
Criteria breakdown
- Multimodal diagnostic dialogue
- Visual medical information interpretation
- State-aware reasoning framework
- Integration with Gemini models
- Simulation environment for dialogue evaluation
Autoresearch
An open-source project that lets AI agents autonomously run LLM training experiments and keep the best model changes.
Autoresearch is an open-source project that enables AI agents to autonomously run LLM training experiments and retain the best model changes. The project allows users to set up a small but real LLM training environment and let an AI agent experiment with it overnight, modifying the code, training for a short period, and checking if the results improve. The goal is to automate the research process, letting the AI agent explore different model architectures, hyperparameters, and optimization strategies without human intervention. The project includes a simplified single-GPU implementation of nanochat and provides a basic structure for programming the AI agent's research process using Markdown files. The project is designed to be extensible, allowing users to add more agents and improve the research process over time.
Criteria breakdown
- Autonomous LLM training experiments
- AI agent-driven research process
- Single-GPU implementation of nanochat
- Markdown-based programming for the research process
- 5-minute training time budget with evaluation metric (val_bpb)
Google AI Co-Scientist
A multi-agent AI system that collaborates with scientists to generate hypotheses and accelerate biomedical research.
The Google AI Co-Scientist is a multi-agent AI system designed to collaborate with scientists and accelerate biomedical research. It is built with Gemini 2.0 and functions as a virtual scientific collaborator to help generate novel hypotheses and research proposals. The system is intended to mirror the reasoning process underpinning the scientific method and uncover new, original knowledge. Given a scientist's research goal specified in natural language, the AI Co-Scientist generates novel research hypotheses, a detailed research overview, and experimental protocols. It uses a coalition of specialized agents, including Generation, Reflection, Ranking, Evolution, Proximity, and Meta-review, which use automated feedback to iteratively generate, evaluate, and refine hypotheses. The system allows scientists to interact with it in various ways, including providing seed ideas for exploration or feedback on generated outputs in natural language. The AI Co-Scientist also uses tools like web-search and specialized AI models to enhance the grounding and quality of generated hypotheses. The system is designed to flexibly scale compute and iteratively improve its scientific reasoning towards the specified research goal. The AI Co-Scientist is particularly useful for scientists who face challenges in navigating the rapid growth in scientific publications and integrating insights from unfamiliar domains. By leveraging recent AI advances, including the ability to synthesize across complex subjects and perform long-term planning and reasoning, the AI Co-Scientist aims to accelerate the clock speed of scientific and biomedical discoveries. The system has been tested in various applications, including gene transfer discovery and transfer re-discovery. While the AI Co-Scientist presents several advantages, its effectiveness depends on the quality of the input research goal and the scientist's ability to provide meaningful feedback. The AI Co-Scientist is a new tool that has the potential to significantly impact the scientific community, but its limitations and potential biases need to be carefully evaluated. The development of the AI Co-Scientist is an ongoing effort, and its future applications and improvements will depend on continued research and testing.
Criteria breakdown
- Hypothesis generation
- Research overview creation
- Experimental protocol development
- Specialized agents for scientific reasoning
- Automated feedback loop
- Web-search integration