🚩 Official Website |
📄 Tech Report |
📌 Arxiv Link |
🧪 FDE-Bench |
Baidu AI Cloud
- [2026-10] 🔥 We release FDE-Bench, a benchmark for end-to-end delivery from underspecified real-world business requests: 49 real customer projects (29 combinatorial optimization + 20 machine learning) with 266 annotated clarification targets. Project Page · arXiv · Code
- [2026-03] We release Famou for Science, an autonomous research workflow powered by Famou, with a nuclear reactor physics example.
- [2025-10] The FM Agent technical report is available on arXiv.
| Work | Type | Description | Links |
|---|---|---|---|
| FDE-Bench | Benchmark | Evaluates whether agents can clarify missing requirements with a simulated customer and deliver a solution that meets real customer acceptance criteria. | Project Page · arXiv · Code |
| Famou for Science | Application | Drives research, experiments, and paper writing end to end with Famou; the example searches for analytical solutions of the 2D two-group neutron diffusion equation. | Directory |
| FM Agent | Framework | General-purpose multi-agent framework combining LLM reasoning with large-scale evolutionary search (details below). | arXiv · Tech Report |
FM Agent is a novel, general-purpose multi-agent framework that addresses complex real-world challenges by synergistically combining LLM-based reasoning and large-scale evolutionary search. Demonstrating broad applicability, our system has been evaluated across diverse domains, including operations research, machine learning, GPU kernel optimization, and classical mathematical problems.
This phase integrates diverse generation agents to produce a broad yet high-quality initial solution space. Moreover, with an optional expert-in-the-loop design, the framework ensures evolutionary search begins from a pragmatically grounded foundation, significantly accelerating convergence, especially in some real-world complex cases.
Our novel sampling strategy orchestrates multiple parallel evolutionary islands, adaptively balancing exploration and exploitation through dynamic resource allocation. This mechanism maintains productive diversity across algorithmic lineages while systematically steering the population toward global optima.
Custom evaluators synthesize multiple critical criteria—including functional correctness, operational effectiveness, and LLM-supervised quality assessment—to generate nuanced, multi-faceted feedback. This comprehensive scoring mechanism provides rich, cumulative signals that precisely guide the iterative refinement process.
Built on Ray, our scalable orchestration framework enables fine-grained, large-scale concurrent evaluation across distributed computing resources. This architecture ensures efficient resource utilization while facilitating rapid and systematic exploration of complex, high-dimensional solution spaces.
FM Agent reaches state-of-the-art results autonomously, without human interpretation or tuning — 1976.3 on ALE-Bench (+5.2%), 43.56% on MLE-bench (+4.0pp), up to 20× speedups on KernelBench, and establishes new state-of-the-art(SOTA) results on several classical mathematical problems.
💥💥💥FM Agent is currently ranked first on the MLE-bench Leaderboard.
If you use FM Agent in your research, please cite:
@misc{li2025fmagent,
title={The FM Agent},
author={Annan Li and Chufan Wu and Zengle Ge and Yee Hin Chong and Zhinan Hou and Lizhe Cao and Cheng Ju and Jianmin Wu and Huaiming Li and Haobo Zhang and Shenghao Feng and Mo Zhao and Fengzhi Qiu and Rui Yang and Mengmeng Zhang and Wenyi Zhu and Yingying Sun and Quan Sun and Shunhao Yan and Danyu Liu and Dawei Yin and Dou Shen},
year={2025},
eprint={2510.26144},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2510.26144},
}If you use FDE-Bench, please cite:
@misc{li2026fdebench,
title={FDE-Bench: Evaluating End-to-End Delivery from Underspecified Real-World Business Requests},
author={Huaiming Li and Can Huang and Chufan Wu and Danyu Liu and Xiaomin Yuan and Qinze Li and Yui Lo and Wenhui Bai and Qianlong Wang and Zengle Ge and Xiyu Yan and Yuheng Cheng and Jinhui Ren and Guokai Chen and Jiaqun Liu and Haoran Li and Qianhui Liu and Annan Li and Jianmin Wu and Dawei Yin and Dou Shen},
year={2026},
eprint={2610.14539},
archivePrefix={arXiv},
primaryClass={cs.AI},
url={https://arxiv.org/abs/2610.14539},
}This project is licensed under the MIT License. See the LICENSE file for details.
FDE-Bench's harness code is also licensed under MIT, but its benchmark data (FDE-Bench/case/ and the data archives) are licensed under CC BY-NC-ND 4.0. See FDE-Bench/LICENSE_STATUS.md for details. The paper and figures under docs/FDE-Bench/ remain with their authors, and the bundled fonts keep their own OFL licenses.
- GitHub Issues: Submit Issue