SMDD-Bench: Can LLMs Solve Real-World Small Molecule Drug Design Tasks?

LLM agents have incredible potential for scientific discovery applications. However, the performance of LLM agents on real-world, small molecule drug design (SMDD) tasks across diverse chemistries and targets is unclear. Current evaluation methods are either ad hoc, too simple for real-world discovery, limited in scale, or restricted to single-turn question answering. In effort to standardize the evaluation of LLM agents on small molecule design, we introduce SMDD-Bench, a challenging, multi-turn, long-horizon agentic benchmark consisting of 502 guaranteed-solvable task instances spanning 5 task types: 2D Pharmacophore Identification, Interaction Point Discovery, Scaffold Hopping, Lead Optimization, and Fragment Assembly. SMDD-Bench tasks span a wide region of chemical space and involve 102 unique protein targets. Completely solving the benchmark would require having strong chemical and biological reasoning and 3D intuition, understanding specialized tool use, and displaying planning expertise over a limited number of oracle calls. We benchmark 7 frontier open and closed source LLMs and find even the most performant LLM, GPT5.4, solves only 40.2\% of tasks. We hope SMDD-Bench provides a standardized testbed to invigorate the field towards training and evaluating LLM agents for fully autonomous computational drug design. We host a public leaderboard at smddbench.com .

📋 한 줄 요약

**[Drug Design / LLM Agent 벤치]** SMDD-Bench가 5 task·102 단백질 target에 걸친 502 guaranteed-solvable multi-turn long-horizon SMDD task 제공, 7 frontier LLM에서 최고 GPT5.4도 40.2%만 해결.

🎯 핵심 기여도

LLM 에이전트의 과학 발견 잠재력 큰 가운데, 실제 small molecule drug design(SMDD) task의 다양 chemistry·target에 대한 성능이 불명확함을 진단.
기존 평가법이 ad hoc·too simple·소규모·single-turn QA 한정 등 한계 가짐을 지적.
SMDD-Bench 도입 — 502 guaranteed-solvable task instance, 5 task type(2D Pharmacophore Identification·Interaction Point Discovery·Scaffold Hopping·Lead Optimization·Fragment Assembly) 포괄, multi-turn long-horizon agentic 벤치.
광범위 chemical space·102 unique 단백질 target. 완전 해결에는 강한 chemical·biological reasoning, 3D 직관, 전문 도구 사용, 제한된 oracle call 하 planning 전문성 필요. 7 frontier open·closed LLM 벤치마킹 — 최고 GPT5.4도 40.2%만 해결.

💡 핵심 아이디어

SMDD에서 LLM agent의 실효성은 multi-turn·long-horizon·실제 chemical 다양성을 포괄하는 guaranteed-solvable 벤치마크에서야 검증 가능하며, frontier 모델조차 40%대 해결률에 그친다는 사실이 자율 computational drug design 도달의 격차를 드러낸다.

🔬 기술적 접근법

**방법론**: SMDD-Bench — 502 task instance·5 task type·102 단백질 target의 multi-turn long-horizon 벤치.
**핵심 기법**: (1) 5 task type: 2D Pharmacophore Identification, Interaction Point Discovery, Scaffold Hopping, Lead Optimization, Fragment Assembly, (2) 102 unique protein target, (3) Guaranteed-solvable task instance 502개 보장, (4) Multi-turn·long-horizon agentic 평가 — 제한된 oracle call·도구 사용·planning, (5) 7 frontier open·closed LLM 평가·public leaderboard 운영.

📊 주요 결과

7 frontier LLM 평가.
최고 모델 GPT5.4가 40.2%만 해결 — 자율 SMDD 도달 격차 노출.
Strong chemical·biological reasoning, 3D 직관, tool use, planning 모두 필요함 확인.
Public leaderboard: smddbench.com.

💭 의의 및 한계

**의의**: SMDD에서 LLM agent의 표준 multi-turn 평가 정립, 자율 computational drug design 진전의 명확한 격차 노출, 향후 학습·평가의 testbed 제공. **한계**: 502 task의 chemical 다양성 한정, oracle call·도구 환경의 sim-real 격차, scaffold hopping 등 task의 평가 객관성, 절대 성능이 여전히 낮음.

🚀 실용적 활용

LLM 기반 drug discovery agent 평가.
자율 분자 design pipeline 진전 추적.
Chemical·biological reasoning 학습 벤치.