Insilico Medicine Convenes an Open Consortium For Benchmark Quality in AI-Driven Drug Discovery
aiinsider ·
Insilico Medicine has launched an open initiative called the Open Drug Discovery and Development Consortium (O3DC) to improve transparency in how computational drug-discovery benchmarks are evaluated. The consortium's first resource is the O3DC Benchmark Index, which catalogs hundreds of benchmarks across 10 categories and documents known biases, limitations, and criticisms for each entry. It also records practical details such as maintainers, code repositories, and activity status, which are verified against GitHub. The index is free, community-curated, and does not require registration, allowing researchers to challenge entries or add new benchmarks. Notably, Insilico applies the same standards to its own tools, including listing caveats about proprietary baselines. The article highlights examples of benchmark flaws, such as DUD-E's ligand-based bias and PDBbind's data leakage issues, which are rarely flagged when researchers use the data. Insilico's founder states the goal is to give critical knowledge a home alongside the benchmarks themselves. The consortium is open to researchers from academia, industry, and independent practice.
aiinsider