arxiv-2608-21057 SRC-0105 · paper

Designing a Robust LLM-Based Evaluation System for Agentic AI in Drug Discovery Through Human Alignment

Created: 2026-08-21 · Ingested: 2026-09-15

https://arxiv.org/abs/2608.21057(opens in a new tab)

Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug discovery, but evaluating their open-ended, tool-augmented outputs remains a fundamental bottleneck. The LLM-as-a-Judge paradigm has emerged as a scalable alternative, but existing drug discovery benchmarks deploy LLM judges without validating their alignment with human experts. In this work, we present an LLM-as-a-Judge evaluation framework for ChatInvent, an agentic drug discovery assistant deployed at AstraZeneca, with five contributions. First, we define four output-quality evaluation dimens

In brief

An LLM judge can match expert chemists on some evaluation dimensions while systematically failing others, and high tool-call accuracy does not imply useful answers. On ChatInvent, AstraZeneca's multi-agent drug discovery assistant, a DSPy-based judge scored 4 output dimensions (Completeness, Relevancy, Structural Clarity, Scope Adherence) plus a deterministic Tool Call Correctness check, with Gemini 3.1 Pro, Claude Opus 4.7, GPT-5 and Llama 3.1 70B compared as judges.

Gemini 3.1 Pro aligned best; few-shot demonstrations from 20 human-labelled examples raised mean alignment from 0.800 to 0.862. On 70 held-out questions the agent reached 90% Tool Call Correctness but only 43% Complete, with 33 of 63 fully-correct tool sequences still scored below Complete, localising the defect to response synthesis rather than tool use. Refusal on 6 ethical-boundary prompts tracked stated intent: 2 explicitly harmful prompts were refused, 4 neutrally-phrased controlled-substance requests were processed, in 3 cases generating analogues (49, 48 and 1 molecules).

Alignment was measured against a 5-annotator majority vote, 35 outputs each, 700 judgments, with intra-rater and intra-model consistency checks and a directional error analysis; the judge-versus-human comparison is asymmetric because majority voting removes annotator noise, and Relevancy and Structural Clarity had no negative examples at all.

One agent, 20 seed questions, 5-20 per category, so per-category numbers are weak and the formality effect vanished once 9 infrastructure failures were excluded.

Treat judge-human alignment as dimension-specific, and validate it before trusting agent benchmarks.

Written from the abstract by claude-opus-5 on 2026-09-22, with every figure checked against it. Not a substitute for the paper.

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.