Getting the Source Right, Not Just the Fact: Source-Aware Verification for MCP Agents
huggingface ·
The article introduces ProvenanceGuard, a source-aware verification layer designed for LLM agents using the Model Context Protocol (MCP). It addresses the issue of cross-source conflation, where an agent correctly identifies a fact but attributes it to the wrong data source. Unlike traditional faithfulness checkers that pool evidence, ProvenanceGuard maintains distinct source identities throughout the verification process. The system decomposes answers into claims, identifies the relevant source, and checks if the attribution matches the actual supporting evidence. In tests on medical agent traces, ProvenanceGuard demonstrated high accuracy in blocking unsupported claims while providing per-claim source verdicts. The framework also includes a repair mechanism that can revise blocked answers or provide safe fallbacks when necessary. This approach ensures that provenance is preserved, which is critical for sensitive domains like healthcare and finance.
huggingface