How an AWS team detects dashboard content failures at scale using Amazon Bedrock
aws ·
The article describes how an AWS team built an automated solution using Amazon Bedrock to detect content failures in business intelligence dashboards, which are often invisible to infrastructure monitoring. These silent failures include blank charts, stale data, and numeric inconsistencies that escape traditional checks, with fewer than 1 percent of incidents being reported by users. The system uses a five-stage serverless architecture: scheduling via Amazon EventBridge, screenshot capture with redaction, AI analysis using Anthropic Claude models on Amazon Bedrock, alert routing via Slack, and telemetry persistence in Amazon Redshift. Two parallel validation mechanisms handle visual integrity and numeric cross-source consistency, with LLMs performing semantic tasks like recognizing elements and deterministic code handling precise comparisons. The team emphasizes designing against false positives and keeping LLMs away from arithmetic to maintain trust. In production over 30 days, the visual validation mechanism detected 802 content failures, and the solution reduced mean time to detection from up to 72 hours to less than 1 hour.
aws