Natural language security queries at Orca Security using Amazon Bedrock
aws ·
Orca Security partnered with AWS to develop Discovery AI, a natural language query agent built on Amazon Bedrock that translates plain text questions into structured Sonar queries for cloud infrastructure. This solution addresses the complexity of querying over 900 cloud models by replacing inefficient initial prototypes with a nine-stage hybrid pipeline combining Retrieval Augmented Generation and schema-aware construction. By allocating specific foundation models like Anthropic Claude Haiku 4.5 and Amazon Nova 2 Lite to different stages, the system optimizes for both reasoning capability and cost efficiency. The architecture features a quick deterministic matcher for high-confidence patterns and a semantic retrieval process using Amazon Titan embeddings reranked by Cohere Rerank 3.5. To ensure accuracy, the team implemented a continuous human-in-the-loop labeling system that allows domain experts to correct outputs and improve the example bank in real time. Production results demonstrate significant improvements, including a 90 percent reduction in costs, a 93 percent decrease in input tokens, and a 75 percent drop in latency compared to the baseline. Discovery AI now processes thousands of daily queries with end-to-end response times reduced from 60 seconds to just 15 seconds. This case study highlights how agentic architectures and multi-model strategies can make complex security data accessible through intuitive natural language interfaces.
aws