Key Takeaways

  • Over 10,000 Stripe employees use Kai, the company's internal AI agent, every week to run operational tasks and inspect internal data.
  • Pointing an autonomous agent directly at a raw SQL data warehouse leads to brute-force queries, schema hallucinations, and crashed databases.
  • Prior investments in developer platforms, curated analytics layers, and clean data catalogs provide the true foundation for reliable AI execution.
  • Stripe prevents agents from writing raw queries immediately by routing data questions through Stripe's 3-Layer Data Agent Query Triage Hierarchy.

The Stripe 3-Layer Data Agent Query Triage Hierarchy

When builders give an LLM an open database connection, the model acts on instinct: it writes a speculative query against raw tables. If the query fails, it guesses again with more joins. Claire Vo notes the operational hazard: “Your data warehouse has to be very resilient to high volume queries because when in doubt an agent will just brute force it.”

To prevent agents from hammering infrastructure or hallucinating metrics, Sharadh Krishnamurthy and Stripe's data team designed the Ask Data skill. Instead of letting Kai compose SQL on the fly, the agent must step down through three distinct tiers:

  • Layer 1: Direct Artifacts & Existing Reports: Route to direct artifacts and existing reports first. Check standard dashboards and pre-existing analytical queries to satisfy the user request without generating new queries.
  • Layer 2: Blessed Analytics Layer: If no existing report matches, query the company's blessed analytics layer where verified key metrics and canonical data definitions live.
  • Layer 3: Data Catalog Fallback: If both direct artifacts and the analytics layer fail, search through the tiered data catalog schema for high-quality datasets and author a custom query.

Krishnamurthy explains that these controls mirror human best practices: “These investments were made for humans but have held up really well for agents because turns out reasoning through it, agents have the same problem. They can answer the question, but they have no idea if it was the right query or the right table.”

When This Works (and When It Doesn't)

This triage model works when your company maintains curated data assets, canonical metrics tables, and an active data catalog. If your organization already has verified dashboards and clean analytics tables, forcing the agent to check existing work first eliminates hallucinated definitions and prevents query overloads on systems like Trino.

It breaks down if your team lacks standard reporting and treats raw production tables as the only source of truth. If you have no blessed analytics layer, Layer 1 and Layer 2 return empty results every time. In that scenario, your agent will constantly drop to Layer 3, leaving your database vulnerable to expensive full-table scans and incorrect joins.

What to Do With This

If you are building an internal data query agent this week, block it from accessing your production SQL writer directly. Take your five most frequently asked data questions (like monthly churn, active subscriptions, or daily signups) and create two predefined tiers in your agent tool definitions:

1. Connect the agent's first tool call to your existing BI dashboard endpoints or saved queries.

2. Create a second tool restricted to querying pre-aggregated summary tables with strict row limits.

3. Keep raw database catalog search as a locked third tool that requires specific schema parameters before executing a query.