Key Takeaways

  • Over 10,000 Stripe employees use Kai, the company internal AI agent, every week across technical and non-technical teams.
  • Stripe treats AI projects as policy containers rather than passive chat folders, letting Directly Responsible Individuals (DRIs) set custom rules.
  • Sensitive teams like HR enforce tool policies that require human-in-the-loop verification before write actions, stopping accidental data leaks to public Google Docs.
  • Project-level model routing prevents overspending by assigning default, low-cost models to simple departmental tasks instead of default frontier models.

Why Blanket AI Rules Fail at Scale

Most companies handle AI safety with a blunt hammer. IT teams either lock down access entirely or hand out open access to frontier models with zero guardrails. Both extremes break down quickly. Lockdowns kill adoption and push employees to shadow AI tools on personal laptops. Open access creates catastrophic data leaks, like an HR team accidentally pasting confidential compensation bands into a public document.

At Stripe, engineering manager Sharadh Krishnamurthy built Kai, an internal AI agent supporting over 10,000 weekly active employees. Stripe avoided blanket company-wide bans by shifting governance directly into the project layer. Instead of enforcing one rigid policy for the entire company, Stripe lets individual project leads govern their own spaces.

Turning Folders into Governance Engines

In most tools, a project folder is just a way to organize old chat transcripts. Stripe turned that mental model upside down. As Krishnamurthy explained, projects serve a dual role: they capture intent and act as security perimeters.

“The two things that we were very intentional about is the idea of projects,” Krishnamurthy said. “Projects are primarily a governance mechanism, but they also let you have this context engine. Projects are almost like intentionally the user is telling you what they are trying to do, and that's a very strong signal of intent, and that lets the AI perform a lot better.”

Claire Vo pointed out that this structural shift changes how teams operate: “What I haven't seen anybody talk about, which I actually think is really interesting, is using projects as a configuration layer and a governance layer on how your team actually uses AI to get a specific job done.”

This configuration layer also stops teams from burning cash on model compute. As Krishnamurthy noted: “A project can say, 'Hey, here's the default model we want people to use. We don't even want to let them use these super expensive models because the job that you're trying to do here doesn't need one of these super models to look at them.'”

Human-in-the-Loop on Sensitive Write Actions

The real risk with autonomous agents is not what they read; it is what they write and where they publish it. A read-only analysis of payroll data is safe, but an agent exporting that summary into an open workspace document creates an immediate incident.

Stripe solves this by letting departmental DRIs set granular tool policies. Krishnamurthy highlighted how HR teams handle this tension: “Let's say you're a person on the HR team who's dealing with a bunch of sensitive information. You really don't want the agent to sort of go rogue and put that sensitive data into some public Google document that all Stripes can access, but you also don't want to tell them, 'Oh, you can't use any tools because your workloads are too sensitive.'”

The solution is conditional human approval. “Someone who is the DRI of a space can decide that certain tools are sensitive for the workloads that these people are going to be using,” Krishnamurthy explained. “So we need a human in the loop to confirm if that action can be taken by the agent.”

By requiring human confirmation only for sensitive write actions inside specific projects, non-technical teams retain the full speed of AI assistance without risking company data.

What to Do With This

Audit your company AI workflows by mapping every active tool integration into read actions versus write actions. Tomorrow morning, pick your most sensitive department (HR, finance, or legal) and insert a mandatory human approval step on every write action before an agent touches a shared doc, database, or external API. Lock down the default model for routine internal queries to a smaller, cheaper model to instantly cut your inference costs.