Key Takeaways

  • Frontier models deliver brilliant output 95% of the time, but Almeida points out that acting like “Mr. Bean the other 5%” (play from 0:07) destroys expected value in automated operations.
  • Enterprise software cannot automate high-stakes internal functions like accounting until systems deliver deterministic “nines of intelligent reliability.” (play from 4:39)
  • Current enterprise AI deployments trap models in consumer-facing sidebars because companies would rather pass verification costs to end users than risk direct corporate liability.
  • True workflow automation requires separating generative text engines from specialized decision models that optimize solely for judgment.

High Variance Destroys Expected Value

Founders often celebrate a demo that runs correctly nine times out of ten. In enterprise operations, that math fails immediately. TypeSafe AI co-founder Diogo Almeida argues that severe intellectual variance stops automation in its tracks, far more than any limits in raw compute.

As Almeida puts it: “You get Einstein like maybe 95% of the time and Mr. Bean the other 5%. And the problem is that in expected value it just does not work out.” (play from 3:32)

If an automated accounting agent runs 1,000 journal entries a week and acts like Albert Einstein on 950 of them, the 50 times it acts like Mr. Bean can trigger tax penalties, audit failures, or regulatory fines. The expected downside of five catastrophic mistakes wipes out the savings produced by the 95 successful runs. Because frontier labs optimize for broad benchmark tests rather than consistent judgment, this variance remains baked into foundation models. The market celebrates high scores on inflated benchmarks, but businesses cannot run unattended software without extreme consistency. True automation demands background execution that resolves tasks indefinitely. To achieve that, software requires what Almeida terms "nines of intelligent reliability."

Why Chatbots Absorb Corporate Liability

Because foundation models cannot hit five nines of reliability, enterprise buyers refuse to grant them unsupervised agency. Instead, they relegate models to conversational copilot sidebars and customer service windows.

Almeida cuts through the corporate spin behind these deployments: “The argument I always make is that LLMs are unreliable. So they cannot be trusted for decisions with stakes to the company. But you can always make them make decisions with stakes to the user because companies tend to not give a [ __ ] about their users.” (play from 28:35)

When a model drafts an internal tax filing, the company owns the financial penalty of an error. When that same model powers a consumer help desk, the end user absorbs the cost of bad instructions, wasted hours, and incorrect answers. A chat box acts as a legal and operational shock absorber. It forces a human being to check the model's work so the business does not face direct exposure. Breaking past this ceiling requires moving past purely generative text engines for core enterprise logic. “And if you optimize for the important stuff to be reliable on which is decision-making and judgment,” (play from 7:04) Almeida notes, “that is how you get real reliability.” (play from 7:08)

What to Do With This

Map every automated step in your core workflow to its error cost. If a 5% failure rate would force an executive to review the output or trigger customer churn, pull the generative model out of the execution loop immediately. Strip away text-generation capabilities from that step and replace the prompt with a constrained decision model restricted to discrete, deterministic actions.