Key Takeaways

  • Abridge, the $5.3BN vertical AI company, generates 40% of its AI outputs from models built entirely in-house.
  • This in-house development targets high-stakes healthcare workflows where "milliseconds matter" for clinicians, like summarizing a patient visit before they leave the room.
  • Shiv Rao explained that for these "binary" tasks, proprietary models deliver superior speed and cost efficiency compared to relying solely on frontier models.
  • For other use cases, where continuous improvement is the main driver and a task "will never be perfect," Abridge intelligently opts to “ride the frontier wave” by integrating external large language models.
  • This hybrid strategy controls performance and P&L in critical areas, while benefiting from rapid advancements in general AI for broader applications.

The Method

In the age of frontier AI, founders face a dilemma: build everything, or rely on external APIs? Shiv Rao, CEO of Abridge ($5.3BN valuation), offers a third path. About 40% of Abridge's core AI models are built in-house, not for ego, but for pure pragmatism in healthcare.

Abridge's method identifies "moments of truth" in a clinician's workflow. Rao emphasized speed is non-negotiable in healthcare. He explained, "The milliseconds matter when you're in workflow. So these are doctors who are they they have no no patience for new technology." This drove Abridge to build highly optimized, specialized models for "binary" tasks. These are functions demanding immediate, precise outputs, like an "approved order" or critical summary, often "before the patient leaves the room."

For these critical, time-sensitive applications, Abridge engineers its own models. This gives them control over latency, inference costs, and medical language nuances general models might miss or slow to process. The goal is invisible AI, seamlessly integrated into clinical practice.

Abridge isn't a purist, though. For use cases needing continuous learning or tasks that "will never be perfect," Rao's team integrates frontier models. He noted, “If the market cares about you always improving on something that you don't think you'll ever be perfect on... it makes more sense to ride the frontier wave.” This dynamic allocation gives Abridge the best of both worlds: bespoke speed where it counts and agility with frontier AI. This strategic split manages performance and P&L, letting Abridge scale without drowning in compute costs or self-built stack development cycles.

Where This Breaks Down

While Abridge's hybrid model offers a blueprint for vertical AI, it's not a universal solution. This method requires significant capital investment and a deep bench of AI talent to pull off. A small, early-stage startup with limited runway will struggle to hire the specialized machine learning engineers needed to build even 40% of its models in-house. The compute costs for training and maintaining proprietary models are substantial, and the operational overhead for an internal AI team can quickly outstrip a lean budget.

Furthermore, if your vertical doesn't have "milliseconds matter" workflows or highly "binary" tasks where speed and specific accuracy drive adoption, the benefits of in-house development diminish. For many businesses, the generic capabilities of frontier models, even with slight latency, might be "good enough" for initial market entry and validation. Trying to build an internal stack for problems that don't demand extreme performance is a fast track to over-engineering and burning cash, especially when external APIs are rapidly improving and becoming more cost-effective for general use.

What to Do With This

Audit your product's core user journeys this week. Map out the 1-2 "moments of truth" where immediate, flawless output would drastically improve user experience or unlock a new workflow. For these specific junctures, sketch out a quick cost-benefit analysis: What would it take to build a highly optimized, proprietary model for just that part of the product? Compare that to the performance, cost, and developer time of using an external API. Use this clarity to define a precise "build vs. buy" strategy for your AI stack, ensuring you invest internal engineering cycles only where they deliver outsized, critical value to your specific vertical.