Key Takeaways

  • Uber's annual AI budget was exhausted in four months, not from unchecked spending, but because usage was far more unpredictable and higher than initial forecasts.
  • Andrew Macdonald, Uber's President and COO, found tangible AI ROI (like cutting a 15-hour capital allocation process down to 2 hours) difficult to translate into direct OPEX reductions.
  • To truly extract efficiency and control costs, companies need to set tighter operational constraints and combine budget pools for both engineering headcount and AI compute.
  • Giving teams better visibility into the real-time cost of their AI usage can make them more conscious and strategic about resource allocation.

The Invisible Drain: AI's Unpredictable Costs

Imagine blowing through your entire annual budget in a single quarter. That's exactly what Uber faced with its AI spend, according to Andrew Macdonald. He clarifies this wasn't a case of "runaway spend" where costs spun out of control, but rather an issue of pure unpredictability. "Praveen wasn't making a comment about like runaway spend like we're going to bankrupt ourselves," Macdonald said, referring to internal discussions. "He was just saying like it… it's hard to predict usage. Usage has been more than I thought."

This isn't about irresponsible spending; it's about the inherent nature of AI. Models can scale in unexpected ways, and user adoption or query volume can explode overnight, making even the most meticulous forecasts obsolete. For a founder or builder, this means that even if you budget carefully for initial development, the operational costs of running AI in production can quickly become a black hole if not managed with a different mindset.

From Time Saved to Hard Dollars: Tightening the Leash

AI delivers undeniable value. Macdonald points to a capital allocation process that used to take 15 hours every week across thousands of global markets. AI slashed that to just 2 hours. “That is tremendous tangible ROI, cuz now you get 2 days of someone's time back,” he notes. The problem? That time saved rarely converts directly into immediate OPEX reductions. You gain efficiency, but rarely do you immediately cut headcount because one task is faster.

So, how do you turn efficiency into actual cost savings? Macdonald suggests a two-pronged approach. First, “the way companies ultimately have to extract AI efficiency, at least from like a pure OPEX perspective, is just in your target setting, hold the constraints tighter.” This means actively forcing teams to operate within stricter cost boundaries, rather than just hoping for implicit savings.

Second, and perhaps most critically for builders, he proposes a fundamental shift in budgeting: combining resource pools. “You have to create combined pools of budgets and then let the people that you trust allocate where they see a higher ROI,” Macdonald explains. This means merging budgets for engineering headcount and AI compute power. Instead of two separate buckets, a team lead now decides: is it more impactful to hire another engineer to build new features, or to allocate those funds to compute for scaling existing AI, or even running more experiments? This puts the decision-making power where the ROI is clearest.

Finally, visibility matters. Imagine a real-time cost counter in the corner of every tool your team uses. “If I literally, you know, imagine a counter in the top right of whatever I'm tool I'm using, that is just showing me the equivalent cost of what I'm doing and as that scales, that would make you more cognizant as a user, right?” This simple step can instill a sense of ownership and cost-consciousness across the organization.

What to Do With This

If you're building an AI-powered product, stop treating AI compute as a separate, unpredictable line item. Tomorrow, sit down with your lead engineer and product manager. Agree to combine your engineering headcount budget with your AI compute budget into a single, fungible pool. Empower them to make real-time decisions on whether to hire another developer or spin up more GPUs, based purely on what delivers the highest, most immediate ROI for your product.