Key Takeaways

  • Dan Shipper expects an applied AI lab to throw out roughly 90% of what it builds.
  • Internal dogfooding on actual daily work is the fastest test to separate genuine utility from short-lived novelty.
  • Every turns discarded experiments into external content to extract marketing value from failed builds.
  • Prototypes that miss core product criteria can be routed to an early adopter customer program to test niche demand.
  • Lab teams move faster when paired in small two-person units of fast prototype builders and structured system architects.

The 90% Scrap Rate in Applied AI

Shipper points out a blunt reality for software teams experimenting with AI models: most ideas fail. When building on fast-moving model capabilities, an applied lab throws away roughly 90% of what it creates. The challenge is not avoiding dead ends. The challenge is making sure those dead ends do not drain capital and time without return.

“The biggest thing that matters on a labs team is making the feedback loop as tight as possible between making something and knowing if it's good,” Shipper says. “And the tightest feedback loop is making something for yourself.”

If an engineer builds an AI assistant for sales reps but has never run a sales pipeline, testing that assistant takes weeks of scheduling, interviewing, and observation. If that same builder writes an AI tool to speed up their own coding, writing, or data cleanup, they know within twenty minutes whether it actually saves time or merely creates friction.

Distinguishing Utility From Novelty

The trap in AI prototyping is confusing technical novelty with actual value. Large language models generate fluent text and plausible code instantly. That makes almost every toy demo feel impressive on day one.

Shipper filters out that illusion by forcing experiments straight into real operational tasks. “All the experiments ideally should be used for actual work that you actually have so you can tell is this useful or is it just new,” he explains. When a prototype fails to survive daily work by the builder who made it, the team scraps it immediately.

Monetizing the Discard Pile

When 90% of builds get trashed, product teams need systematic ways to recover value from discarded code. Shipper uses two specific distribution channels to make abandoned experiments net-positive:

First, Every converts technical experiments into editorial media. “One thing that we've done a lot at Every that has worked really well is we turn them into external content,” Shipper notes. Writing an essay or recording a video breakdown about why a specific prompt approach or retrieval pipeline failed generates audience trust and attracts technical talent. The experiment fails as software, but wins as brand marketing.

Second, Shipper feeds unpolished prototypes directly to select power users. “Another thing to do is use these experiments to feed an early adopter program,” he explains. Dedicated customers often tolerate rough edges and bugs in exchange for access to new tools. If an experiment solves an edge-case problem for twenty core users, it provides clear product signal and keeps VIP customers engaged, even if it never touches the main production codebase.

What to Do With This

Pick one internal operational bottleneck you or your co-founder handled manually this week. Have an engineer spend exactly four hours building a raw AI prototype to automate it for your own screen, then test it on tomorrow morning's workload. If you abandon it by lunch, publish the exact prompt setup and the failure logs as a technical postmortem by Friday.