Key Takeaways

  • Supabase released Supabase Compute to handle long-running background tasks like embedding and chunking, collocating workloads on the database's private network instead of hitting short timeouts on Deno Edge Functions.
  • OrioleDB serves as a drop-in storage engine replacement for Postgres heap tables, running 1.5 to 2 times faster while eliminating vacuum bloat.
  • Supabase hardened OrioleDB using PG regress tests, Antithesis software testing, and nightly TPC-C benchmarks before releasing it under an Apache 2 license.
  • Partnering with Vitess creator Sugu Sougoumarane, Supabase built Multigres, an open-source operator that survived 1,000 sequential failovers without losing data.
  • Multigres prioritizes complete query compatibility before sharding, meaning developers will get horizontal scaling without modifying application queries.

Moving Long Compute Next to the Data

Serverless functions work well until AI workloads enter your stack. For years, Supabase ran Edge Functions on Deno. They booted fast, but hard execution limits choked tasks like text chunking and vector embedding generation.

Developers had to pull data out of the database, send it across the public internet to external worker pools, process it, and write it back. Every step added latency and egress costs. Paul Copplestone saw the wall his users hit.

“What we're finding is that well what we've heard from our community for a long time is that they want to run much longer workloads,” Copplestone said. “They want to do things like chunking and embedding. So tomorrow we're announcing and by the time this comes out superbased compute which is you can run anything. It's longunning. It's serverless. It is running the same private network as your database. So it's colllocated has very good latency.”

Placing arbitrary compute inside the database network fixes the latency penalty. Instead of shipping data over the wire, you run the code where the rows already live.

Replacing the Postgres Heap Engine Without Breaking SQL

Postgres relies on an engine design called heap storage. When you update a row in Postgres, the engine does not overwrite it in place. It writes a fresh copy and marks the old one dead. A background cleanup job called vacuum runs constantly to reclaim that space. When write traffic spikes, vacuum falls behind, tables swell, and queries stall.

Supabase backed OrioleDB to replace that storage layer entirely.

“It is a drop-in replacement for the the current Postgress storage engine which is heap and it runs about 1.5 to two times faster and you don't have to change anything else,” Copplestone said. “It's just completely compatible. It completely removes the need for some of these background tasks like vacuum upload.”

Replacing a storage engine risks data corruption. Supabase subjected OrioleDB to extreme verification before publishing the Apache 2 beta. Copplestone noted: “We put it through PG regress full compatibility with Postgress antithesis. We put it through TPCC nightly benchmarks and JSON to make sure that it basically can never lose data.”

Zero-Downtime Sharding Without Code Rewrites

Scaling beyond a single machine usually forces developers into distributed databases with broken SQL compatibility. Supabase took the opposite route by hiring Vitess creator Sugu Sougoumarane to design Multigres.

Sougoumarane previously scaled MySQL at YouTube using horizontal sharding. With Multigres, he insisted on locking down compatibility before touching distribution.

Copplestone explained the design philosophy: “he said I want to focus only on compatibility first and so the way we've designed it is in such a way this groundup rewrite is specifically so that we know when we release sharding you actually won't have to change anything for your database you'll continue to run the same queries and multira will just be able to handle it zero changes to your code.”

The team tested the system under deliberate chaos. “We actually ran it again through very rigorous testing and we've done a 1,000 sequential failovers and it didn't lose any data,” Copplestone said. The result is high availability now, with transparent sharding built on top later.

What to Do With This

Audit your background worker architecture this week. If you run external workers that query your database over the public internet solely to batch text, chunk documents, or generate embeddings, calculate your network latency and egress bills. Move those long-running batch jobs to worker processes inside your database's private network to cut round-trip times and eliminate timeout failures.