Key Takeaways

  • Point-to-point ETL pipelines built on tools like Fivetran and dbt were designed for static human queries, not dynamic agent workloads.
  • The volume of incoming operational data is exploding beyond human maintenance capacity, making manual schema mapping unworkable.
  • Databricks and modern catalogs handle data organization after the fact, but agents require real-time autonomous data ingestion at the source.
  • Building custom pipelines for every SaaS application slows product development and blocks teams from deploying autonomous workflows.

The Death of Manual Pipeline Engineering

For a decade, enterprise data infrastructure relied on a predictable loop. Engineers wired Fivetran to pull records, ran dbt models to transform schemas, and dumped tables into Snowflake or BigQuery so analysts could build dashboards. That architecture worked when queries were written by humans asking predetermined business questions once a week.

Agentic systems destroy that pattern. When autonomous agents operate across customer support tickets, code repositories, and sales records, they do not wait for a data engineer to write a fresh pipeline. They generate and request unstructured data continuously.

During the conversation on No Priors, Elad Gil asked how much of the existing data stack survives: “How much of the existing data infrastructure do you think survives all this? So you know there's all the ETL data engineering infrastructure that people have been building and deploying over the last decade. Did that stick around?”

The answer from Eon co-founder Ofir Ehrlich is clear: point-to-point pipelines cannot scale to this volume. Ehrlich noted: “The amount of data being ingested is absolutely insane especially comparing to earlier. We see trends continuously both us and other companies that we're seeing in data. You see data is growing out of proportions.”

Post-Hoc Catalogs Cannot Keep Up

Traditional enterprise infrastructure attempts to solve data sprawl by indexing files after they land in storage. Ehrlich pointed out the limitation of relying solely on warehouse catalogs: “You've seen companies like Databricks, one of the most incredible companies on the planet in my opinion and looking at I have more and more data coming in. I don't necessarily know where it is. I'll help you catalog the data and make use of that, but it's an after effect.”

Cataloging data as an afterthought fails when software agents need immediate, contextual data access to make operational decisions. If an agent has to wait for batch processing or manual tagging, its execution loop stalls. Instead of static pipelines, infrastructure must shift toward autonomous ingestion that organizes unstructured context on arrival.

As Gonen Stein explained, the objective is to remove the engineering bottleneck entirely: “The goal is really to enable this culture of builders and the culture of agents with the ability to automatically help them understand what's there automatically help them ingest the data without having to build manual pipelines for each and every application.”

When systems ingest and clean data automatically, product teams can build without waiting on infrastructure tickets. Ehrlich summarized the upside: “Today you understand that you can collect if you're able to smartly collect and clean all your data and make sure you store in efficient manner and if you can activate that efficiently, you can let a team go wild with all the data that they have.”

What to Do With This

Audit your roadmap for any planned custom data connectors or point-to-point ETL pipelines this week. Freeze projects where software engineers are manually writing sync scripts between individual SaaS tools and internal databases. Replace those point integrations with autonomous ingestion layers or direct agent data interfaces that bypass rigid batch schemas.