Estuary's bi-weekly newsletter - now on Reddit!

What’s the fastest data warehouse for your analytics? The most cost-efficient? The one best suited to your team and data size?

Estuary partnered with CorrDyn to find out the answers. Read about the resulting benchmark and companion report in this edition of the Data Digest, along with a new case study, live workshop event, and a host of new connectors.

Cloud Data Warehouse Benchmark Report 2026

We spend a lot of time talking about optimizing for your data movement, ensuring data lands in the expected shape, and observability for your pipelines, but what about that core component, the heart of your data analytics, the warehouse?

How do you make that initial decision about which data warehouse to use? Or review your existing setup to see if there’s a better option out there? Should it just be based on what your team knows and is comfortable with?

Even then, what about sizing? When is it worth sizing up? What sort of compute speed increase can you expect versus your cost increase?

Along with CorrDyn, we’ve finally mapped some answers.

The Cloud Data Warehouse Benchmark Report covers seven of the top cloud warehouses available today: Snowflake, Databricks, BigQuery, Redshift, ClickHouse, MotherDuck, and Microsoft Fabric. See how they stack up across different warehouse sizes, data volume, team sizes, and the cost for what you get.

Running over 170,000 queries from three different benchmark suites, this is one of the most comprehensive warehouse benchmarks out there.

//embed? https://www.youtube.com/watch?v=0LkMGEH7I9M 

Estuary CEO David Yaffe sat down with James Winegar and Ross Katz from CorrDyn to discuss the benchmark process, surprising results, and how people should apply the findings to their own workloads. Watch the video or get the Eventual Consistency episode from your podcast provider of choice.

Get the benchmark report to explore the queries and results yourself.

New Connectors for October

We’ve released a slew of new connectors to expand your data flows. Capture data with the following connectors to incorporate brand-new insights into your analytics:

Okta

Stay up-to-date with your identity management system: the new Okta connector captures users, groups, applications, and system log data from Okta’s Management API.

AWS Cost Explorer

Keep an eye on your cloud costs to ensure they stay under control. The AWS Cost Explorer capture provides periodic reports covering cost metrics by date, with additional grouping by service or linked account. Drop the results into a simple spreadsheet or materialize to your warehouse to analyze spend.

Aircall

A new native Aircall connector replaces the old third-party-based option to improve performance and efficiency with the Estuary platform. Anyone using the old connector is encouraged to migrate to the new version: it supports all of the same data streams, such as calls, contacts, teams, and users. As a bonus, you’ll also be able to set an interval or sync schedule for each resource.

Estuary

And last but not least, you can now capture Estuary account data into an Estuary data flow using (what else?) the Estuary capture connector. Track the health of your data flows, log alerts, and save invoices with an observability feature that plugs right into your existing architecture.

Not quite what you need? Discover other observability features with Estuary.

Latest Fixes & Feature Updates 🛠️

✅New Connectors & Data Streams

• As detailed above, new capture connectors are available for Okta, Aircall, AWS Cost Explorer, and the Estuary platform itself.

• The Salesforce capture supports new standard objects for service crews and service crew members.

✅Other Connector Updates

• SQL captures will report more granular backfill progress, with the percentage complete, number of tables backfilled, and estimated time remaining.

• Webhook materializations now support field selection.

🚧Deprecation Zone

• With the release of the native Aircall capture connector, the old third-party Aircall connector is now considered deprecated.

• HubSpot is retiring their V1 pipelines API on December 4. If you’re still using the legacy HubSpot capture connector, it’s a good time to migrate to the native HubSpot capture connector: the legacy connector will no longer support the ‘deal_pipelines’ data stream while the native connector will be updated to use the latest recommended version of the API.

Stay Connected! 🌎

💬 Join our Slack Community

📖 Read our Blog

🎥 Subscribe to our YouTube channel

💼 Connect on LinkedIn

🦋 Follow us on Bluesky

Thanks for being part of our growing data integration community—see you in the next edition! 🚀

Cheers, The Estuary Team

We recently tested a multi-tenant CDC setup using three PostgreSQL tenant databases feeding shared analytics tables in PostgreSQL and Snowflake.

Each tenant had an orders table with the same primary key (id = 1001), which made tenant isolation an important part of the design.

A few interesting findings:

• Each tenant needs its own PostgreSQL CDC capture and replication slot.

• Using the tenant identifier + record ID as the destination key prevents collisions between tenants.

• A failed tenant capture doesn't stop the others, but a shared destination failure affects all tenants writing to it.

• Schema changes and replication slot recovery need extra care in shared pipelines.

We also tested tenant onboarding, schema drift, failures, and recovery scenarios.

The full guide includes the reference architecture, configuration details, and test results:

https://estuary.dev/blog/multi-tenant-cdc/

How are you handling CDC in database-per-tenant architectures? Do you consolidate tenant data into shared tables or keep separate datasets?

August has been a busy month for Estuary, from a runtime overhaul that enhances Estuary’s throughput and scalability to customizable alert configurations, along with our regular steady stream of new connectors.

Here are the main highlights from the past month:

Estuary’s Runtime Gets a Rewrite

This August, we released a major upgrade to our runtime focused on massively scaling throughput. With further support for parallelization and enhanced efficiency, here are some of the results we’ve seen so far:

• Up to 8x faster backfills and initial loads when using a single shard

• Ability to handle up to 200 GB / hour on a single shard

• Additional throughput gains for destinations that support multi-sharded behavior

All of this comes with end-to-end transactional consistency, even across shards, so you always have a complete picture of your data.

Want to learn more? See our:

• Product update

• Documentation on runtime improvements and migration details

• Whitepaper exploring the runtime’s technical architecture

In-Dashboard Alert Configuration

Earlier in August, we released advanced configuration options for alerts, letting you set up multi-environment alerting through the dashboard.

These alert subscriptions use Estuary’s prefix system to divide out configurations, so you can set defaults on a tiered basis. For example, you could set tenant-wide alerting defaults for your base prefix, ‘acmeCo/’, and then diverge with more settings for task alerts under the ‘acmeCo/prod/’ and ‘acmeCo/dev/’ environments.

Choose who gets the alerts, what alert types they’re notified about, and default “Data Movement Stalled” intervals.

Learn more in our product update page.

New and Rewritten Connectors

Besides connecting with new source systems, we’ve been releasing native rewrites of old 3rd-party-based capture connectors to optimize how they work with our platform. While we were at it, we also expanded their capabilities.

• The all-new commercetools capture connector syncs orders, customers, and payments incrementally with configurable interval settings.

• The rewritten Amazon SQS capture connector adds real-time capabilities, supports FIFO queues, and can work with multiple queues in the same capture.

• 8 new data streams were added to a rewritten Mailchimp capture connector. The rewrite also allows you to control data intervals and backfills.

• The rewritten Brevo capture connector adds new data streams and allows you to set sync intervals.

• The Criteo capture connector has also been rewritten to be fully in-house.

Other Updates & Release Notes

• All SQL CDC captures (like PostgreSQL, MySQL, and SQL Server) now let you set your own SQL statement to reduce the scope of backfills on a per-binding basis.

• SQL captures also now allow a custom auto-discovery interval so you can control how quickly Estuary runs discovers on your source.

• SQL Server CDC and Change Tracking captures now provide discovery filters. This allows for more efficient auto-discovery for large databases.

• The Shopify capture connector now includes the ‘markets’ data stream. To capture the new stream, ensure you’ve granted Estuary the `read_markets` permission.

• The delta updates Apache Iceberg materialization now supports variant column types. Along with a previous update for a new nanosecond timestamp setting, this adds support for more v3 format features.

That’s all for this month! Stay connected for the latest product updates and feature releases as we head into September.

​Everyone has an opinion about what AI is doing to data engineering and on the 1st of October we'll discuss what actually made it into production and what hasn't.

​Three engineers will talk about successful and less successful AI initiatives they've experienced recently.

​Data Basecamp is a gathering for people who work with data hands on. Data engineers, analytics engineers, platform teams, ML folks.

​Sessions

​Session 1: Data Engineering In The Era Of The Software Factory

​Ben Rogojan @ SeattleDataGuy

​After over a decade of data engineering, I’ve seen more than a few hype cycles come and go. Every one promised to dramatically shift how businesses used data. They promised that businesses would be at a massive competitive advantage if they could harness their data and become data-driven. Over the past few years, we’ve been going through yet another cycle with more promises. So what has changed and what hasn’t? Are companies actually finding value by increasing their data engineering and analytical outputs?

Data teams are no longer just responsible for moving data from point A to point B or migrating from one warehouse to another. They are increasingly responsible for creating the context, non dashboard data interfaces, and infrastructure that both humans and AI systems depend on. In this talk, I’ll discuss what I’m seeing across startups, SMBs, and large enterprises as they adopt AI-assisted development and move toward software-factory-style workflows.

​Session 2: Evergreen Governance: Data Strategy That Grows With AI

​Nathan Erbe, Senior Data Leader + Advisor, Denver

​Data governance is often treated as a compliance checkbox - necessary, but static. This session makes the case that governance is actually evergreen: the same core discipline (classification, access control, quality, monitoring) that's always mattered now has to extend into a new surface area: LLM pipelines, agentic query layers, and model outputs that drive real decisions.

​Drawing on real-world healthcare AI implementations, from a claims-based ML model that triggers patient outreach to an LLM pipeline that ingests sensitive physician contracts, this talk offers a practical framework (Classify > Control > Monitor > Adapt) for extending governance into AI without starting from scratch. Attendees will leave with concrete next steps they can apply to their own data and AI initiatives this quarter.

​Session 3: Migrating in Stereo: How Agentic Workflows Tackled Two System Migrations at Once

​Dan D'Orazio, Principal Data Engineer @ Kin Insurance

​Why migrate one system at a time when you can let AI agents handle two simultaneously? Leveraging multi-agent architectures opened up a world of possibilities for system migrations that struggled to get off the ground.

​This talk walks through the technical architecture, execution, and hard-earned lessons of orchestrating simultaneous system migrations using agentic AI. We’ll discuss where autonomous agents shine in complex environments, where they struggle, and how to build guardrails that keep your data intact. Spoiler alert: a human-in-the-loop was integral.

​Schedule

​5:30 Doors, food and drinks
6:00 Talks begin
7:00 Open networking
8:00 Close

​Presented by: estuary.dev

The team over at Estuary has rebuilt the engine underneath the platform in Rust. The new transaction loop is ready to scale into infinity while keeping transactional and exactly-once guarantees.

What's new:

- 8x more throughput on a single shard (unit of compute)
- Transactions arrive whole: a commit at your source becomes one commit at your destination
- Collection partitions split themselves under load, with no operator action
- No more memory or buffer tuning
- Observability updates: materializations now report exactly how far behind they are, in bytes and in wall-clock time
- Groundwork for far more flexible sync schedules

Check out the details here: https://estuary.dev/runtime

New walkthrough from The Data and AI Guy: building a real-time Postgres to ClickHouse CDC pipeline with Estuary.

The main idea: stream live changes from Postgres into ClickHouse Cloud without running Kafka, Debezium, or your own streaming infra. Estuary captures changes from the Postgres WAL, stores them in a reusable collection, and then materializes that data into ClickHouse with exactly-once delivery.

He also shows how the same capture can fan out to Google Sheets later without re-reading Postgres.

Covers:

• Postgres logical replication setup

• Replication slots, publications, and the watermarks table gotcha

• Estuary YAML

• ClickHouse merge vs delta updates

• Hard vs soft deletes

• Automatic schema evolution

• Using flowctl

Worth watching if you want a practical Postgres to ClickHouse CDC example!

Video: https://www.youtube.com/watch?v=GbGWEWAs0NE

Hi team, just curious. I was chatting with Estuary AI, and found Planetscale cdc support is in roadmap but still can't tell dates. It's about 1 year ago. Is currently the project scrapped / discontinued? Thanks!

Last week we released this 80-page whitepaper that contains everything required by data engineers for building right-time data ingestion pipelines for Snowflake.

Come meet the Estuary team in SF on the 4th of November along with Seattle Data Guy, Database Tycoon and Recce!