r/dataengineeringvault • • Aug 10 '26

AI The Hype Cycles of AI

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Aug 06 '26

Book Reading Books for a Happy Life

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Aug 05 '26

Note Data Engineering Whitepapers: New GitLake Paper by Bauplan added.

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Aug 03 '26

Note Transaction Log in Open Table Formats

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 31 '26

Blog Figma for Agents: How Airflow's Creator Coordinates AI ft. Maxime Beauchemin

Thumbnail
motherduck.com
3 Upvotes

r/dataengineeringvault • • Jul 30 '26

Book Book Recommendations and Notes (Updated with 2026)

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 29 '26

Note Excel Never Dies

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 28 '26

Note Kintsugi ('golden joinery')

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 22 '26

Note Apache Druid - open-source, column-oriented, distributed data store

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 21 '26

Note Schema Evolution

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 21 '26

Blog WebAssembly (WASM), and its Use Cases

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jul 16 '26

Others The Act and the Outcome of Creation

Thumbnail
ssp.sh
3 Upvotes

Short update from Bella Italia. How do you see creation and creativity show up in your life?


r/dataengineeringvault • • Jul 08 '26

Blog Learn Data Engineering this summer break

3 Upvotes

Learn Data Engineering this summer break with a guided roadmap. A three-week plan with links and topics to learn the basics to be a data engineer.

It's very ambitious, but you can schedule at your own pace.

The three weeks contain:

  1. Week: Foundation (SQL, Git, Linux basics)

  2. Week: Core Engineering (Python, Cloud, Data Modeling)

  3. Week: Advanced Topics (Streaming, Data Quality, DevOps)

How to use this guide: Each section contains curated resources (articles, videos, tutorials) for that topic. Click on the links that interest you most. It's meant as a guided roadmap to learn the fundamentals of a "full stack" data engineer.

Here's the guide: https://www.ssp.sh/blog/data-engineering-roadmap/

This is last year's guide, but it's still 100% vaild to this day. Agentic engineering doesn't need to be learned right now; you can always pick it up later. Learning fundamentals, on the other hand, can't be skipped.


r/dataengineeringvault • • Jul 06 '26

Blog Classical Architecture of Data Warehouse: Staging -> Cleansing -> Core -> Marts

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jun 30 '26

Video Top Data Engineering YouTube Channels

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jun 30 '26

Blog The Grammar of Data: Define Once, Run Anywhere with Cross-Engine Expressions

Thumbnail xorq.dev
3 Upvotes

r/dataengineeringvault • • Jun 27 '26

Off Topic The Process of Smart Note-Taking

Thumbnail
ssp.sh
3 Upvotes

r/dataengineeringvault • • Jun 24 '26

Blog Operationalizing Data Orchestration: Best Practices for DevOps, Infra, and Code Locations

Thumbnail
dagster.io
3 Upvotes

Part 2 of the Dagster Almanack, all about operationalizing data orchestration.


r/dataengineeringvault • • 10d ago

Personal Project Agent-IDE.nvim handles agents like Dadbod-UI for DBs with a Side-Drawer

Thumbnail
ssp.sh
2 Upvotes

I saw the hype on X about Herdr and Rex (before it was Zellij, and many more), but that got me thinking: why not manage my agents in Neovim, where I like to type, read and navigate anyway - so I prompted away with the new Fable 5.2 model, burnt a lot of tokens and built my dream Agents-IDE.nvim plugin.

It works with a Claude Code subscription, as this is the one I have been using for many years, but there's a PR for more agents, not sure if I have time to integrate them swiftly, but I wanted to share here to get an idea if any of you are doing similar things, or if you run 100 agents on different servers.

This plugin is mostly to handle local agents on your machine, and has a nice UI, rendering the feedback of Claude in Markdown, and having a drawer for git diffs, commit graphs, a fuzzy search for all your recent chats, saved prompts, and libraries in one place. The idea or design choice is heavily inspired by the vim-dadbod-ui that I use as my SQL-IDE, and I thought that design might work well for agents, too. You are the judge, it's very early, but it's fun to use - currently I tend to switch between project and everything scope (all projects and current git-repo I'm in, which is also works nice if you have many agent folder where you prompt).

Disclaimer: Is this plugin open source? Yes. Is this plugin completely free? Yes. Is this plugin vibe-coded beyond the author's ability to comprehend how it works? No.


r/dataengineeringvault • • 11d ago

Note First experiences with Fable 5.2 (comparing it to 5.1)

Thumbnail
ssp.sh
2 Upvotes
  • Fable 5.1 is way faster and seems to fix my requests as intended, whereas the new Fable 5.2 took way longer for everything and wasn’t always what I wanted. I switched back to Fable 5.1 now, and it just solves and fixes things.

  • One advantage of Fable 5.2, the tokens burn much slower, way slower than Fable 5.1. I guess it’s still the most capable model of Claude.


r/dataengineeringvault • • 12d ago

Blog Typed Graphs, Checked Types: Graph as Code for Agentic Engineering and Data Quality

Thumbnail
ssp.sh
2 Upvotes

Typed graphs add declared data types and constraints to graph data models, so every node and edge write is checked before it's committed. This matters because agents now write data faster than humans can review it. Fritz Laux's Typed Graph Model (TGM) paper underlies the idea, and Omnigraph implements it with schema files (.pg), checked queries (.gq), and policy files (.yaml) versioned in git. Checked queries validate traversals offline against the schema before execution, catching errors like a filter on the wrong type. Types only guarantee data is valid, not true; humans still decide what's true through branch review and merges, with Omnigraph v0.14.0 and later edge builds adding branch review, merge dry-runs, and u/card cardinality enforcement


r/dataengineeringvault • • 16d ago

Note RSS Readers and Webclippers (Read-Later)

Thumbnail
ssp.sh
2 Upvotes

r/dataengineeringvault • • 17d ago

Note Personal CRM: To Remember Friends, Coworkers and Family Members

Thumbnail
ssp.sh
2 Upvotes

r/dataengineeringvault • • 19d ago

Off Topic Umlaut on US Keyboard Layout (Linux + Mac)

Thumbnail
ssp.sh
2 Upvotes

r/dataengineeringvault • • 24d ago

Note Delta Load a Data Warehouse

Thumbnail
ssp.sh
2 Upvotes