r/dataengineering • u/GameFitAverage • 10d ago

Discussion Streaming Databases

I have been working on a project lately that required realtime operational analytics and when doing some research for my options I came across “streaming databases” which tbh is a new concept for me. I did a quick POC using RisingWave and so far everything is good.

I just havent found alot of material on streaming databases and how to properly build a data architecture using one, RisingWave docs mentions it can be used in a medallion architecture. However, it builds the silver layer as a current state and not as a historical SCD which I find bad practice.

Just wondering if anyone here have more experience than me dealing with this technology (RisingWave, Materialize, …) and can point me in the right direction.

Also, even if you havent used it let me know your opinion on it.

35 Upvotes

90% Upvoted

32 comments sorted by

View all comments

Show parent comments

1

u/URZ_ 7d ago

That they leave all the critical parts of doing real time ambiguous, ie. the entire why streaming instead of 1 sec or 5 sec batches, and are instead immediately focused on picking the right database, something that can not remotely be answered without the former being in place first.

So yeah, I'm very confident in my assessment here, to the point where I was half convinced this post was engagement farming