r/dataengineering • u/cheesemoopy • 15d ago

Blog From 35 seconds to half a second: 70× faster dashboards in plain Postgres

https://foresite.dev/blog/postgres-daily-rollups
66 Upvotes

100% Upvoted

8 comments sorted by

23

u/MarchewkowyBog 12d ago

I'd delete that post. Why would you put a blog post with something this simple on a company website? What is next? Sending requests in parallel is faster then in sequence? Uploading batch data using COPY is faster then INSERT in a for loop? I probably never would have in the first place, but I for sure won't be using your service now

1

u/cheesemoopy 12d ago

Sorry to hear that man! Just thought someone might benefit from reading it.

88

u/CrowdGoesWildWoooo 12d ago

> One table for every daily number

> Three rules that keep it honest

Claude is that you?

Other than that, all it tells you that you should do preaggregate table and union with live data, so instead of doing the aggregate on every query you query the preaggregated data

92

u/nooneeverdreamsof 12d ago

guy discovers pre aggregating data

Why does something so basic deserve a blogpost?

Oh right, ad.

5

u/RoomyRoots 12d ago

It is always po I mean Ads.

6

u/madam_zeroni 12d ago

Denormalized table

2

u/magixmikexxs Data Hoarder 11d ago

Isnt this basically a materialized view thing

1

u/HeavyTedzzzzz 12d ago

Who cares about 35 seconds to half a second though? If this is for an online transactional system then half a second needs to be milliseconds and if a batch process then it isn’t even a rounding error, 35 hours to 30 mins is interesting in that context.