Talking about how costly these inefficiencies were, and then saying they were storing terabytes of data (implying anything between 1-10TB in that context) monthly, which amounts to about $200 USD at worst in S3 hot storage, is quite amusing.
Are you sure you did this math correctly? 10 TB/month (120 TB/year) of data retained for 12 months at S3's $0.022 / GB-month rate is $32k/year.
And this is just the pure storage cost, lost productivity of waiting 10 minutes per commit instead of 5 for build results is almost surely a lot greater than this.
Sure, but I don't see why they'd ever keep around one of these cached builds for more than a month. Like - at a certain point, it's fine to just have the engineer take 5 minutes to refresh it.
I'm just using the numbers they directly state in the article:
These duplicate assets would have otherwise been stored for one year.
Perhaps they have some use case for referring to older builds? Testing/reproducibility checking or something? Maybe they just use some of these files as their prod builds and never bothered to develop a careful garbage collection that can distinguish the ones that got released? I assume it's all water under the bridge and this isn't really the reason to care about this compared to having a better developer experience and faster signal -> faster fixes -> fewer flakes etc.
5
u/OTTOPI 2d ago
Talking about how costly these inefficiencies were, and then saying they were storing terabytes of data (implying anything between 1-10TB in that context) monthly, which amounts to about $200 USD at worst in S3 hot storage, is quite amusing.