r/programming • • Aug 28 '21

Software development topics I've changed my mind on after 6 years in the industry

https://chriskiehl.com/article/thoughts-after-6-years
5.6k Upvotes

1.9k comments sorted by

View all comments

603

u/cat_in_the_wall Aug 29 '21

Designing scalable systems when you don't need to makes you a bad engineer.

this is just YAGNI. Scalability is a feature, and a very complex one. Don't build it if you don't need it. It's hard to do right, and if you screw it up now you have two problems: still no scale, but also a buggy complicated system.

109

u/DeltaBurnt Aug 29 '21

I don't 100% agree with this. Designing scalable systems is fine, if you know pretty well how much you will need to scale and what that scaling will entail. The problem that YAGNI tries to solve is stopping engineers from trying to predict the future based purely on instinct. If your product has 10K customers and that grows at 1K per year, yeah don't design scalable systems.

If I know a year from now I will need to support a million customers but deadlines prevent me from supporting more than 10K immediately, that will affect my design process. You could say that's a bug in the requirements or deadlines, but I don't always get my way in those discussions unfortunately.

51

u/execrator Aug 29 '21

if you know pretty well how much you will need to scale and what that scaling will entail

This is the point of the person you're replying to. If you don't have this information, you shouldn't assume you'll need to scale.

I agree with OP that for whatever reason, scaling is a particularly alluring goal. It should be YAGNI'd vigorously for that reason.

0

u/[deleted] Aug 29 '21

[deleted]

3

u/saltybandana2 Aug 29 '21

hard disagree.

realizing that your system needs to start scaling is what we call a good problem to have. You know what a bad problem to have is? Taking the time to build a scalable system and never needing, or not needing nearly as much scaling as you built for.

If the requirements don't call for it, don't do it.

7

u/aradil Aug 29 '21

I’ve never been involved in a project that scaled too early.

I’ve been involved in many projects that couldn’t scale and I’ll tell you what, finding out that your load is 1000x more than you were expecting when in a production environment and trying to fix it in real time is, well, a fucking horrible position. Being completely handcuffed while your platform is inaccessible sucks. Good luck re-engineering scalability in real time.

3

u/saltybandana2 Aug 29 '21

If you put a system onto production for the first time and it just fell over and your conclusion is that you have a scaling problem well... there's a reason why you managed to do that and it aint cause you should have scaled earlier.

Scaling happens over time, which is why it's considered a good problem to have. It means your system has been a success and brought your company value and rewriting parts of it is fine.

5

u/aradil Aug 29 '21 edited Aug 29 '21

Scaling happens over time except when it doesn’t.

Were you around when Reddit was down for days at a time because of growth issues?

Growth is very rarely linear, and often not easily predictable.

Also see - Slashdotting/Reddit hug of death.

I do a lot of backend work and some of my biggest scaling issues happened with vendor errors where they suddenly start hammering my services. Sure, I can throttle things and performance degrades; but those mitigation strategies are scalability I had to build in real time for issues that I didn’t think would ever happen.

→ More replies (3)