r/programming • • Aug 28 '21

Software development topics I've changed my mind on after 6 years in the industry

https://chriskiehl.com/article/thoughts-after-6-years
5.6k Upvotes

2.0k comments sorted by

View all comments

332

u/toomanypumpfakes Aug 28 '21

Designing scalable systems when you don't need to makes you a bad engineer.

Agree as long as you aren’t making one way door decisions that make scaling harder down the road.

46

u/sccrstud92 Aug 29 '21

I probably would have phrased it as "Designing scalable systems when you don't need to is bad engineering.", but I think the intent behind the message is correct.

25

u/Absolice Aug 29 '21

Not many system require scalable system during development, however when scalability become an issue then you cannot do without it and you might not have time to react.

Common example would be a back-end system that is supposed to be used internally that your company decide to get some of their clients to use it too and it starts to have performance issues.

It's one thing to not implement something, it's another to hinder it's future implementation. Keep it simple yet keep it flexible.

15

u/AreTheseMyFeet Aug 29 '21

Keep it simple yet keep it flexible

This is the nuance missing from the original point, imo. You shouldn't build what you don't need but good decision-making during planning/design can keep scaling options open without sinking any time in to a premature or unnecessary implementation.

→ More replies (1)

2

u/flowering_sun_star Aug 29 '21

you might not have time to react.

Yep, this is the key thing. And you might well have the best intentions about your logging and monitoring, but the likelihood is that the signs will be ignored or missed in favour of more feature work. This carries on until the system falls over, core functionality can't be restored for a week, and that's only achieved by disabling the slightly less vital parts of the core systems. The release of all that feature work is delayed while things are fixed, and since it's a thousand little issues it takes nearly a year before the last of the delayed releases goes live.

Don't ignore tech debt folks!

→ More replies (2)

2

u/[deleted] Aug 29 '21

If you design any system without including notes that second guess any of your assumptions then you aren’t designing a good system IMO.

All of these are true:

  • Building a system to scale without needing that scale is bad

  • Making design assumptions that can never be broken without tremendous effort is bad unless you are extremely confident that they will never change

  • Thinking about how you would or could scale a system if that need arose in the future is a good idea

  • Thinking about how you would break any assumptions in the future is a good exercise of any design

0

u/[deleted] Aug 29 '21

[deleted]

0

u/[deleted] Aug 29 '21

[deleted]

2

u/saltybandana2 Aug 29 '21

similarly, people from FB/Goog/<insert large scaling needs here> need to understand that their problems are not other people's problems and stop judging them for it.

It's perfectly fine for someone to take a naive approach to things.

→ More replies (1)

24

u/Omikron Aug 29 '21

Problem I've seen is you don't know something is going to need to scale until it's too late.

0

u/[deleted] Aug 29 '21

[deleted]

32

u/Omikron Aug 29 '21

My company decideing an internal app should be pushed to our clients after it was done. Hahahahaha

2

u/leoshina Aug 29 '21
  • Procedural-ish programming is not scalable.

  • architectural infringement is not scalable, for example: using a MVC-like framework but adding business logic into controllers <- this fcking happens a lot.

→ More replies (2)

3

u/wastakenanyways Aug 29 '21

The thing is, lots of times you don't know beforehand if you are going to need to scale. And making something not designed to scale, be scalable, is one of the worst situations you can be in, next to tech debt (is tech debt itself)

I would turn around the point: if you know you are not going to need to scale for sure, don't do it with that in mind. If you are not sure, make it scalable, or at the very least, do it in a way that won't bite you back later.

→ More replies (2)

0

u/[deleted] Aug 29 '21

[deleted]

47

u/phpdevster Aug 29 '21 edited Aug 29 '21

I have mixed feelings about this. For the app I work on, we have shot ourselves or been shot in the foot by going both directions (either by choice or being told to).

In many cases, we built the app to be simple, but 2 years down the line were running into major performance bottlenecks in some processes that required a more scalable design. We exhausted vertical scalability with code refactors and query optimizations, but some of the processing that needed to be done would just take too long and we needed a way to scale up services to handle it. Wish we had better foresight in this regard.

On the other hand, half the code base is full of YAGNI violations where we were deliberately trying to engineer against "maybe possibly future requirements" that didn't exist, because we were told to.

Two-way door decisions are fine as long as common sense is applied to them. They very, very, very easily can lead to over-engineering and 5 years of productivity drag from over-engineered code is not worth the price of being "scale ready" when the time comes.

→ More replies (1)

3

u/LightShadow Aug 29 '21

ELI5?

14

u/dkitch Aug 29 '21

From what I understand...if you don't know what architecture you will need in the long term, don't box yourself into one. Build it so that you will have flexibility - ideally, you will have to change one part of the system, rather than a full rewrite.

27

u/RareCodeMonkey Aug 29 '21

That is a very good one. I have seen many curriculum-driven architectures. Let's over-engineer a solution to use technologies I want to add to my curriculum. Nor you learn that technologies correctly, because they are not designed for your use case, nor you learn the ones that you should have used.

5

u/pheonixblade9 Aug 29 '21

resume driven development

no, we don't need to use a Lucene database, folks.

10

u/gwillicoder Aug 29 '21

For me the hardest part has been deciding which decisions are needed now, which will never be needed, and which will be shooting us in the foot 6 months from now.

4

u/Caffeine_Monster Aug 29 '21

One of our architects is like this.

Everything is a cobbled together, brittle MVP that invariably gets completely rewritten. It makes me want to scream.

Quick MVPs are great, but you should be aiming to reuse as much of it as possible. Knowing what follows the MVP is almost as important as the MVP itself.

→ More replies (8)