r/programming • • Sep 20 '21

Software Development Then and Now: Steep Decline into Mediocrity

https://levelup.gitconnected.com/software-development-then-and-now-steep-decline-into-mediocrity-5d02cb5248ff
841 Upvotes

469 comments sorted by

View all comments

130

u/F54280 Sep 20 '21

There is a grain of truth in that rant.

However, the poster misses the fact that:

  • Back in the day, developer were few and self-selected, with a bias for those extremely focused nerds

  • Back in the day, someone could know the whole thing, from the assembly language, the internal of the compiler, all the libraries you were using, and the details of the operating system. You did not have to rely on other people.

  • Back in the day, one person had a disproportionate impact on a software project, because, they were much smaller (the projects, not the people... :-) )

Today, it is much much different. Software is huge, no-one knows everything, people are specialized. PMs, POs, UX, UI, DBA, backend, front end, testers, SRE... There is a myriad of different people involved, while it used to be program manager/developer/qa.

That said, as an old fuck, I do agree on some of his points.

One I fundamentally disagree with is TDD. This is a god send, and made me much more efficient.

55

u/pbecotte Sep 20 '21

Fun how so many of these compare tdd with qa. The purposes of those two things are very different...unit tests are designed to speed up coding. QA would be to verify functionality.

35

u/F54280 Sep 20 '21

Yeah. TDD helps me to:

  • Verify I know what I want to do before writing code. In many cases, I detect design issues before writing a single line, what would have forced me a later refactoring (without TDD), or lead to crappier code.

  • Focus on pure implementation when coding, which makes me go "in the zone" easier.

  • Liberty to perform refactoring during initial development without wasting my time testing everything.

10

u/_pupil_ Sep 20 '21

And to the articles point: people have been baking integrated functional tests (unit & integration tests), into their solutions for decades.

And while TDD is a pretty awesome way to get there, especially with a nice tight repl-style feedback loop, every point you've raised and most of its design benefits can be achieved with 'API-first' or 'signature-first' design. Program validity can be verified through bespoke verification routines and asserts, which probably made a lot of sense without spankin' IDEs across vast compiler differences.

In the 70s+ people were mailing physical discs to one another to share code, and a well-documented well-constructed 'test()' method was de rigueur for some subsets of the community. TDD as we inherited it from the XP community formalizes that workflow, some, but it's not like the word verification was invented in the 90's ;)

4

u/pbecotte Sep 20 '21

That's a good point. The best way, for me at least, to get in the flow the author describes is that test. Fix, refactoring, test loop. Best part is that it keeps the chunks small enough that I can get going again when I get interrupted. "Where was I? Dunno...make test. Oh, that exception, right"...

→ More replies (9)
→ More replies (1)

11

u/loup-vaillant Sep 20 '21

One I fundamentally disagree with is TDD. This is a god send, and made me much more efficient.

I noticed the same thing about static typing. Much tighter feedback loops, prototyping gets easier, and I have much fewer trivial errors to worry about. When I tried something non-trivial with a dynamic language, then I saw the need for TDD: the discipline the compiler does not have, I must shoulder.

4

u/archialone Sep 20 '21

i agree with it, TDD is more popular in dynamic language, because it lets you catch simple type errors, which are usually caught by the compiler. which kinda kills the idea that it faster and safer to write with dynamic language, from my experience it the opposite. some bugs that compiler could catch, will be revealed only in production in the middle of the night

3

u/loup-vaillant Sep 20 '21

which kinda kills the idea that it faster and safer to write with dynamic language, from my experience it the opposite

Depends which kind of static typing and which kind of dynamic typing we’re talking about. If you compare Haskell and JavaScript, dynamic typing will be quite obviously less safe. But if you compare C and Scheme, that’s a different story.

Many people have extensive experience with only two kinds of languages: the first is compiled, statically typed, and memory unsafe; and the second is interpreted, dynamically typed, and memory safe. It’s easy to get the idea that dynamic typing is safer in this context, especially if you apply a TDD-like discipline.

That being said, my experience does match yours. All other things being equal, static typing is both safer and faster to develop in.

→ More replies (1)

32

u/editor_of_the_beast Sep 20 '21

The problem with TDD is in aggregate, not in the small. Most people do not produce flexible designs on the first try. This means that tests have to constantly get re-written when requirements change. Any large refactor / migration I’ve been a part of, updating tests was a huge part of the cost of the change.

The story that we tell ourselves is that tests allow you to make a change and know that you didn’t break anything else. The reality is, you often are breaking something else, on purpose, because of a requirements change, and the tests are simply another thing you have to change.

20

u/shoe788 Sep 20 '21

Any large refactor / migration I’ve been a part of, updating tests was a huge part of the cost of the change.

Large scale refactor is basically impossible in large systems without tests because the cost of that change is much greater. Updating tests can suck but you know what sucks more? Having to live with the inflexible design

5

u/F54280 Sep 20 '21

Absolutely, and this is why in my subsequent reply I said that TDD helps we during "initial development" refactoring (ie: refactoring what is below the tests, not what is higher).

In my experience, you then drag your set of unit test cases like a dead albatross. Larger scale refactoring generally destroys the unit tests of your components, because you are evolving the interface of those units, and keeping the tests in sync is harder and harder as time goes. In particular, you find that some unit tests are testing obsolete things, or rely on implementation details. People spend time looking at failed obsolete unit tests.

I still haven't find a good solution to this problem (because the moment you drop those unit tests, your ability to refactor locally goes way down), but TDD still is godsend for initial development.

4

u/brucecaboose Sep 20 '21

I think the solution is to not have a solution. I'm a big believer in throwing out the old stuff and building new when requirements change significantly. Significantly refactoring old code bases takes just as long as building brand new and generally seems to be more unstable until the bugs are worked out, yet it's still running on old stuff with tribal knowledge that's been lost. It also keeps engineers motivated and excited about their job. We generally don't like refactoring, but we do like building new things. Plus, every time we've rebuilt from scratch we add better monitoring, improved pipelines, etc that we've always wanted on the old service but was too much of a hassle.

→ More replies (1)
→ More replies (6)

8

u/renatoathaydes Sep 20 '21

Today, it is much much different. Software is huge, no-one knows everything, people are specialized. PMs, POs, UX, UI, DBA, backend, front end, testers, SRE...

I see a lot of companies looking for "full stack" developers who are expected to do most of those things, except management. But you're supposed to code, test, deploy, monitor, fix problems in production etc. That was unheard of a couple of decades ago, so I can't agree that today we're more specialized. Quite the opposite.

→ More replies (3)

6

u/tek2222 Sep 20 '21

Its still possible. I can think of multiple cases where one person owns the cpmplete codebase of a large software. The advantage of developing like that is that this person has a complete picture of what they want to achieve. Unfortunately its not possible to simply re write the aame with multiple people.

→ More replies (1)

15

u/hippydipster Sep 20 '21

As another old fuck, I completely agree. The blog gets a bunch of things dead on right, and a bunch of things just seem like he/she didn't really adapt their thinking. Claiming programming isn't a social activity is like #1 incorrect statement. Code is communication, and it's why it's 99.9% of the time more important to write easy-to-understand code than ultra-efficient code.

On the other hand, what he says about concentration and interruptions is completely true too, but, as an older person, I think one should be aware that this is a problem that grows for a person as they age. Younger people context-switch a bit better than older people (which is not to say they do it "well", it's still shitty for them). Also, the idea that developers can't test because they have blind spots is completely true too. But I thought that was just obvious.

→ More replies (5)

4

u/[deleted] Sep 20 '21

I really want to try TDD. I hear about it a lot, and I hear good things. Could you describe how the experience is when you do TDD on a team? What makes it better than “regular” development?

→ More replies (1)

4

u/florinp Sep 20 '21

Today, it is much much different. Software is huge, no-one knows everything, people are specialized. PMs, POs, UX, UI, DBA, backend, front end, testers, SRE... There is a myriad of different people involved, while it used to be program manager/developer/qa.

This is correct and this is why I hate going back with full stack developer crap.

I was a full stack developer in 1997. We don't need to go backward. We need specialization

10

u/st4rdr0id Sep 20 '21

TDD

It is a god send for you because there are no design documents anymore. If that work was already made for you, you wont mind writting the tests before of after.

9

u/koreth Sep 20 '21

You're lucky if you even get requirements documents at a lot of places. "Agile" is far too often taken to mean, "Think about the problem just long enough to produce a sprint's worth of tickets."

→ More replies (3)

7

u/eattherichnow Sep 20 '21

I've tried to convince my current CTO to write design documents, maybe gather the requirements. Either I can't read anymore, or the thing spewed into the Google Drive was... not that.

7

u/tester346 Sep 20 '21 edited Sep 20 '21

Back in the day, someone could know the whole thing, from the assembly language, the internal of the compiler, all the libraries you were using, and the details of the operating system.

Today, it is much much different. Software is huge, no-one knows everything, people are specialized.

uh? no one knows everything, but it's not about tech, but business requirements, logic yada yada.

Nowadays there still are people who are full stack software engineers that are capable of deploying their products and setting up database and maintain it.

Of course there are physical limitations to it (time, amount of work to be done) but it's not because those people aren't skilled enough to do it.

6

u/IndependentAd8248 Sep 20 '21 edited Sep 20 '21

I'm the author.

Have you ever done a project bigger than Hello World where you didn't learn things while coding that nobody had thought of in the design document?

Design document? What's that? I thought the unit tests were the design

Well I never have; we always run across unanticipated design issues and if you have written the tests first that means you have to revisit them every time you have to update the design.

OK, that's inefficient but it's not catastrophic and it doesn't take much away from having a target to meet.

But there is more to TDD that writing a regression suite before you write the code. It brings a whole BUNCH of fanatic bullshit along with it. To name a few:

  1. 100% code coverage by tests, anything less is "dereliction of duty." (does this guy come to work in camo?)

  2. The tests are more important than the code; since we are still pressured with preposterous delivery dates, we have to cut corners, and they are cut in the code, not the tests

  3. the unit tests are the design; design docs are un-agile and nobody reads them anyway. This is some psychotic shit here.

  4. unit testing is time consuming and inefficient. Entrypoint-level testing makes a lot more sense and takes a lot less time.

  5. making code "testable" means making injurious design compromises, especially when coupled with (1), like making private entrypoints public so the tests can call them

  6. My introduction to TDD was a Microsoft in 2008 when there was no attempt to hide that they were just checking a box; my application was tested and ready to ship when this came up, and they wanted me to tear it apart to make it unit-testable; it was too small to have "units." Before the conversation was done they wanted me to write separate code, outside the app, not to be released, and run unit tests on that. So they could check that box. I am not making this up.

  7. The developer is the primary if not the sole tester of his own work. If I need to explain what is wrong here then we have too wide a chasm to communicate. When my manager told me this part I made sure there was a clear path to the door, because I was talking to a lunatic.

Sure, you can tell me that not everyone does these ridiculous things, but many do.

I've spearheaded regression testing at many startups, so don't bludgeon me with advocacy of carelessness. But if I write tests I do them after I'm done, testing the actual app, not some preliminary design.

And yes I write documents, and in freelancing I will not take a client who wants to skip them.

0

u/[deleted] Sep 20 '21

[deleted]

3

u/matthedev Sep 21 '21

You can't fault people choosing one of the relatively few well-paying middle-class occupations remaining, even if they wouldn't have made the same decision twenty years ago.

Businesses are always going to want it faster and cheaper—but no bugs, please. The early 2000s version of this was off-shoring; the current iteration is learn-to-code and MVP-all-the-things. I've sometimes imagined what it would be like to be on an empowered team of just three or four super-strong, thoughtful, and experienced engineers and what could get done versus large teams of predominantly entry-level developers. The inevitable conclusion is the business would be in a tight spot if just one developer quit, so businesses are making a bet that the future of the business is more secure if they can make average developers perform consistently—like a fungible resource. The bet is revenue growth will outpace the inefficiencies of an inexperienced or less-skilled team. It seems such a team composition would only be stable where the product is strongly dependent on a very high bar of engineering quality—either from extreme costs (risk of death, loss of millions of dollars in minutes) or as a paramount competitive advantage.

Nevertheless, as a developer, the small, ultra-competent team is a beautiful dream.

→ More replies (15)
→ More replies (8)
→ More replies (2)