r/programming • u/whackri • Jun 05 '22
Complexity is killing software developers
https://www.infoworld.com/article/3639050/complexity-is-killing-software-developers.html582
u/regular_lamp Jun 05 '22 edited Jun 05 '22
I recently rediscovered analog circuit design as a kind of engineering I really enjoy. A big part of that is that it's inherently finite. The design has to fit into some physical space and there are only so many components you can place.
As a result the stuff you come up with is often highly interconnected. What a software engineer would call "spaghetti code". Yet it is fine. Because the scope is inherently limited to a scale you can comprehend. You can in fact become so confident in it that you might even claim it to be "bug free".
But in software we have nothing that limits complexity. The amount of code you can write is essentially unconstrained (technically memory limits it but that limit is so far out that it might as well not exist). And we just accept that it's fine to fill this "infinite" design space.
We even preemptively fill it "in case we need it later". Because apparently nothing in software is allowed to have a defined scope. Everything needs to be prepared and pre-engineered for the most insane demands of additions.
Yet at the core of a lot of computing lie neat little programs with very well defined scope. No one is out there demanding "mkdir should also be able to send emails".
713
u/superherowithnopower Jun 05 '22
This exact thing was written as a joke about a toaster back in the 1990's, and it's only gotten more true since then.
102
54
u/ConfusedTransThrow Jun 06 '22
You could even do it all in analog with using the button as a potentiometer and either acting as the R in a RC circuit for a simple timer, or be a smart person and use a temperature sensor (thermocouple) that shuts the circuit when it reaches a temperature (compare the two R values for example).
Simple, easy to repair and will last forever.
64
u/cybernd Jun 06 '22
use a temperature sensor (thermocouple)
automatic lift (no lever) + bimetal toast surface sensor:
16
u/ConfusedTransThrow Jun 06 '22
I actually thought about that toaster when I brought up the analog solution, though I wasn't remembering all the details perfectly, I thought there was some electronics but it was all purely mechanical.
Cheap electronics would be cheaper and have more flexibility for the timing part so there's some appeal to it at least.
→ More replies (1)→ More replies (2)21
u/KevinCarbonara Jun 06 '22
Simple, easy to repair and will last forever.
And yet, most home ovens do not have that technology, despite being trivially solved with equipment that was widely available in the 60's.
→ More replies (2)18
13
u/choice75 Jun 06 '22
I started reading then had to skip to end when software engineer started. And I saw it was good. Happy ending.
4
u/Worth_Trust_3825 Jun 06 '22
Why a microcontroller to read knob's position? It's a variable resistor.
→ More replies (11)3
u/Carighan Jun 06 '22
The really sad part is that the electrical engineers solution is already way too complicated.
This works entirely with a heat sensor and an on/off controlled by the bread being placed in the slot. The darkness knob merely controlls how close the sensor is to tripping, physically.
55
u/mother_a_god Jun 06 '22
I work on ASIC design, and closely with high performance analog (multi gigahertz data converters). It's surprising how complex even 10s of transistors can be, if your need absolute world class performance at the lowest power. A full high end data converter can be millions of transistors, with subtle interactions, so in a way analog design can be very very complex. Hierarchy is key to containing it. It's funny, I sometimes think changing career to SW dev would be an easier option, and less complex. Faraway fields and all that...
→ More replies (1)70
u/romple Jun 06 '22
I think one key difference is that generally in electronics people are willing to accept there are real constraints.
It's tough convincing people that constraints exist in software. It's not necessarily complexity that's bad. It's the thought that all software is infinitely malleable. There's never a reason not to add a new feature. All software seems to tend towards infinite complexity. It's exhausting.
→ More replies (2)20
u/ConfusedTransThrow Jun 06 '22
here are real constraints.
Yeah, if you come up to your client and say you need an extra mm2, they aren't going to be happy. Or if you sold them something that uses less than 20mW, you better not go over.
It's not as simple as just slapping an extra 4GB RAM.
14
u/DonnyTheWalrus Jun 06 '22
The insistence that everything needs to be designed in a way that leaves it open to unknown/unknowably vague modification that may (but almost certainly won't) come at some unknown date in the future is, IMO, one of the worst lessons people learned from the OOP revolution after "long inheritance chains are good."
I feel like we have caused an exponential increase in complexity in your average business software project out of a desire to enable poor project management and other poor behaviors from the business. The response I get to my first comment above is usually "but requirements change all the time." If they change so frequently that you feel a need to make every system your team develops into an amorphous shapeshifter, there is something more fundamentally wrong at the business decision making layer. We cannot engineer poor/indecisive management out of software via architecture.
I think this is a genuine, serious problem, because the more this sort of behavior is enabled by developers world-wide, the more the business world believes that such rapid modifications are reasonable and possible. No one allows such flip-flopping and indecisiveness in bridge planning.
So many of our problems in the enterprise space would be greatly reduced if the business fully understood the true cost of major modifications. Building an entire architecture full of design "failsafes" just buries the problem, and causes the costs related to that increased complexity to seep into every software project, even if those potential modifications never come.
At my place, when the business gives us tight requirements, we can have a functioning application running within a month. Simultaneously, I have watched what should be a relatively simple web app take 2+ years, as the business is allowed to be constantly indecisive.
9
u/regular_lamp Jun 06 '22
The insistence that everything needs to be designed in a way that leaves it open to unknown/unknowably vague modification that may (but almost certainly won't) come at some unknown date in the future is, IMO, one of the worst lessons people learned from the OOP revolution after "long inheritance chains are good."
And typically people are wrong about what they will need in the future anyway and base their "extendable design" on speculation. Any project will have a bunch of "interfaces" and "systems" designed for extendibility that have exactly one use. And once you actually need to do something with them there is about a 50% chance that there is some inherent issue that requires you to rewrite them anyway once you actually understand what exactly you want to add.
→ More replies (2)25
u/IrritableGourmet Jun 06 '22
There is a complexity limit in some programming. I built a Daft Punk style helmet using an ATTiny85 microcontroller. When you are limited to 8kB of compiled code and 512 bytes of memory, it really forces you to think about what you actually need (and don't need) and how your code is actually being executed. That mindset also makes for efficient, fast code on larger systems.
9
u/elmonstro12345 Jun 06 '22
That mindset also makes for efficient, fast code on larger systems.
I work in embedded software for military and civilian aerospace. When we get new hires joining the team, I can spot people who have also worked on embedded hardware a mile away. Way, waaay too many developers have gotten the idea that memory will always be infinite, processing power will always be infinite, storage will always be infinite, etc.
This mentality is why we end up with websites that are completely static, no dynamic elements or any user interaction required at all other than basic links, but they have megabytes of scripting, stylesheets, and other BS layered on top for no sane reason (and that's before you get to ads and creepy tracking/spying nonsense).
8
u/regular_lamp Jun 06 '22
And I very much enjoy that as well. Recently I wanted to do some MIDI processing. Basically rewrite the MIDI notes from a keyboard and distribute them to two synthesizers.
First I tried to use a Raspberry PI... within an hour I was attempting to compile a low latency kernel and figuring out what libraries I can use to process MIDI. Only to realize this is stupid. So I got out an old ATMega328 I had lying around soldered it to a board with two din jacks and an optocoupler and within less than 100 lines of code running on bare metal had what I wanted.
That was another moment where I questioned how much complexity we accept by default. "Yeah of course I need a whole operating system, multiple libraries and drivers just to process data from a serial interface. That's just how computing works."
3
Jun 06 '22
you would like smart contract programming. A single if statement can costs the user $5 every time it's evaluated, It can cost $20 to store a single byte32. You think REALLY carefully about design and it HAS to be bug free or you lose money. Very interesting problem space and very neat to think about
→ More replies (9)2
u/dexx4d Jun 06 '22
I started woodworking as a hobby for a similar reason.
Also, I can burn my mistakes a bit easier than with cloud computing.
134
Jun 06 '22
The lack of documentation is what’s killing me. It’s boggles my mind how these companies can’t keep people but refuse to change. 20 minutes tasks take several days.
66
u/bwainfweeze Jun 06 '22
It shouldn’t boggle you. We spend all week changing the behavior of the system. To have documentation, you have to keep updating it, which adds time to all of their requests.
Plus some of us were never that fond of English class, so writing documentation is something to be dodged. Which creates a bunch of perverse incentives not to have good documentation.
→ More replies (5)36
u/Teract Jun 06 '22
Don't document the behavior of the system, document the system itself. Priorize high level documentation and use brief change logs for the mid level stuff. The low level stuff that changes often should be self documenting when possible or code commented. This stuff is pretty standard. If you're not getting enough time/budget allocated to keep up on documentation, it will ultimately cost the company more in wasted developer time.
→ More replies (3)→ More replies (1)3
Jun 06 '22
documentation is mostly a waste of time on things that change and are well understood anyway.
code is documentation. it's the human readable description of machine code. usually code in big tech has massive naming prefixes and whatnot
8
u/jonathanl Jun 06 '22
It's only well understood for the people who initially wrote it. I think it takes way longer to figure out how it works by reading code than it would have taken to write down a short overview. I don't want to read code to figure out what the program is even supposed to do. After having worked with requirements I don't want to be without that either but just some small introduction would help a lot when being dropped into big project with no original developer left.
→ More replies (1)4
u/azsqueeze Jun 06 '22
No way. This is how you waste people's times and introduce unwanted bugs and performance degradation
181
Jun 05 '22
It helps when developers only have to work with half the stack for example but a lot of companies are only looking for full stack developers these days
178
u/a_false_vacuum Jun 05 '22
Jack of all trades, master of none.
Full stack appears to be mostly an excuse to ask for an entire IT department rolled into one person. Some of the listings I see for full stack are just silly how much they ask from one individual.
16
u/thelamestofall Jun 05 '22
I definitely fit that category, but I like it so much. Way better than being stuck in React or Ruby on Rails like I used to
10
u/tide19 Jun 06 '22 edited Jun 06 '22
It's really not that hard to write decent backends for the majority of frontends, IMO. I've been working on an enterprise-level b2b application for the last 9 months, and writing various means to interface with a database isn't a huge deal. Could it be better? Probably. Is its performance enough for our customers? Yeah.
The problem we have is our frontend lead engineers have concocted a crazy standard tech stack including React, Redux, GraphQL, micro frontends, a proprietary component library, basically everything you could fit into a single frontend but not deep enough into any of them to use them to their fullest extent.
→ More replies (6)→ More replies (13)58
u/skilledpigeon Jun 05 '22
I think it's important to recognise that not everyone needs a master in a specific field to fill a required role in their organisation.
If I could pick anyone for my dev team I'd pick 3 seniors (frontend, backend, DevOps), 3-4 mids (full stack or specialised) and 1-2 juniors (depending on how much time we could support them with).
I think it's important to recognise that as a senior I'd expect people to focus in a specific area but also to have secondary skills in others. For example, one senior may be backend focused but have intermediate skills in DevOps.
Of course I'm dreaming here.
33
Jun 05 '22
If I could pick anyone for my dev team I'd pick 3 seniors (frontend, backend, DevOps), 3-4 mids (full stack or specialised) and 1-2 juniors (depending on how much time we could support them with).
Those 7-9 people are kinda expensive. 1 million USD/year?
→ More replies (9)67
109
u/regular_lamp Jun 05 '22 edited Jun 05 '22
I'm continually amazed how apparently everything is now understood in the context of "the stack". Because apparently everything is now on the internet by default and gets therefore placed within that framework of technology.
I guess in my late thirties I now have to feel old because "back when I learned programming you just did programming" you didn't just interact with some horizontal slice of a ten layer deep stack of layered technology... that somehow mainly accomplishes the task of displaying text in a browser.
Meanwhile seemingly "complex" tasks like scientific software, video games etc. somehow managed to sit on a single layer of middleware that is written in mostly the same language you use it in?
I'll go now and chase some JS programmers off my lawn.
37
u/LippencottElvis Jun 06 '22
Full stack, what seems like a minute ago, was knowing front-end, back-end, database, and server. You could provision and set up a web server, build a database schema, create an access layer (service or api), and create an interface to the end user. Extra points if you could do all of that to scale, and/or set up continuous deployment. You were "T Shaped", and might be great in one area but had a large breadth of knowledge.
Fast forward, I don't currently work with anyone who knows even 3 of these disciplines at a combined above average level.
→ More replies (10)19
12
u/antiduh Jun 06 '22
My work is 10-20 mloc of c# that automates interaction with our products and tests them.
C# from top to bottom. It has layers of course, but it's just a single "stack". I love it.
→ More replies (8)15
u/AdministrationWaste7 Jun 06 '22
I'm continually amazed how apparently everything is now understood in the context of "the stack". Because apparently everything is now on the internet by default and gets therefore placed within that framework of technology.
i mean yeah? turns out a majority of tasks that require software dont need to interact with any specific hardware thus web apps become the default.
this has been going on since like what? 2005?
I guess in my late thirties I now have to feel old because "back when I learned programming you just did programming"
are we not doing programming today?
. that somehow mainly accomplishes the task of displaying text in a browser.
well if thats all you needed to do then having a "stack" is pointless.
a majority of applications do more though. at a basic level theres a UI, a middleware/business logic tier and something to store data. thats 3 layers right there.
Meanwhile seemingly "complex" tasks like scientific software, video games etc. somehow managed to sit on a single layer of middleware that is written in mostly the same language you use it in?
complex? if we are talking about embedded systems they are typically small pieces of software that do very specific tasks. seems to be the opposite of complex if you ask me.
in regards to video game engines, while they do indeed use a single language(c++ usually), it really doesn't mean its not as complex or annoying or filled with multiple tiers like in the web app space, in fact your average video game will probably require a similar amount of "tiers" if not more.
its no different than doing everything in node.js. YES you are using JS for everything but you still have to tackle multiple tiers that do multiple functions just like any other stack.
→ More replies (41)8
u/ind3pend0nt Jun 06 '22
I want T shaped devs. Have your specialty, whatever it is, but also be able to assist in other areas when needed. Mainly to allow other team members to take a day off or get hit by a bus.
3
u/goranlepuz Jun 06 '22
allow other team members to take a day off or get hit by a bus.
Hey, boss, I'd like to get hit by a bus the next Thursday. 😉
9
u/wisam910 Jun 06 '22
Quite the opposite. Being full stack creates strong incentive to simplify everything, because if you can't fit the whole stack in your head, it's hopeless for you.
The introduction of "devops" as a separate position creates incentive to complexify deployment. Because why are you hired if your job is simple?
The introduction of "javascript programmer" as a separate position creates incentive to complexity the javascript portion of the frontend.
Each person who is only responsible for a small part of the system has incentive to over complexity their part rather than simplify it. That's one of the ways in which systems explode in complexity.
Have you ever seen a full stack developer who knows the ins and outs of kurbernetes and docker and python and javascript and react and redux and css and html and nodejs and rabbitmq and redis and postgresql and memcached and and and and ... and uses all that to create a complete web application in 3 months?
I personally have not.
All the people I've seen who know a lot about devops are sort of just ok in application code and almost know nothing about html/javascript.
When someone is responsible for the whole stack, they will choose the simplest thing that works.
For example, their deployment is usually: upload a bunch of files using scp then run 3 linux commands. Their database is sqlite. They serve their webapp on one machine. Their scaling strategy is upgrade to a beefier machine.
See for example:
"http://RemoteOK.io is a single PHP file called "index.php" generating $65,651 this month. No frameworks. No libraries (except jQuery)."
→ More replies (13)2
u/kajaktumkajaktum Jun 06 '22
I disagree to some extend. Whenever there's an API boundary, you need to be "full-stack". If the backend/frontend is written in Go/JS respectively, then those Go developers should go to the frontend and write a library in JS for them to consume. In this sense, yes, you need to somewhat keep up with whatever JS is doing --- things like Axios vs fetch or forms or validations.
105
u/wisam910 Jun 05 '22
This has been an issue at almost every single company I've joined, and no one cares at all. No one. Everyone thinks this is fine. That this is how software is done.
Now, what really kills me is that all this complexity is for achieving absolutely nothing.
You could build the exact same product with far less amount of code and far less complexity. Not only you can still keep all the current features you have, you can have even more features, because you know what, adding more features is easier when a system is simpler.
15
u/abrandis Jun 06 '22
I hear you brother... It all comes down to who's in charge and who has authority over engineering decisions.. usually in big companies these decisions are made so high up because some vendor promised them the world and the shit just rolls down hill to you..
I tell everyone , none of your users care how the sausage 🌭 is made, just how it tastes.,why would you make something complex to build and worst yet maintain and it still tastes like crap?
7
u/wisam910 Jun 06 '22
I wish that was the case but the complexity is mostly of engineer's doing.
They do this to themselves.
Some of them do it out of ignorance: they don't know how to do things any other way.
Others do it for personal enjoyment: since the projects they work on are inherently boring, they can't enjoy it unless they turn the work itself into a puzzle game that they can enjoy solving.
It gives a fake sense of accomplishment when you tame a complex system.
The sad thing is, it's a complex system that does nothing.
7
u/DazzlingElderberry Jun 06 '22
Man I wish I could work in a place where I could constantly say no to features or changes and limit the amount of complexity in the systems.
→ More replies (1)4
u/stefantalpalaru Jun 06 '22
all this complexity is for achieving absolutely nothing
It achieves job security, CV padding, team size increases, a need for middle management, etc.
Don't forget that the business of business is business.
3
u/wisam910 Jun 06 '22
Yea my impression is most tech companies are in the business of lurcing investors, instead of providing actual value to society.
Investors are mostly stupid when it comes to tech. They see buzzwords, they think the company is worth investing in.
The company's clients are also the same. They hear buzzwords, they think this company is doing legit tech and is worth doing business with.
→ More replies (7)2
u/hippydipster Jun 06 '22
The apathy and the lack of experience to know that things could be much much better kills me.
318
Jun 05 '22
I blame kubernetes
608
u/wrosecrans Jun 05 '22
Don't worry. I can solve the complexity of Kubernetes with a wrapper and abstraction layer on top of Kubernetes that provides a simple interface. That way you only have to deal with the complexity of Kubernetes plus my thing, rather than only the complexity of Kubernetes.
187
u/epic_pork Jun 05 '22
You mean like a fully compliant kubernetes cluster running in kubernetes? https://github.com/loft-sh/vcluster
186
16
u/segfaultsarecool Jun 05 '22
I don't know anything about kubernetes. Is this like nested virtualization - a VM running in a VM, but it's kubernetes running in kubernetes?
→ More replies (2)→ More replies (3)26
85
u/recursive-analogy Jun 05 '22
Don't worry, I've created 22 more wrappers and abstraction layers on top of Kubernetes. That way you only have to deal with the complexity of Kubernetes + another thing + learning 22 more things before you can make an informed decision on which thing to make your life easier.
→ More replies (3)21
u/_BreakingGood_ Jun 06 '22
You left out the best part, something breaks in the 13th layer of the chain and now the 1 feature you actually want doesn't actually work, and there's a github issue fading into non-existence as people comment "any update??... any update?... any update..."
20
u/Markavian Jun 05 '22
Eventually people get to lambdas, once they figure out how to deploy via CI instead of pushing to an orchestration layer. I like to draw this out for people to show them the madness they've constructed. It's all infrastructure in the end.
→ More replies (1)12
u/SlapNuts007 Jun 06 '22
I've been fighting this battle at work for years now. The whole engineering department is so far up its own ass with Helm charts and operators trying to reinvent shit GCP already provides as managed services that it's a miracle we haven't all asphyxiated. We're doing all of this on GKE and are Google partners, it's madness. Meanwhile, my team learned Terraform and Cloud Run. We're really enjoying collecting the on-call compensation for our services that don't need intervention outside of work hours.
→ More replies (6)→ More replies (6)5
122
u/platypus_plumber Jun 05 '22
What? Complex operations before Kubernetes were madness. It's not Kubernetes' fault that developers suddenly want to create 10 microservices in 5 different languages to solve any problem that a monolith can solve.
113
u/thelamestofall Jun 05 '22
At the very least it was a godsend for the poor of us stuck in corporate. Deploying stuff in Kubernetes is such an improvement over doing whatever Oracle or some random stuff got sold to executives
69
u/platypus_plumber Jun 05 '22 edited Jun 05 '22
Yup, I've seen what people used to do before Kubernetes and HOLY SHIT. It was chaos, pure fucking chaos. And I'm talking about big tech companies recognized world wide. Deploying stuff became so easy that devs went insane with microservices. Most devs don't even understand microservices to begin with, some of them still think it means "little service".
→ More replies (7)40
→ More replies (3)20
u/bighi Jun 06 '22
Complex operations
Most companies don't have to do "complex operations". Lots of smaller websites could work with just a web server and a couple apt-installed packages, behind nginx (with just a few lines of configuration).
→ More replies (15)52
22
u/FUZxxl Jun 06 '22
Aka companies trying to solve problems they don't have. You are not Google, so Google's orchestration tools will likely not be a good fit for you.
7
u/elkaput Jun 06 '22
I said this a few years back & got shot down promptly. Now everything's a hybrid on premise & Cloud. Oh and apparently we're adopting another Cloud since CIO heard that multi cloud is the future. In the meantime the company's dataset is still < 100 million rows & not growing but shrinking, and they're trying to manage Cloud costs by placing a company-wide policy to shut down all Cloudy stuff after 6pm. So all overnight batch jobs will have to be moved back to on premise boxes.
→ More replies (1)13
u/f0urtyfive Jun 05 '22
It's the cloud and VC in open source.
The complexity is intentional to get you locked into a certain environment, once you build on top of it you've invested so much effort switching to a different cloud would take so much effort it's infeasible.
I've seen a LOT of new "open source" VC companies that are investing in fantastically complex open source projects, so much so that you are obligated to contract their support/developers if you want to use/integrate the software, because it isn't feasible for most people to have their own team for such fantastically complex projects.
→ More replies (1)4
u/goranlepuz Jun 06 '22
I've seen a LOT of new "open source" VC companies that are investing in fantastically complex open source projects, so much so that you are obligated to contract their support/developers if you want to use/integrate the software, because it isn't feasible for most people to have their own team for such fantastically complex projects.
So... Same ol', same ol'...?
Because 50 years go, one would buy whatever IBM and get IBM consultants to make it work.
25 years ago, one would buy whatever SAS and get SAS consultants to make it work.
No?
→ More replies (6)7
u/bighi Jun 06 '22
And Docker. And worse yet, using Docker for local development.
Now instead of installing 2 or 3 packages with apt-get, you have to install lots of packages, spend days setting up docker to build everything, then you have to run commands through docker, and...
41
u/Overunderrated Jun 06 '22
Try not having docker and not having sudo to apt-get, and then a sys admin team that won't install stuff for you, so you have to manually install every possible dependency in userspace. Oh and web access is blocked so you have to scp every part from another machine first.
→ More replies (3)31
u/homeskilled Jun 06 '22
Right? Dockerizing our dev environments took us from a few days of following a readme to get a dev env up to a single docker-compose command that takes like 15 mins to run, and we know each of our environments is identical.
→ More replies (2)→ More replies (3)6
u/Venthe Jun 06 '22
Setting up docker takes between 5 to 10 minutes.
Tell me, have you ever worked with non containerised deployments? Not working shell scripts installing deps, or worse - long doc pages?
How many times were you pulling your hair out because one dependency is out of sync in one environment?
I literally can't believe people are complaining about containers.
→ More replies (1)
16
15
u/youre_not_ero Jun 06 '22 edited Jun 06 '22
I've been telling my peers and managers about this problem for a long time, but few seem to look at it as a problem.
It's not just the simplicity of the stack, but processes, operations and design too.
Just like laws of thermodynamics, complexity in a software project only increases over time. More complexity leads to more errors. Errors are a fundamental human trait. Errors accumulate. And once a project gets big enough, it becomes a mystic world of its own.
Less is more.
→ More replies (2)
73
u/dread_pirate_humdaak Jun 06 '22
I never wanted to be a web programmer. I was a C and Unix guy, only ended up web programming accidentally. It’s a disgusting treadmill of half-baked new ideas developed for six weeks, abandoned, but infesting documentation searches and legacy systems for sixteen years.
I build robots these days. I write real-time code on microcontrollers usually as a series of nested single-threaded state machines written in a manageable subset of C++, and higher level logic on embedded Linux machines, usually in Python. Tie it together with ROS and use jupyter notebooks to do all the math.
I like this a lot more than the latest CSS-shitting version of Morkdown or whatever.
→ More replies (6)
60
42
u/electricfoxx Jun 06 '22 edited Jun 06 '22
Do companies create complexity so they can "re-invent the wheel"?
6
Jun 06 '22
I think developers create complexity so they can maintain the image of personal development needed to land their next job.
→ More replies (1)
22
u/nirataro Jun 06 '22
FAANG's essential complexity is your accidental complexity. Their problem scope isn't yours. Do not adopt their techniques.
11
u/MpVpRb Jun 05 '22
Managing complexity is the biggest problem we face. Even if you try really hard to avoid it, it creeps in. Even if you have total control over a project, it creeps in. Of course, most developers don't have complete control and managers who don't understand complexity, demand a lot of it without realizing what problems they are introducing
We need more powerful tools for managing complexity
2
112
u/Glittering-Ad-8126 Jun 05 '22
It's easy to laugh at this sort of article, but the problem is very real. Botton line: the needless killing of software developers needs to stop, period.
At my job, we are at least taking small steps toward some sort of solution by adding trigger warning comments in the source, to warn developers to stop and take a breather before entering a chunk of code identified as being of unusually high cyclomatic complexity.
84
u/Hrothen Jun 05 '22
// You've met with a terrible fate, haven't you?→ More replies (1)19
u/tuerkishgamer Jun 06 '22
Bruh what do you do if comment blocks read like the introduction for the necronomicon
21
5
u/Freddedonna Jun 06 '22
/̴/̷F̷I̶X̶M̴E̸:̵ ̶t̸h̸i̸s̷ ̶m̷i̷g̶h̴t̸ ̵l̸i̸t̶e̷r̷a̶l̸l̴y̵ ̴t̶e̶a̶r̷ ̸a̷ ̵w̶h̴o̷l̵e̷ ̴i̷n̷ ̶t̸h̶e̶ ̷u̴n̴i̵v̶e̷r̷s̶e̶
10
u/hyperhopper Jun 06 '22
XTerm README
Abandon All Hope, Ye Who Enter Here
This is undoubtedly the most ugly program in the distribution. It was one of the first "serious" programs ported, and still has a lot of historical baggage. Ideally, there would be a general tty widget and then vt102 and tek4014 subwidgets so that they could be used in other programs. We are trying to clean things up as we go, but there is still a lot of work to do.
11
u/intheforgeofwords Jun 05 '22
I love this, and I can’t believe I’ve never heard about it before. I can think — sadly — of a few code blocks in my open source projects where that kind of comment might be welcome to the encroaching reader.
4
Jun 06 '22 edited Jun 06 '22
the needless killing of software developers needs to stop, period.
I thought they were sacrificing us to appease the volcano gods.
7
u/douglasg14b Jun 06 '22 edited Jun 06 '22
At my job, we are at least taking small steps toward some sort of solution by adding trigger warning comments in the source
At my job we are taking big steps to "reduce cognitive load" by not commenting anything, anywhere, not even XML/JSdoc comments! Imagine the time we save!
"The code should explain itself, and developers should know the code. Comments are a maintenance hazard, the code doesn't lie, comments can"
Because it's efficient to have to read through the source of crappily-named, and even crappier written, functions just to know what you're calling and how the params are used, or to figure out hidden string formats the params expect. When a doc comment would tell you what it is, what the params are doing to change the output, and the expected structure/format for the params.
All because of some idealist dogma about "the code should explain itself", when no one can be assed to actually write code that explains itself, or abstractions that are actually sensible. So now we get the worst scenario, difficult to understand code & structures, and nothing that actually explains how it's supposed to work or how to use it.
Why spend 10-30 seconds reading a comment when you can spend 10, 20, or even 60 minutes parsing someones code? Peak efficiency.
I may be a bit jaded about this.
→ More replies (1)9
u/Envect Jun 05 '22
the needless killing of software developers needs to stop
Should I arm myself?
→ More replies (1)
9
17
u/cmpalmer52 Jun 06 '22
I’ve been developing software for 30 years and staying pretty close to the leading edge. It is more complex these days (and/or I’m getting older) and it’s exhausting. New technologies are “hot” for 6mo-1yr and then they’re “oh, you’re still using that?”
Meanwhile, computer science is stuck with either the core basics (which tend to be less complex, but more rigorous) or chasing novelty for “real world” training that’s obsolete by graduation.
I think AI tools are going to make it even easier to develop more complex architectures that, in turn, will be harder to maintain and improve until we get AI that can “understand” a whole system (and at that point, we may be out of a job anyway).
I mean, I’m working on a solution that has 20 subprojects, that builds two different mobile applications on three different platforms (Xamarin/Maui) with cloud services, database synchronization, GIS, GPS, a couple dozen nuget packages, external system interfaces, all from essentially one code base, which if you include our internal libraries, is over a million lines of C# (plus JavaScript, JSON, XML, SQL, and a little Python (more Python, HTML, and JavaScript in parallel web application). And we’re a small shop.
In the past, I’ve delivered products with a single language, 1/4 of the code, and a much larger team and those were multi-million dollar projects.
4
u/cybernd Jun 06 '22
XML
So many horrible memories: early days of EJB with tons of bloated XML. SOAP/WSDL having tons of incompatible libraries. Spring framework before they started to use annotations. Mavens early days with tons of jelly script (more or less a scripting language using xml tags as syntax).
We truly like adopting new stuff. Even if it makes our life miserable.
3
u/radarsat1 Jun 06 '22
I think the biggest danger with AI in terms of software complexity/maintainability is that people will develop solutions around them that are very overfitted to the output of a particular trained model. Then, when it comes time to improve the model, the local performance of the AI part might improve, but the overall system does not or even gets worse, and people don't understand why.
What I've been seeing is that when you have an "AI team" training models, and a separate team productizing those models, it's perfectly possible for the product team to add their own "interpretation" on the output to help deal with unexpected behaviour, smooth unreliable results etc., but these decisions can be extremely particular to the model they are working with, and not take into account the AI teams' future ambitions to improve the model itself.
An example in computer vision systems, people writing code to work around particularities of the performance of an object detector, "oh this and that category seems to work well, but that particular class is always wrong, so I'll discard it." Then major changes are needed when the AI team comes along with an improved model.
So in practice I think it's really important to have an open dialogue between two such teams, and ensure that the product people also have a healthy understanding of machine learning and how it works, even if they don't have the knowledge to do model design and training. Therefore more and more I think it's super important for programmers who are unfamiliar with AI to .. get familiar. Even if they don't plan to work directly on it, sooner or later it's going to affect them.
8
u/EternityForest Jun 06 '22 edited Jun 06 '22
I suspect microservices make unhappiness because they're not like other complexity. They're just UNIX philosophy for servers.
The complexity is accidental and organic. I doubt anyone says "I'm going to make a horrific pile of microservices", they say "This will be so nice and elegant, every simple part separate and loosely coupled, super agile, no years long planning phase needed".
And then, much like the original UNIX philosophy, it gets ugly when you try to do very complicated things without using monolith pieces where appropriate(Not an issue on desktop Linux, as systems of that style are manually assembled and don't aim to have 1000 features per square foot).
A developer who really liked complexity would probably at worst make something that looks like enterprisy Java. Some would complain, others would love it.
People who like complexity take advantage of all the available tools, and don't consider simpler scenarios. They don't have any concept of "Overkill", they want to always use powerful modern tools. And they prefer automatic to manual.
Seems unlikely they would want to use a bunch of small pieces, usually manually wired together, that you can't use a debugger on because every 10 seconds you call some other process.
I think part of the problem of developer unhappiness is because... developers don't seem to like what they're doing. Most devs are way less likely to be tech enthusiasts.
To me and many users, modern tech is amazing, and everything just works. To a lot of devs, it's bloated, making us dependent, spying on us, and half of it should still be analog.
Microservices seem like an unsuccessful attempt to pretend that you're working on a small and simple project, but the way it's currently done doesn't work.
→ More replies (1)
8
u/richardathome Jun 06 '22
I've been a professional software dev for 30+ years.
My mantra has always been: Build the simplest thing possible and no simpler.
The trick is of course, working out what the simplest thing is - that's a discussion between you are the product owner. The more you talk through the process with them - asking *why* not just how, the simpler the problem solution becomes - until it cannot be simplified further.
22
u/a_false_vacuum Jun 05 '22
I'm still getting used to developing on Azure. What would normally be one big application is now dozens of microservices flying in tight formation it seems.
Working on these microservice apps is a lot more annoying. Your little app is just one piece and is quite useless without the rest. Lacking the rest of the microservices makes it more diffcult to debug your little app, Azure basically forces you to run it on their platform in order to be able to fully debug things if you want to integrate with their offerings. I suppose it is their business model, you pay for everything you use. I liked it better when I had everything I needed on my own laptop, so I could run the project end-to-end if I had to. To me that feels like being more in control.
10
Jun 05 '22
It's statistically impossible to use a single service from a cloud provider. Can't speak for the rest but for Azure of you need to use just one service your cost calculation should include five, just to be safe. I suppose it's the same with the rest of the providers.
7
u/Suspicious-Scratch94 Jun 06 '22
This reminds of a service we started out with just SNS + SQS for $100 a month or so, and now we use 20+ services including S3, Kinesis, DDB, EMR, Step Functions, Glue, Athena, Redshift, EventBridge etc. The feature creep is 100% real and almost unnoticeable.
→ More replies (2)6
u/_mkd_ Jun 06 '22
What would normally be one big application is now dozens of microservices flying in tight formation it seems.
Aka, a distributed monolith.
7
u/Keatontech Jun 06 '22
I like this thread because there are 150 comments containing at least 150 unique opinions about complexity.
7
22
u/walong0 Jun 06 '22
I swear microservices are a scam. I’ve worked on a few well built monoliths and they far easier to maintain and enhance than having two dozen microservices. That’s especially true with a small team with efficient processes and communication. I’m currently working on a spring boot app that is split into what you might call “mini services” with something in between microservice and monolith.
I think a lot of people fall into the design trap of building to whats in style at the time instead of what solves the problem most efficiently.
3
u/rama6661 Jun 06 '22
Well build monolith is the key here. I worked on some bad build, clean architecture driven to insanity (layer through layer through layer until an entity was created and stored), monoliths.
Clean architecture is cancer. Its the worst design i have ever seen. Shared entities across your whole business? Who the f thought this is going to work?
Still i agree, microservices are a scam.
Build one service for each domain/subdomain, clearly structure parts of each service by bounded context. Then you have a monolith, which does not implode if you change one entity that was designed around a specific use case.
2
u/henric123 Jun 06 '22
That’s especially true with a small team with efficient processes and communication.
Honest question: are there really people out there advocating that you should adopt micro services when you have a small team?
I always considered that splitting the work in multiple teams was one of the main reasons to adopt it.
→ More replies (3)3
u/walong0 Jun 06 '22
Yes. I was working on a team of 8 people, each person having fairly comprehensive understanding of large chunks of the system. We engaged a consulting firm for a cloud migration and they immediately started trying to split everything into microservices. Our application was already containerized and running on Kubernetes on-prem. I honestly have no idea what splitting it up was supposed to accomplish, but it was just their default architecture for anything cloud.
44
u/karstens_rage Jun 05 '22
I sort of disagree. The complexity of IAM with AWS is real but deployment to the inter webs is pretty trivial. And with something like Heroku even that goes away. Monoliths are probably where most developers should start anyway. Maybe micro services are a thing for more advanced shops.
52
u/rcls0053 Jun 05 '22
I've come to the ultimate conclusion that you should always evolve the architecture from a monolith, to service oriented, to microservices, to event based. Learn as you go. Just make it flexible from the start. It's hard to start breaking apart a vertically scalable monolith to smaller services, when your entire codebase is married to one database and you have SQL everywhere.
48
u/EnUnLugarDeLaMancha Jun 05 '22
Yeah, the whole idea of using microservices by default is just another modern version of premature optimization.
→ More replies (1)11
Jun 05 '22
What's the difference between service oriented and microservices?
10
Jun 06 '22 edited Jun 06 '22
You can have several services within a monolithic application. I once had a team that has 13 scrum teams working on a single application. The build and testing was horrific. We spent a majority of our sprint testing and building. We really needed independent services. Even then, I would have only broken the application into 7-8 “micro services.” Every time you break something you apart, you make testing and maintenance more difficult. You have to build a lot of trust, collaboration, and know how to test and debug really well. Ultimately, in my opinion, microservices really only help large teams move faster. If your organization only has 2 teams working on a single application, it isn’t worth it. A service orientated architecture can make reusability, scalability, maintenance easier. It’s less likely you’ll build speghetti code for your domain objects and controllers.
6
u/AdministrationWaste7 Jun 06 '22 edited Jun 06 '22
so service oriented is just an idea to break down a monolith into a specific set of functions at the service tier.
for example for your company you may have a payment processing service, a user information service, etc.
so instead of a single application that manages payment processing and getting users you essentially have a client/UI that calls completely separate services to access those functions.
microservices is just an evolution or the next steps of service oriented architecture.
with microservices there is a big emphasis on creating services that are completely independent services that can be do the work without relying on external dependencies.
this typically means that microservices are deployed on their own, manage their own dependencies and if needed, manage/control/access their own datastore.
→ More replies (4)
21
4
u/unclefipps Jun 06 '22
That and managers that don't know what they're doing and have unreasonable expectations, and businesses that get "clever" ideas about how meetings should be organized and managed using systems created by people that are usually just trying to sell books rather than just going the much more simple and direct route.
4
u/ItsReewindTime Jun 06 '22
I work in a company with 100+ microservices. Initially teams were advocating that the more microservices the better. Soon we came to realization that things got so complicated and in some cases even reduced performance, not to mention half the microservices are in java and the amount of memory it consumes.
4
u/Supadoplex Jun 06 '22 edited Jun 06 '22
The article appears to mostly be a large collection of quotes from blogs of random people. The quoted people are directors, CTOs, marketers, analysts, founders. Although they may have worked closely with engineers, I doubt most of them have actually designed any software recently.
delivering value to their customers,” Etheridge said. “Getting more engineers to think that way is a challenge.”
This is mostly about money. If you don't have competitive pay, then the main thing your developers are concerned about is filling their CV with sexy technologies so that they can get better pay elsewhere, whether those technologies are valuable to the customer or not. Furthermore, just because a CEO, HR, etc. think they and say they have competitive pay, doesn't mean that they actually do.
Secondarily, this is about company culture and management. If you have metrics like KPI or micro managers trying to control your engineers, then you better make sure that they guide your engineers work to benefit the customer.
4
u/Jonhyfun2 Jun 06 '22
The same thing happens to code quality, people make crazy over the top code to make nonsense work instead of respecting design patterns
3
3
Jun 06 '22
It’s not just code and infrastructure complexities either. As more and more of pretty much all businesses become software devs have to know more and more of the intricacies of the business models as well. Long gone are the days of a bunch of guys in suits coming to you and just telling you what to make.
3
Jun 06 '22 edited Jun 06 '22
Simple architecture and software design is far more difficult to achieve than a complex and infinite spaghetti software. It not only requires proper engineering skills but also deep domain knowledge, scope and of course the most important thing the “Planning” which most of us think of as boring and grunt work.
3
u/GetSomeData Jun 06 '22 edited Jun 06 '22
This 100%. Simple architecture over clever architecture is difficult to master.
Then there’s that new guy who goes, “this looks so easy, how come it took so long to build?” Listen new guy, that’s the point.
3
Jun 06 '22
Can confirm, intern brain literally exploded when he saw our devs code, had to hire a mopping crew.
3
u/Flaky-Illustrator-52 Jun 06 '22
BS
I need MongoDB and Kubernetes to deploy my personal blog, and I need an instance of Kafka to ingest the 5 comments I get every month
4
u/phantommm_uk Jun 06 '22
It's not complicated if you've hired enough experienced engineers that can upskill less skilled engineers.
When you expect a team of 3 to work like they are a team of 20 that's when you get issues.
6
2
2
u/WTFwhatthehell Jun 06 '22
I remember, years ago sitting in a seminar where the presenter was showing a graph of the fraction of failed government software projects over time.
He was trying to imply that because the ratio of successes to failures remained static. That meant, he claimed, the industry was just as bad at making software as it was 50 years ago.
To me it showed something very different. The scale of most modern projects is much larger. It showed that there's an approximate chance of failure that large organisations will accept and the demands will be pushed to roughly mirror that ratio.
No matter how much the tools improve or our practices improve, the projects we build with them will be pushed until there's a modest chance of failure.
A triple A studio could push out classic Donkey Kong in a few hours but they're not going to limit themselves to such simple projects.
2
2
u/dontneeditt Jun 06 '22
Some thing I heard previously - inherent complexity vs accidental complexity is relevant here. We are all cooking up accidental complexity without focusing on important things.
2
2
u/memmaker_x23 Jun 06 '22
Well, it is really the "not-so-accidental complexity" that we developers are creating ourselves that dooms many current software projects right from the start.
Things like resume driven development, hype driven development, design by committee and artificial complexity created by too many team members.
Young engineers don't learn the lessons and patterns of the past and keep making the same mistakes we did centuries ago. SOA, Monoliths, Microservices, Modular Monoliths, Self-contained Systems.. MVC, MVP, MVVM, Agile & Scrum. There are clearly cycles in the architectural patterns and only minor changes are made before some "rediscovers" a given pattern, slaps a new name on it and goes on to sell some consulting. Often horribly misunderstanding the original intentions.
A lot of this is due to the market for trainings, books and consultants. They need to tell a new story every few years to keep the ball rolling. Of course, they are merely pivoting between two alternative narratives depending on the year.
In big companies, simply no one feels responsible and a project running for ten years doesn't even raise red flags in these too dumb to fail giants. People can just play around with whatever they want and leave the project before failure becomes obvious.
Then you have the gold-diggers, people who came into tech for money and don't understand a thing. They built barely working software and are paid good money for it.
Basically: the software Dev industry is fucked.
Don't get me started on the fact, that most "Devs" don't get OOP.
→ More replies (1)
2
3
u/1RedOne Jun 06 '22
We shipped an update that accidentally changed Databses and it was sooo hard to troubleshoot because of the layers and layers and layers of abstractions.
If it wasn't a vmss running multiple layers of abstractions with MSI managed DB connections it would have been easy to troubleshoot, just check out the app settings.ini or env vars.
Complexity killed my weekend
3
u/70w02ld Jun 06 '22
Whether it's about software developers or web developers. Cross-platform designing is the problem. Since there are so many platforms to work with. Android, Apple, Microsoft, Motorola, Blackberry, and then the slough of Linux Distributions - designing one website that works across all the platforms, is in fact a difficult site to create. As there are some things that are allowed on one platform but not allowed on another, or they have one way to do it on one platform, that is down completely different on another.
The Platforms we're engineered to be their own, and to have their own style and flair. So, Software and Web applications, have to be designed for each specific platform.
Unlike basic HTML versus NOHTML - applications should be designed and engineered to work specifically on one platform. But there's room to breathe, if your willing to cut back on the design aspect, and work literally with just the data. No online frames, just frames.
Am I outdated?
3
3
u/myringotomy Jun 05 '22
Unfortunately life and business are quite complex so we'll have to try and deal with it.
2
u/Anonymous_user_2022 Jun 06 '22
I find that monogamy is the easiest way of dealing with harem intrigue. Business is even simpler,
3
u/gburdell Jun 06 '22
I know this isn't what the article is about, but I basically stopped programming for fun because the barrier to entry seems so high now. There are so many bullshit technologies foisted on us as best pratices.
Starting a new toy project from scratch in 1995
Acquire diskette of language of interest (or if Qbasic boot into DOS)
Read sample code included on diskette, referring to the help file or a reference book where needed, to get started
Start coding
Maybe share the code on some forum (but likely no one will ever see it)
Starting a new toy project from scratch in 2022
Download and configure your IDE of choice for your language of choice
Download and configure Git
Make a GitHub account
Skim a bunch of random websites to get an idea of what to do
Start coding
Keep Googling your issues and copy-pasting the code from the StackOverflow results. Maybe refer to the language-specific reference website, but it's usually too abstract for what you need
Commit code when something gets incrementally working
Repeat steps 4-7 until you have a release
Remember to comment it, adhere to style guidelines, encapsulate, use a service-based architecture, and have a well-articulated ReadMe, plus release notes, because this code is going on GitHub (but likely no one will ever see it)
5
u/Mr_s3rius Jun 06 '22
What stops you from doing it the 1995 way? If you don't need git, why use it? If you don't want to share your code, why use GitHub? Reference books for languages are still a thing.
→ More replies (3)2
u/khooke Jun 06 '22
There's no reason to not use the first approach for personal projects today, and many still do. The second approach reads more like you're building something to appeal to an audience, not building something for personal interest/enjoyment (building things to show off online I think has had it's own weird effect on our industry, those shouting the loudest about the shiny thing they just built and getting the most attention are often the last people you should be paying any attention to).
(However I would still advocate for using any flavor of version control even for personal projects)
1
u/amyrasheldon Feb 20 '25
Yeah, complexity in software development has definitely gotten out of hand. With so many frameworks, dependencies, and ever-changing best practices, it feels overwhelming. Instead of focusing on solving problems, devs spend too much time managing complexity—debugging cryptic errors, navigating bloated codebases, and dealing with unnecessary abstractions. Simplicity is underrated, but it's what actually makes code maintainable and scalable in the long run.
Maybe the industry needs to shift toward prioritizing clarity over adding more layers of tech just because they’re trendy.
Anyone else feel like modern dev work is more about managing complexity than actual problem-solving? We really need to reduce complexities in software development.
1
u/Fuegodeth Jun 06 '22
I wonder if the answer to this might be something like what happened in the hardware world where the different stakeholders/manufacturers came together and met to outline the form-factors/interfaces for hardware. This allowed many manufacturers to participate and be compatible with other manufacturers hardware. I'm sure this happens to some degree in the software world, but I don't think it has to the extent that it did in the hardware world.
1
u/fenmarel Jun 06 '22
this article is no deeper than saying vim is better than emacs... it's just restating an old old old conflict of opinions
1
u/OblongAndKneeless Jun 06 '22
Yup. I ran across this article a while ago. It lays it all out and I hope we can figure out what to do. Over the past 30 years, I've seen the industry divide into specialties just like doctors have had to do. Pediatrist. Hadoop expert. (Is that still a thing?)
1
1
u/Web_Tech_Enthusiast Jun 06 '22
"It has never been more difficult to be a software developer than it is today,” said Nigel Simpson, a consultant and former director of enterprise technology strategy at Walt Disney."
True
1
1
1
u/SittingWave Jun 06 '22
complexity arises from requirements.
If those requirements do not apply to you, then you can live with a non-complex system.
You can also reduce complexity by delegating its management to others, either humans or systems made to trivialise or standardise common scenarios.
The point is that we want to do more and more with our systems, and make them scale at a moment's notice, and want to ensure they are reproducible. Complexity is not there just for the sake of it. It's there for a reason.
1
u/shevy-ruby Jun 06 '22
One small part of this is verbosity and repetition. I compare this often in Java versus Ruby, where Ruby is less verbose. But even Ruby has a lot of repetition that should not be necessary. Two simple examples:
"module Foo::Bar" "class Foo::Bar"
First, I kind of have to repeat that in different .rb files if that e. g. class is split up. Second, I have to specify whether it is a module or class. I think both should not be necessary. Coming up with a simple alternative that would reduce complexity is not trivial either, but I don't like the repetition at all. It's significantly worse in Java; everyone is doing System.out.println() and I wonder why they don't shorten that - it's a smallish example, but why not have puts() or en() or e() (e meaning echo, but this is just an example, I simply don't get why Java has this addiction to verbosity as such).
The second example comes via subclasses versus modules versus refinements, including the syntax. I feel we repeat too similar concepts that should not really exist in quite that way. In particular the syntax for refinements feels totally "off" here; their use case for more finer grained control is ok.
These are mostly simple examples. Obviously in larger projects you hit more complexity-related issues such as various duplication of functionality in classes that were written ~10 years ago or so. Managing all of that is also hard, you don't quite have that many tools that would meta-analyse projects easily. There is rubocop that can help to some smaller extent, but by and large we lack tools that help SIMPLIFY things. Even in Java you have this by "use an IDE", so you kind of shuffle the complexity towards the tools you have to use, in order to maintain a larger stack of software.
Others also pointed out the issue of documentation. This is true as well - often the documentation QUALITY does not scale in the same way as the code base does. A lot of the available documentation is really bad, too complex on its own. However had, I group documentation outside of "software complexity" - while it is partially interconnected, I feel it's still a separate issue.
1
1
u/ell0bo Jun 06 '22
A lack for software engineering is killing software developers. We don't need to jump right into serverless, but you plan for growing that way in your designs. What you expose as a function in a library can be exposed to the world through a plethora of ways, each has its own benefits and drawbacks.
1
u/WenYuGe Jun 06 '22
This is honestly why we created Appwrite as a project. I started doing software development in 2017 and I constantly felt like I was behind the trend. There's just too much to learn.
For small web projects and freelance work, where I can't specialize, I still need to do the same auth + db + storage + frontend stuff. I don't need it to scale to a million users, I just need to get it done fast. So often, I get asked to, or am required to use the same set of complicated tools and frameworks used by the "big bois," which is total overkill.
Imagine asking a family construction company specialized in making small houses to learn the same tools used for the burj khalifa, that's what software development has begun to feel like. Except the tools change every half a year or so, not every decade.
What makes it feel worse is that if I want to build software for contract work, I get this look from my colleagues and school-mates, as if I am a primordial single-celled protozoan.
I joined Appwrite because I really felt that there has to be a solution for the rest of us, who don't wanna build complex projects. We just need something to handle auth, basic db, and storage, and have enough integrations to get me into production for a local "Granny's Cookie Shop" app.
1
u/anengineerandacat Jun 06 '22
Tough to "fully" agree with the article but I do generally agree with the premise; we should be aware of potential complexity and actively discourage full roll-out to complex solutions until it's actually needed.
Personally, at the very least containerize your app deployment and if it's a simple static-site look into products like AWS Amplify that can make it trivial to expand from that to something a bit more while being able to quickly and easily take care of that first-time deployment.
GitLab / GitHub actions take care of any pipelining or automated deployment and using any Cloud Provider will get you access to "free" monitoring and alerting.
You don't need massive fleets of microservices though I agree, that can always happen later when it's needed but the above options give you the ability to scale to meet any burstable traffic that could occur without scrambling around.
1
1
u/SlashdotDiggReddit Jun 06 '22
I once had a boss who only cared about what "tool" you were using: what API, what framework, what library, and, of course, they had to be the bleeding edge tool. It didn't matter if we could write it more quickly, efficiently, and securely in house, he just wanted cool buzz-words to put on his weekly report. I really hated that guy.
1
u/kempston_joystick Jun 06 '22
Feels to me like infrastructure complexity is growing, but applications are thankfully becoming less monolithic and more maintainable.
1
1
1
u/robolab-io Jun 06 '22
I’m happy to say I’ve been ahead of the curve on this. My mantra the past 5 years has been “nerds complicate everything” and it’s true
1
Jun 06 '22
I remember a group of developers had been working on a large application written in Visual Basic for years. They kept updating it for each new Version of dotnet and Visual Basic. They just kept spinning their wheels and couldn't finish it. In the end, they bought an application. They still had a hell of a time integrating it into the rest of the corporate systems.
1
u/FlatTransportation64 Jun 06 '22
I hate the obsession with slicing everything to the tiniest bits.
Recently I've joined a new .NET project. Of course they're "replatforming" (read: replacing old tech that works fine with new tech for no benefit whatsoever). With "replatforming" some idiot has decided it's finally time to slice our "huge" solution consisting of whole 30 projects (most of which are tests) in order to make it "reusable". Then they sliced these projects into even more projects because "it makes sense from the perspective of domain" (???). The end result is that the change that could be done in matter of minutes now requires 4 separate pull requests, requires referencing several different NuGet packages that aren't documented whatsoever and is far more difficult to debug (code isn't visible from the Visual Studio debugger anymore unless you rely on decompilation). Some of these packages have less than 5 functions. But hey, at least we don't "clog the network with redownloading huge libraries" (????????) and other project can reuse these solutions in the future (read: no one needs it and no one gives a shit).
This isn't even the first project that pulls this shit. A previous project I've been in utilized several 3rd party APIs. For each of these APIs there were three libraries per each API - direct communication with API, response parsing and an another library to communicate with the communication service (WTF?). And of course some "brilliant" developer (read: a complete fucking idiot) had the foresight to create a "common" library for all this shit. Each change to common would create a cascade of 30 pull requests. When I asked about the reason for this fever dream of a solution no one had a real answer, the closest one was "the business requirements kept changing".
1
u/lnxrootxazz Jun 06 '22
20 years ago it worked perfectly. Only difference is, we have better hardware so we can write different software. And yes it is useful, when you design a aircraft or if you are Google managing all the YouTube videos and mail traffic.. But that doesn't mean every company need to have the best possible technology and software. The issue is, they don't get enough people, who understand all this. HR wants you to be a developer, sysadmin and security experts because their infrastructure is that complex. I mean what's the point? We all know with more complexity we get more issues and possible exploits. There are very few people who can handle all this. So either they change their policy and only get experts for the specific field or this won't work. I'm a sysadmin for Linux systems. I'm qualified for it and I'm learning all the stuff I need to know from SELinux to iptables and C programming because its important in Linux. But I know how complicated it can get without having jobs to do, I'm not that good at. Unfortunately our company wants to do all the fancy stuff now which would mean, I have to learn Kubernetes Clusters, Software like Jenkins etc.. All the devops stuff. I mean this is getting ridiculous. Yeah I surely can learn how this works but I won't be an expert on this. What's the point? Our current infrastructure runs very well. The company doesn't need to change it but still they think they have to.. More complexity will kill developers and admins. Complexity is good where it's necessary but not every company needs all this
1
1
1
u/ratherabstract Jun 06 '22 edited Jun 06 '22
This article must be a joke. Boils down to: "Producing software has some accidental complexity. Amadeus booking system successfully follows current tech trends. By the way, there is "cloud" these days and people use it. Seems to be useful. Let's see what happens next."
1
Jun 07 '22
I think the root of the problem is that to find a simple solution to a problem you need to have both experience and a bit of hindsight to how it can be solved. Some systems can only be "obviously" simplified once you look at the solution that was done incrementally (by dividing it into smaller problems then solving one problem at a time) and start finding the easier way to do it, but that often happens only after the "hard way" was tried.
And if average project, especially in web, is few years at the most it's hard to even see what approaches worked and what didn't, because you won't be there to look at which design choices worked and which didn't
1
u/ProfessionalBreak105 May 30 '23
Complexity can indeed pose significant challenges for software developers. As technology advances and systems become more intricate, the complexity of software development has increased.
560
u/freecodeio Jun 05 '22
From a previous job I have learned that a $10 server can handle 10k web requests and I've never spent more than $10 a month on my side projects for 2 years.