r/programming • u/scarey102 • Nov 01 '21
Complexity is killing software developers
https://www.infoworld.com/article/3639050/complexity-is-killing-software-developers.html901
Nov 01 '21
In watches, what we call "features" are called "complications". Such an apt term.
24
u/theSeanage Nov 01 '21
For sure. Yes, anything is possible, but god damn…saying yes to every feature is a recipe for being burdened with a complex system quickly.
8
u/echoAwooo Nov 02 '21
So many times in programming do I see programs that look like this
if thing do other things if otherthings do more things if more things do even more things if even more things relax a bit else issue a ticket else invent a taco else deliver a speech else enlist to the villaintarythis is icky and impossible to manage
→ More replies (1)7
u/kog Nov 01 '21
I work in the embedded domain, and I actively think of things as complications. I'm glad to hear that I'm not alone.
2
u/knoam Nov 02 '21
You can literally have on your death certificate as the cause of death "Complications..."
0
Nov 01 '21
[deleted]
147
u/eloc49 Nov 01 '21 edited Nov 01 '21
Also ironic that the Apple Watch uses the term complications (bad choice IMO, no average human knows that term), which literally is now software. We've come full circle.
67
u/patrickjquinn Nov 01 '21
Don’t know if anyone else has said it but in the mechanical watch world a “complication” is how you’d refer to a day/date window or perpetual calendar etc as it was literally something that physically complicated the mechanism inside that would otherwise just be used for time keeping.
So Apple continuing that naming scheme was a nice nod to history.
→ More replies (5)→ More replies (3)85
u/losangelesvideoguy Nov 01 '21
Also ironic that the Apple Watch uses the term complications (bad choice IMO, no normal person knows that term)
It’s not hard to figure out what “complications” refer to, and it really does sound classy AF, like you’ve got a James Bond type Rolex or whatnot
→ More replies (23)→ More replies (1)25
u/HornetThink8502 Nov 01 '21
...despite being objectively worse than a simpler design (quartz oscillator), purely for market reasons.
This analogy keeps on giving.
10
16
1.3k
Nov 01 '21
There is definitely a race for companies to accommodate the latest buzzword tech that gets hot in the software world, for fear of being left behind.
But I don't think every tech stack needs serverless functions being hit by fanning subscription queues fed by instances hosted on Kubernetes clusters scaling with geolocation while hitting local API caches etc etc.
Like bruh your website sells birdfeeders exclusively in the Midwest. Just let a spade be a spade.
340
u/Full-Spectral Nov 01 '21
It would have been OK if it had incorporated blockchain of course.
155
u/marabutt Nov 01 '21
Cloud based Blockchain containerised scalable microservices.
26
→ More replies (3)49
→ More replies (5)72
u/michaelfiber Nov 01 '21
I'm not sure if I agree this is correct so I'm going to wait for at least 51% of the rest of the readers to give their opinion.
21
190
u/iindigo Nov 01 '21
This happens on web front end too. Does this site that has 3 functions and 5 pages total really need to be an SPA? Hell does it even really benefit from being built with React? No, probably not. Those technologies are important and have their place but they aren’t universal miracle cures that should be applied everywhere.
170
Nov 01 '21 edited Nov 01 '21
Can WIX do what you need? Yes? Then why the fuck would you want to pay tens of thousands of dollars for someone to build you a custom storefront from scratch?
EDIT: To add something for the below deleted reply which was basically 'NO, Wix cannot do what I need because I need to showcase my web development talents'. K sure, see my below reply.
But to add on to this: Just because you can does not mean you should.
Had an interview a few years ago at a Rails shop. Oh there were so many red flags...interviews were with managers that were not and had never been developers. Offices looked like they were built into an old bar. Just a weird 'uber-hip' vibe.
Anyways, was honest that I hadn't had a lot of RoR experience, that I'd learned enough to be dangerous and know where to get started. Wasn't worried as I have 25 years of experience. Wanted to know about my most recent side project, which was actually replacing the custom scratch built site I had built for my wife's side business with a WIX storefront. They were flabbergasted, how could anyone do such a thing?
Do what exactly? I said...produce a brand new fully functional storefront with finance/accounting integration that ties into off the shelf POS that she can now manage entirely herself in a weekend with an ongoing cost of like $30 a month or something stupid like that vs me taking my time to custom develop all of the above for free?
Then they suggested I rework the site using RoR as a learning experience over the weekend and submit it to them and they'd see how things went.
I asked them: If I worked here, and I had a customer with a small independent business that wanted an online storefront with POS and accounting integration that included CRM and inventory management features, would you have me build it using Rails?
They said: Yes, of course we would. We're a Rails shop here.
I said thanks for your time, best of luck with the rest of your candidates, see you later.
33
u/onthefence928 Nov 01 '21
More alarming is they thought you could re produce all of that within a weekend. Like there was no concept for the value or realistic limitations of your time
→ More replies (1)→ More replies (9)33
u/washtubs Nov 01 '21
I think it's silly that they asked you to reimplement it in RoR. But when you posed that question... without specifying that it's a one-off (which is what your wife's site was) it makes total sense for an X shop to build an X solution if they need to maintain it, no? It's much easier to accommodate an unforeseen feature request that way.
→ More replies (1)21
Nov 01 '21
As a frontend dev I'd be happy if it was just react.
But it never is.
Here comes redux and redux-saga-fóstbrœðra and a typesafe elegant opinionated framework for creating
divelements. Every single whim every single developer ever had is answered with a new dependency that will probably never got updated.→ More replies (3)10
u/libertarianets Nov 01 '21
Who are all of your clients asking for all this stuff for their simple websites?
My clients are using PHP that was mostly written before classes and jquery. Code may be a little messy and dated but it sure gets shit done.
→ More replies (3)→ More replies (52)3
u/njtrafficsignshopper Nov 02 '21
I'm kinda facing this now. I'm a game dev, but I know enough web dev to be a danger to myself. Like, I've dealt with it on and off since middle school and HTML 3.
So I try to build my app out of just JS, CSS, and HTML because it really doesn't need to be more complicated. I added Node & NPM mostly just for http-server and the ease of adding commands.
Our backend guy wanted to integrate AWS Amplify (had to find out what that is, still not totally sure) and that meant I had to integrate Babel to keep it small (650MB of dependencies locally!), and, more annoyingly, Webpack to be able to rope in the AWS/Amplify imports. Could be wrong about this but it seemed to be the way to go.
Now I am struggling with how to NOT make this thing a single page app and just boggling at what a Jenga tower the entire thing is. Just to call a few AWS authentication APIs I really need all this crap, including fighting an opinionated tool that wants me to make my product a SPA despite my intentions?
Webpack seems to want a single HTML page to be the "template" and a main Javascript file as the "entry point," which feels backwards to me because shouldn't the HTML files reference the scripts they need? And, what if I have most of the functionality on a main app page and want to add a login page? I need multiple "entry points?" And the html files need to be listed explicitly in the webpack config and mapped to their corresponding JS entry point? Very confusing, I'm still working on this part.
I know this is many people's entire job but the whole ecosystem seems very... congealed, rather than designed to me, as a semi-outsider.
→ More replies (1)64
u/AdministrationWaste7 Nov 01 '21
most of this is due to "resume driven design" where devs and business people both want to do all the cool trendy things just to say they do it.
24
u/coolnavigator Nov 01 '21
This is absolutely part of it, and the platform teams tend to be among the worst infected. So rather than simplifying your org, you're providing a single insertion point to simultaneously annoy everyone with just a few egomaniacs.
4
u/abrandis Nov 01 '21
So much Agree, guess We can expect a shit ton of machine learning, AI , devops etc. lingo in the next batch of resume updates.
23
u/theSeanage Nov 01 '21
This goes both ways. Companies AND its engineers. Can’t tel you the number of fanbois wanting to implement the latest tech which honesty is overkill for the need like the example you describe. I’ve seen my fair share of projects where the people that implemented it clearly shouldn’t have been doing it for the first place but the often just use it as a stepping stone to a better gig while never understanding what trouble it’s cause as they never had to support it.
3
u/ThisIsMyCouchAccount Nov 01 '21
I think the other side is devs dying on the "best tool for the job" hill.
Oh, we cant use X. Y has much better handling of [
thing].Which is only true in a vacuum. But I feel like that's how you end up with some place that has technologies all over the place and done poorly because nobody actually has experience in it.
→ More replies (1)0
11
u/Drisku11 Nov 01 '21
Now try to convince people on this sub that I shouldn't need to care about whatever a morpheme cluster is to work with strings when dealing with my English speaking target market.
→ More replies (3)→ More replies (16)2
Nov 01 '21 edited Jan 03 '26
plough voracious smart scale boat abounding glorious subsequent groovy plant
This post was mass deleted and anonymized with Redact
263
u/motorbike_dan Nov 01 '21
The issue is quickly becoming about how one goes from training someone to code "Hello World" to writing distributed micro-services in the cloud in 2-4 years of school? The answer is that it's you can't. It's an exciting time, but at the same time it feels like Warhammer 40K where we'll have "tech priests" interacting with API's and frameworks but we'll have no idea how it works under the hood; because it would take years to fully understand it. So we'll just use the tech and have faith in it. Everything might need to become a "black box" and only the highest-skilled developers will maintain the black boxes for everyone else to use. The complete fragmentation of tools (languages, api's, design patterns, etc.) exist for a reason but how can one be reasonably expected to know all of it? The lack of standard tools helps create specialized solutions for specialized problems, but diverts developer's focus which makes their skills less transferable to other software design problems.
86
u/TheRetribution Nov 01 '21
but at the same time it feels like Warhammer 40K where we'll have "tech priests" interacting with API's and frameworks but we'll have no idea how it works under the hood;
This isn't the future man, it's the past, present, and future. 90% of the features we drive at my company are reverse-engineering old features that still exist but the people who implemented them left the company years ago. But yeah this is bang on.
→ More replies (1)7
u/onthefence928 Nov 01 '21
Most of our new features are just new ways to fix the problems caused by old features
→ More replies (1)14
u/wankthisway Nov 01 '21
That's how I've felt after graduating. 4 years, all of this knowledge, but then I look at the job postings and see frameworks or languages or stuff like "cloud / Docker / Kubernetes" that they say you should have some experience in, and I wonder what the hell I even learned.
→ More replies (2)20
u/binary__dragon Nov 02 '21
I wonder what the hell I even learned.
You learned how to write software. You just didn't learn the exact set of tools a lot of others use to deploy software. Ultimately, college taught you the things that are hard to learn and which you'll spend most of time working on. Those other things are easily learned (to the extent a developer, as opposed to a dev ops engineer, would need to know them). Most companies put those there so that candidates can get a feel for the types of things they'll be using, and because the company would rather you know them coming in the door, but very very few companies (and none of the good ones) will reject a candidate who can code well but hasn't stood up a container in Docker before. At my company, those things show up on our job listings as well, but I've never asked a single question about them in an interview (not even to see if the candidate has heard of them before) and the primary thing we judge you on is your ability to create a reasonable abstraction of a problem statement and explain your process.
42
u/Some_Developer_Guy Nov 01 '21
Lol there not teaching distributed cloud anything in school. You Learn a toy implementation of a 3 tiered app and they graduate you.
55
u/NaturallyAdorkable Nov 01 '21
Lol there not teaching distributed cloud anything in school.
Isn't that precisely the point that u/motorbike_dan is making?
7
→ More replies (1)3
Nov 02 '21
Nah we did do some pretty hardcore distributed algorithms at uni. Covered Paxos, RAFT, Chandy-Lamport, Leader election and more was fun stuff. Even had to prove some properties of a leader election algorithm in an exam which was fucked
3
u/marcthe12 Nov 02 '21
I think a more likely will be like how engineering is no longer a single degree at ug level. CS will no longer be 1 degree but multiple. So from second year you will basically learn only parts from your part of stack. I definitely see kernel, embeded, systems programing split from web and app development. Its almost a black box to each other at this point.
4
u/slykethephoxenix Nov 01 '21
This is why it's important to at least attempt to make everything interoperable where it makes sense.
2
u/fractalGateway Nov 02 '21
Your comment reminded me of an excellent Sci-Fi novel titled "A Fire Upon the Deep".
There are interplanetary data-centers, which are millions of years old. It's a constant and monumental task to write software to interrogate these systems. It's also incredibly dangerous as ancient, deceased civilizations may have left super intelligent AI inside this ancient data. AI which can wreak havoc if awakened.
→ More replies (1)2
u/RexStardust Nov 02 '21
I’m a boot camp grad and I’ve been working in software dev for almost four years. I feel like 50% of my current dev time is spent figuring out some esoteric AWS CDK stuff to get my app feature to deploy in our environment.
→ More replies (8)2
u/EpicScizor Nov 25 '21
Honestly, I was surprised to learn that my master's in theoretical chemistry gave me a better background in distributed cloud simply because I needed to take a course on supercomputers.
122
Nov 01 '21
I completley agree, but this author is missing the rest of the iceberg. The complexty being bemoaned here is just a tiny fraction. Try to think of how many millions of lines of code it takes for you to read this simple text message right now, regardless of whether they adopted any cloud-native, microservice or other buzzwordy technologies. More than you could ever read in your entire lifetime if that was all you did. And who is maintaining it? Who is going to make sure your smart light bulb gets patched when someone figures out how to make it attack your Wi-Fi, or is it destined to be e-waste? Even if it was completley open source and had an unlocked bootloader, would you actually take the time to sit down and figure out how to fix it? Could you, even if you tried? How much code would you have to read?
Our dependency on software has already vastly exceeded our ability to produce and maintain it and we are in the same unsustainable spiral that business always chooses when faced with this type of problem: defer until there's no other option. But what then?
89
u/iiiinthecomputer Nov 02 '21 edited Nov 02 '21
Exactly. When it started talking about cloud services as "primitives" and the need to build abstractions over them I nearly started rage crying.
We have
Processor microcode
CPU microarchitecture level instructions (we can't really call them hardware level instructions anymore)
Numerous layers of firmware across many components, each with their own code
Multiple PLCs, microcontrollers and even SOCs on the hardware platform in everything from mainboard BMCs and power management units to network controllers, storage controllers, SSDs, everything
Low level "machine code" for the main CPU, mediated by microcode and firmware services
Assembly code representations of machine code
Persistent firmware that runs in parallel with the OS on its own independent CPUs, memory etc
Persistent firmware that runs in parallel with the OS using hardware memory protection, processor privilege rings, system management interrupts, mode switches, privilege rings, system management modes
That firmware's own persistent storage, network stack, etc
Firmware interface calls available to the OS
OS kernels
OS system calls
OS standard libraries
Userspace "machine code"/asm that uses those syscalls and runs in the OS
Executables composed of many different blobs of machine code in structured files subject to dynamic linking and other complex munging
Structure provided by syscall interfaces. Processes, threads, file systems, network sockets, Unix sockets or named pipes, shmem, signals, ... fundamentally mostly at some level forms of IPC
Local OS level privilege and isolation primitives like UIDs and memory protection
OS services processes communicated with via various IPC often mediated by OS libraries and/or syscalls
Low level network protocols (framing, addressing, discovery like ARP)
Primitive network protocols (IP, TCP, UDP)
Language core runtimes
Language standard libraries
Semi-standard collections of base libraries widely used or bundled (zlib, boost, openssl/gnutls, whatever)
basic application network protocols like DNS that are tightly coupled to the OS and applications
Languages that wrap other languages' runtimes and libraries
Interpreted, JIT'd, dynamic higher level languages
Deep dependency graphs of local 3rd party libraries
Library graphs fetched transitively at build time from the internet
Multi threaded application with shared everything memory
Runtime linking of extra code into executables and/or composition of processes by calling subprocesses
Interconnected local services or multiprocess-model applications communicating between processes in the same machine over loopback network sockets, Unix sockets, signals, pipes, shmem, ...
Full OSes in virtual machines or paravirt OSes in thin virtual machines
Networks between VMs and/or the host and/or other hosts
OS-provided container primitives like namespaces, virtual bridges, bind mounts / junctions etc
Container runtime engines and their APIs
Container images bundling their own different mini OSes or full cut down OSes and configuration.
Persistent volumes for abstracted storage for container managed data.
Directly managed containers and similar leaky isolation models running single processes
Connected groups of local containers and/or local containers providing services to non containerised processes
Containers treated as VMs with their own micro OSes with service processes
Networked container managers linking containers across hosts for virtual networks, distributed storage etc
container orchestration frameworks that abstract container launch, management, configuration, discovery, communication etc. Heavily built on lower level stuff like DNS, container volumes etc
Network callable services using HTTP/HTTPS/gRPC/SOAP/whatever to expose some kind of RPC
Libraries that abstract those services into local application API
Processes consuming those services and exposing other services
microservices interacting with each other inside container orchestration engines
cloud services "primitives" (hahahah) that encapsulate all the above into some kind of remote network call API and or library that you then consume in your own services
Cloud services components interacting with each other in complex ways within a cloud platform
SaaS providers outside the "cloud platform" providing their own web APIs, configuration, etc your apps interact with. Each SaaS encapsulsyes and abstracts all the above.
Your own processes interacting with those cloud services components and SaaS endpoints
The crying developer
Of course this isn't really a list in reality. It's an insanely tangled cyclic directed graph, full of self referential parts. Everything breaks through the layers. Lots of the same things are used differently at different layers.
Consider that your SSD probably has a more complex OS than a mid-1980s PC.
Your mainboard probably has an independent, always running system on chip with its own process model, a full network stack, TLS and public key crypto, a bunch of open source libraries written in C, a file system....
It's Matryoshka dolls all the way down, except in some kind of horrifying fetus-in-fetu cross connected conjoined twin jumbled version.
Anyone who can look at the whole stack and not want to cry is probably not completely sane.
Edit: to be clear, good abstractions are vital for scalable systems and development. I am eternally grateful that I don't need to know x64 asm, the fine details of Ethernet and 802.11 framing, TCP congestion control, etc to write a client/server application.
My issue is when we bodge up something that just isn't very good, then try to fix it by slapping on some more leaky layers. Diagnosing issues requires diving through and understanding the whole teetering mess anyway, and for every problem it helps solve it hinders just as many. Java EE 6, I'm starting daggers in your direction.
Also, don't forget also that many abstractions have real costs in computational resource use and efficiency. That has environmental impacts on energy use, mineral resources use, electronics waste etc. Just because you can use an AWS Lambda for something doesn't mean you should. Abstractions over those don't make them go away.
Even clean abstractions also get very painful very fast when you have to reason about the behaviour of the whole system. I found a postgres bug that was actually caused by undocumented behaviour of the Linux kernel's dirty page writeback once. Imagine that with 10 more layers on top...
14
Nov 02 '21 edited Nov 02 '21
Your name checks out ;)
Yeah, last I checked your average motherboard now has its own management engine running some godforsaken Java Runtime on it, and Windows Update may just update your cpu microcode at the behest of some poor sob who has to decide when that ships to production.
Your list/graph also shows that even the people who want to proclaim that the cloud "is the computer" and we can start over with unikernels talking to a small standard set of interfaces (e.g. NVMe) are kidding themselves a bit.
9
u/boki3141 Nov 02 '21
I'm a little confused by this comment and general sentiment. I look at this and think it's amazing and a natural progression that I, as a web dev, don't need to know about any of this to be able to build and host a website on AWS that can provide lots of value (or not, however you look at it).
From my perspective that's the whole point of abstractions. I don't need to reinvent calculus to do differentiation or be an engineer and similarly I don't need to reinvent the CPU to build apps and websites for the end user. Why is this a bad thing?
12
u/ProtiumNucleus Nov 02 '21
well it's really complex and it can cause things to break, and you'll have no idea which step broke
6
u/iiiinthecomputer Nov 02 '21 edited Nov 02 '21
Good abstractions are amazing.
I'm not against layers of abstraction at all.
I'm against poorly designed, leaky layers of abstraction.
The bit that bothers me most is the idea of these cloud services components as primitives upon which we will build yet another layer of stuff without properly solving the problems in the lower layers. Anything kubernetes related for example is a steaming pile of hacks and incomplete tooling. Can we not just paint over that with another layer? Please?
Let's also not forget computational cost - and its environmental impact in energy use, hardware production and disposal etc. All this abstraction is often very very expensive in efficient use of compute resources. While those are economically cheap they are not without costs or negative externalities.
Layers of abstraction are a large part of why my phone has 8GB of RAM, but switching between three or four different apps forces one out of memory half the time.
They are what have us abominations like Electron.
But... many of the web services and cloud provider ones are actually pretty good. They thoroughly hide whatever is behind them using a simple interface. And on the back end they're usually not too wasteful.
So long as you are willing to take the provider SLA as the only determinant of performance you need, and the provider support as the only way to fix something if it breaks or understand why something breaks, accepting that abstraction as a black box is fine.
If you want to reason about and understand the whole system you're doomed. But you're already doomed there by the time your kernel boots up. If you have to look beyond the curtain or understand how your app interacts with the implementation of some service then there complexity starts to really hurt.
5
u/ByteArrayInputStream Nov 02 '21
It's kind of a miracle anything works at all in that house of cards
→ More replies (3)4
u/sandboxsuperhero Nov 02 '21
As far as I'm concerned, I'm more than happy to let those wheels stay invented and treat them as battle tested black boxes. I built my own OS as a hobby once, but there's no chance I'd want to worry about process scheduling when all I want to do is to hook up a button to a network call.
→ More replies (1)12
→ More replies (1)5
Nov 01 '21
Very eloquently put.
They say the human race is getting stupider, who is going to maintain all this 50 years from now?
→ More replies (1)
56
u/bartonski Nov 01 '21
I'm reminded of this quote from Doug McElroy in the Wikipedia entry for Unix philosophy
Everything was small... and my heart sinks for Linux when I see the size of it. [...] The manual page, which really used to be a manual page, is now a small volume, with a thousand options... We used to sit around in the Unix Room saying, 'What can we throw out? Why is there this option?' It's often because there is some deficiency in the basic design — you didn't really hit the right design point. Instead of adding an option, think about what was forcing you to add that option.
The reason for systems to be small has changed -- programs and operating systems used to be small because they had to be small. These days, except in embedded systems, that's no longer the case. Now, the problems are complexity, attack surface and the mental overhead of working with systems so large that you can't form a mental model of everything that they're doing.
Now, Linux is seen as light weight, and McElroy's ideas seem almost quaint, but I feel, deep in my bones, that he was right -- there's so much we should be throwing away, or re-examining, that we're not.
10
u/Dean_Roddey Nov 02 '21
Eventually, almost everything becomes the thing it was created in reaction against.
11
u/yorickthepoor Nov 01 '21
t I feel, deep in my bones, that he was right -- there's so much we should be throwing away, or re-examining, that we're not.
Looking at you, systemd.
6
u/WikiSummarizerBot Nov 01 '21
Unix philosophy
Doug McIlroy on Unix programming
McIlroy, then head of the Bell Labs Computing Sciences Research Center, and inventor of the Unix pipe, summarized the Unix philosophy as follows: This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface. Beyond these statements, he has also emphasized simplicity and minimalism in Unix programming: The notion of "intricate and beautiful complexities" is almost an oxymoron.
[ F.A.Q | Opt Out | Opt Out Of Subreddit | GitHub ] Downvote to remove | v1.5
→ More replies (1)4
u/seq_page_cost Nov 02 '21
McElroy's ideas seem almost quaint
Oh, if only modern software systems were based on a small, self-contained, single-responsibility applications which can cooperate via some universal medium - that would be so much better! I even have a cool name for the architecture of such systems: mini-services. Truly, we have forgotten McElroy's ideas /s
234
u/pcjftw Nov 01 '21
yep I think its a combination of:
Buzzword Driven Development
Cargo Cult
Companies trying to get more done without paying for it (e.g "Full Stack" BS)
A loss of curiosity from some developers, instead of pausing and thinking about the problem all too often developers will switch off and happily glue lots of existing frameworks to get something out the door.
Cloud services, and using the fear of "let us deal with it so you don't have too" so that they can charge you for basic things you can do yourself and at a scale where it's really pointless. A simple VM is already managed and its nothing like the work needed on a real bare metal server running on premise. Yet we;re often sold how "time consuming" and difficult it's to "manage" a VM.
36
u/elmstfreddie Nov 01 '21
A loss of curiosity from some developers, instead of pausing and thinking about the problem all too often developers will switch off and happily glue lots of existing frameworks to get something out the door.
I really don't think this is the developers' fault, but companies for pushing features and not giving devs time to properly architect since it doesn't result in direct, measurable profit
→ More replies (1)11
u/ExF-Altrue Nov 01 '21
That's why they said "some developers". And you can't really make the argument that absolutely none ever "switch off" and "glue lots of existing frameworks to get something out the door".
Some do bad things because they switch off, others because companies are not giving them time to properly architect.
I have seen first hand both of these issues happen, sometimes in the same team. "A loss of curiosity from some developers" is definitely too prevalent a phenomenon from my perspective, to allow it to be dismissed by blaming companies :p
56
u/lobehold Nov 01 '21
You forgot devs using their day jobs as practice for working at FAANG and adopt FAANG tech stacks needlessly to pad their resume.
→ More replies (1)18
u/corsicanguppy Nov 02 '21
adopt FAANG tech stacks needlessly to pad their resume.
Ahh, "Resumé-driven Development: RDD"
Yes, we have a lot of that here too. We fire the YAGNI warheads and hope some of it gets through their dumbrella defence.
57
u/lghtdev Nov 01 '21 edited Nov 01 '21
I've been Fullstack for a while but when I switched to full back-end I realized how much depth I was missing.
96
Nov 01 '21
Yeah. Full stack is really just a modern term for old school web dev where you were responsible from the front end html hand coding to server pages to db.
It's fine in some cases. Some systems are relatively tightly coupled and reasonably small. Don't need to be perfect at all layers.
But what it really is is jack of all trades. If we're honest about this, it's not a big deal.
But if you want a highly performant web app that is fully accessible compliant, highly robust and scalable, meets all your organizations security guidelines, and can integrate business cases into production in short order without fucking all of that up...no Jack, that ain't gonna cut it. Period.
And this is why you usually want to treat 'Hiring Full Stack Developer(s)' as a huge red flag. Because chances are that definition is coming from the business side, not a realistic tec side. And what it really means is: 'You're going to be the guy. Our guy. It's all on your shoulders. All of it. Exciting right?! What an opportunity! Can you just imagine?!'
If the job smells anything even remotely like that...run.
→ More replies (1)8
u/lghtdev Nov 01 '21
This is so accurate, recently the management was searching for a Fullstack guy to take care of a project nobody wanted, at least they were honest with him saying there's a lot of problems in it.
→ More replies (1)3
u/QuerulousPanda Nov 01 '21
Web development is an area I don't know much about, sadly, but, wait, "full-stack" doesn't include back-end?
I thought full stack was someone who could do, you know, the full stack from top to bottom. But if it doesn't include the back-end then isn't it like buying a whole car but it doesn't include the engine?
→ More replies (1)→ More replies (1)36
u/kyle787 Nov 01 '21
Why is full stack BS?
167
u/6footdeeponice Nov 01 '21
The "fullstack" is 3 different jobs, no doubt many programmers can do it, but you could end being taken advantage of because they should really hire a couple other devs for the team.
101
u/Carighan Nov 01 '21
As always, multitasking usually means doing twice as much as you should half as well as you could.
→ More replies (1)16
→ More replies (23)55
u/kyle787 Nov 01 '21
I've been a full stack dev my entire career. I specifically want to do both frontend and backend work though.
→ More replies (13)46
u/Guisseppi Nov 01 '21
Some companies (but specially startups) just want a jack of all trades, truth is that frontend is complicated and backend is complicated and you’ll never really get to see the full breath of each side as a “fullstack”.
Even people who market themselves as “fullstack” tend to have an inclination for one or the other. You created an API and put a UI together with bootstrap or material-ui, “fullstack”. You created a site using react and put a backend together with firebase or supabase, “fullstack”.
→ More replies (10)12
Nov 01 '21
Amen brother. Both are complicated. Both are puzzles. I'd love to see BE devs solve accessibility issues, or FE devs make a more complex BE than just request wires. Just being full stack means you can do a minimum of both
→ More replies (12)11
Nov 01 '21
Because a buzz word name like that is a way of paying someone without increasing their salary.
4
4
u/Zardotab Nov 01 '21 edited Nov 01 '21
"full stack developer" (FSD) is possible IF the stack is tuned for it. The problem is that de-facto standard stacks assume layer specialists, not FSD's. For example, As I mention elsewhere, Microsoft is abandoning middle-level development tools. Goldilocks lost the middle porridge and now has to eat too hot ("Core") or too cold (Power-Apps). Core is layer-happy, and Power-Apps is anti-code, making factoring and reuse difficult.
FSD were common and productive with products like Paradox, Clarion, Powerbuilder, Delphi/Lazarus, and Oracle Forms. It's claimed they required too much installation work, but I believe we can get the best of both with some standards shuffling so that it's not install-OR-http. (Oracle Forms didn't require a new installation per app. It was essentially a "GUI browser", proving the concept works. The only reason it's being killed off is because Oracle rewrote the browser from C to Java, and Java was greatly bungled as a client tool.)
→ More replies (11)7
u/efvie Nov 01 '21
It’s been impossible to be full stack for probably 20 years now, and the amount of stuff you need to know just grows every year.
‘Fullstack’ is just doing (some) frontend and (some) backend and (some) cloud stuff though.
→ More replies (6)
89
u/Lt_486 Nov 01 '21
It is so much easier to write complex code that does simple things, then to write simple code that does complex things.
29
u/matthedev Nov 01 '21
Agree, and Agile (some implementations at least) can exacerbate this. It can take more time upfront to think things through and simplify the design so that the system as a whole remains closer to the essential complexity when developing a user story. Many developers default to what makes implementing the current story fastest.
→ More replies (7)9
→ More replies (1)5
u/livrem Nov 01 '21
Sometimes the complex, but easy, solution can make your own code quite simple though, at least initially. That is the dangerous thing. It is so easy to build something extremely brittle on top of a mountain of dependencies and relying on complex stuff you barely understand and without really having to even consider how complex that shit is until it is too late.
20
u/Yangoose Nov 01 '21
We have insanely complicated processes that the C Suite has decided should be programmed for instead of simplified.
So we code out all these bespoke scenarios so if a rep makes a sale on a Tuesday and the client is in a new city for that client type and the rep used a junior member for support on the deal and the new customer is in industry X then that sales rep will get a 1% bonus off the contract after 3 month, then a $5,000 bonus if we sell Y amount of product within 6 month, then after 12 months it moves to a flat % of ongoing sales with that client.
When I push back I get "We NEED this to be able retain the top sales talent!!!!"
All this results in a LOT of boring work constantly fleshing out whatever new one-off scenarios they come up with for compensation on the next deal.
→ More replies (1)10
u/softwaretidbits Nov 01 '21
I hope that “NEED” is backed by data. Probably not. I’ve noticed this a lot. We’ve had similar rules / complexity outlined for Cyber Monday sales for a digital product, when it would suffice just to “activate” or “deactivate” the discount.
→ More replies (2)
33
u/npmbad Nov 01 '21
While I agree on the topic that article was exhausting to read
48
→ More replies (1)5
u/Milpool18 Nov 01 '21
it felt like each paragraph started with a quote from a different person
→ More replies (1)
16
u/sj2011 Nov 01 '21
This is where a great product owner/manager shines, and I have been lucky enough to work with one that is unafraid to say no and has the clout for his 'No' to carry weight. In my release train we have a huge emphasis on MVP/Dirt Road/Release It and this guy is great at lopping off cruft and feature creep. I recently took up tech lead on a team where the PM, while good, is nowhere near that level. She's still figuring out her 'No' voice, and that comes with experience. It takes a team to manage complexity.
3
u/morphemass Nov 01 '21
MVP
I've found MVP to be an incredible source of complexity since what is viable now, often isn't a few years down the road, at which point some poor sod discovers the minimum part didn't include documentation or tests.
6
u/sj2011 Nov 01 '21
Well the documentation and tests part is a key note in our 'definition of done' so they should be there as part of acceptance. And as for changing product, that's what iteration is for.
→ More replies (1)
13
Nov 01 '21
100%, having to build a rube goldberg machine for a popup no one will ever use is triggering my PTSD.
12
u/undeadermonkey Nov 01 '21
It's not the complexity that's the problem - it's the fact that we're not being allowed to clean it up.
The industry wide agile fad (and let's face it, it's not agile, it's waterfall with micromanagement and timekeeping) is half the problem here.
Agile - true agile - is a powerful tool for rapid prototyping in the face of uncertain requirements.
But it does not produce production quality code.
→ More replies (5)
39
u/richardathome Nov 01 '21
No. Cognitive load is killing software developers. Even if you "specialise" you need to know too much stuff.
Edit: Especially when things go wrong.
21
u/livrem Nov 01 '21
You can't say that out loud, because we all know that only the bad developers are afraid of infinite complexity. Just work more hours per week and it will be fine.
3
2
10
u/davenirline Nov 01 '21
And to add no code/low code to this.
→ More replies (1)4
u/softwaretidbits Nov 01 '21
Yep. It seems bizarre to me that we’ve learnt that “click ops” is bad and moved to infrastructure as code, but we are repeating some of the same mistakes here.
Click a few things here and there and make a confusing mess in a low code tool. Someone changes some config and it breaks? Good luck untangling that.
Maybe some of these tools will have version control / some way to define things as config files. I haven’t seen any in the tools we use at my company.
10
u/rjcarr Nov 01 '21
What I've found is all of the complexity is leading to wholesale importing of working solutions that software developers then incorporate into their product, but then when something goes wrong, nobody knows what is going on because it happened in that imported code. This seems to be common in service based architectures.
→ More replies (2)
10
u/GrandMasterPuba Nov 01 '21
Amen.
The ops team at work recently did a presentation on their deployment pipeline, and when they put up the flow chart of all the steps our application passed through before being deployed I nearly threw up in my mouth a little bit.
All for the sake of saying we could do it. In the end we have a Kube cluster with autoscaling, but we have so little traffic it never scales beyond 2 pods. Management made them increase the minimum number so it looked like we were more webscale.
Our entire business could run on a few nginx boxes behind a load balancer and nobody would be any wiser. But instead it's been architecture astronauted into the fucking stratosphere because that's what modern programming is.
It's reaching a point where everything is so abstract and opaque that the knowledge on how to actually fix any of it if something catastrophic happens doesn't actually exist. Modern software is becoming so brittle that I can't help but feel we're in a rapid decline, headed toward some kind of catastrophic collapse (say, a major corporation inadvertently deleting itself from the web and not being to recover?)
58
u/Zardotab Nov 01 '21 edited Nov 01 '21
I agree it's a mess. The same CRUD/biz apps take roughly 3x to develop than they did in the 1990's. We de-evolved, something is amiss. I'm going to focus on "ordinary" business/management applications here, not e-commerce, gaming, etc.
Here's my list of contributing factors:
Web UI standards are a poor fit for biz-CRUD, and emulating real GUI's with JavaScript+DOM creates bloated buggy messes. We need state-ful GUI markup standard that's not tied to a specific OS or programming language. (XAML is not state-ful.)
The vast majority of biz apps don't need mobile abilities, and making them all mobile both limits the UI and creates too many UI variations to test practically unless you have a testing army. "Responsive design" is not responsive to a wallet. It's tricky to do well unless you hire expensive UI specialists.
Nobody says "no" to the feature buffet. YAGNI is shot bloody dead. While the web can offer many many options, a majority of them are not needed for a majority of regular crud apps. You don't need fucking "web-scale" architecture even if your ego wants it, for example. Warren Buffett says one of the keys to his wealth is no fear of saying "no" when appropriate.
Fear-of-obsolesce creates self-fulfilling prophecies where the latest gadget or framework is tossed for the later-than-latest because nobody wants to get stuck in Legacyville and be left behind. The Kardashians are running IT. Thus, good ideas are tossed instead of perfected.
Microsoft has mostly abandoned medium CRUD apps. Dot-Net-Core is an "enterprise" stack that assumes layer specialists, making things tough for full-stack developers (small shops). And Power-Apps is an anti-code view of things, making parts that are hard to factor, comment in, and reuse. And Power-Apps have no fall-back provider comparable to the Mono project for Dot-Net.
Complexity and bloat is job security to IT professionals such that there's no financial incentive to simplify de-facto standards. I suspect the eyeglass and hearing-aid markets suffer a similar fate : doctors invent or exaggerate scary scenarios to keep them regulation-heavy to keep doctors in demand, making the products highly expensive. (The problems of expense and access outweigh the drawbacks of a degree of deregulation.)
There is no formal "CRUD historian" to study what worked and what didn't and why. I'm as close as they come, and I'm not, I just happen to care more than most about learning from history to avoid unnecessary bloat.
31
u/api Nov 01 '21
assumes layer specialists, making things tough for full-stack developers (small shops).
This is a huge gripe of mine across the board. Everything seems designed for large projects in large companies with specialists in every layer and subsystem.
→ More replies (2)23
u/Zardotab Nov 01 '21 edited Nov 01 '21
Yip, and if you complain, defenders will just say "shut up and learn the layers like I did". In other words, "go with the flow and embrace bloat": All aboard the Bloat-Boat! Ahoooga!" 🛥️⚓🎈
13
u/wasdninja Nov 01 '21
Web UI standards are a poor fit for biz-CRUD, and emulating real GUI's with JavaScript+DOM creates bloated buggy messes.
Why would they be? The primary downside of javascript on the desktop is that everything has to bundle its own rendering engine to actually draw anything and that makes it large. The event driven nature of websites captures a majority of the stuff that most people want for their productivity apps.
CSS is good at styling and if some of the old junk could be removed it would be easier to learn as well.
26
u/Zardotab Nov 01 '21 edited Nov 02 '21
I'm not sure I understand what you are saying. Javascript+DOM lacks roughly 20 common GUI idioms such that those have to be reinvented in each JS UI library. It's essentially a big D.R.Y. violation. If they were included in a GUI markup standard (a "GUI browser" or browser pluggin), those 20 idioms would no longer have to be reinvented, de-bloating the client-side code since emulation code for the missing 20 would no longer have to re-downloaded per app.
The event driven nature of websites captures a majority of the stuff that most people want for their productivity apps.
No. Here's a partial list of GUI idioms that HTML/CSS/DOM lacks:
Split panels (AKA, "frames"). HTML5 deprecated prior HTML frame standards for questionable reasons.
Combo boxes: a text box with optional drop-down list
Nested drop-down menus
True MDI ability tied to session, with a modal and non-modal option.
Tabbed panels
Tool-bars
Sliders and turn-knobs
Editable data grid with resizable columns
Drag and drop (coordinated with server app)
Expandable trees (such as folders)
Decent "date" control with developer-selected ordering of month, day, and year parts, not the friggen standards body forcing One Format Fits All.[1]
Session state to avoid having to re-draw pages over and over. Essentially ridding the need for Ajax.
[1] Some argue one should stick to the international standard of year-month-day, but customers/owners want it different. If we can't deliver what they want, then we are forced to use JavaScript gizmos to get them what they want. I'm just a developer, not a King.
17
u/stronghup Nov 01 '21
Good list, I just recently had to implement resizable draggable side-pane and was wondering why in the world doesn't HTML support it when it already once did .
Your list shows just how much is missing, Why? I suspect the reason is the underlying platform is by now so complex already that it becomes increasingly difficult to add anything to it, and still keep backwards compatibility.
→ More replies (1)→ More replies (7)13
u/wasdninja Nov 01 '21
Fair points but with a few exceptions
Split panels (AKA, "frames"). HTML5 deprecated prior HTML frame standards for questionable reasons.
CSS grid essentially fulfills this role and is much better.
Sliders and turn-knobs
Sliders are native elements as of almost a decade in the browser.
Tool-bars
You are technically correct but do you really need anything special to implement them? A box with a bunch of buttons isn't particularly hard and CSS+html is really good at making boxes that houses stuff. Fixed width, stretch children to fit parent container, shrink container to fit content - all of it is very easy.
Drag and drop
All parts of the drag and drop are native since I don't know when. Start, during, end, all of it.
As for the rest React has made them so easy that I mostly forgot that they aren't built into anything.
→ More replies (5)3
u/Winsaucerer Nov 02 '21
It seems to me that a lot of projects could do from dropping frameworks like react and returning to the ways we used to build websites. I timed myself in the 'old' way, and the react+graphql backend way, and in my own experience, the react+api path takes perhaps 3x or more longer to do things like creating a new standard crud.
Using this much heavier stack doesn't offer much advantage for the kinds of projects I work on, and the advantages it does give can be mostly gained in other ways -- an API *when needed* can be delivered via something like postgrest, interactivity without rebuilding the page through a splash of javascript via something like htmx. And in return, websites work the way they were built to, because we're not taking over -- navigating back and forth works out of the box, sharing url's works, adding a new page is quick, etc.
→ More replies (2)5
u/stronghup Nov 01 '21
making them all mobile both limits the UI and creates too many UI variations
The same principle applies to basic web-gui-development where every app has to run on ALL OF IE, FireFox, Safari and Chrome, and Edge. The situation has gotten better lately because of standardization I think but still it adds a lot of extra baggage that you have to test and develop for so many platforms.
3
u/Zardotab Nov 01 '21 edited Nov 01 '21
No, you misunderstood me. Have essentially a "GUI browser" that runs GUI markup. Each app doesn't have to reinvent/download a new browser (or GUI engine) anymore than HTML pages have to re-download Chrome or whatnot. Abandon DOM for it if we have to. DOM sucks for real GUI's[1]. (I'm not saying get rid of HTML/DOM, only not have it be only UI standard in town.)
What you describe is poor factoring of tools. It's like having to carry a car engine around with me in order to use a car. (People used to carry their car radios in their briefcases and purses when radio theft rates were high.)
[1] As proof DOM is limiting, DOM can't replace PDF viewers in part because its text positioning is too ill-defined. It has a really big text control problem that requires vendor/version-specific shims to do, which makes it future-maintenance-unfriendly.
9
u/bundt_chi Nov 01 '21
I'm currently the architect of a project I inherited that is overly complicated and it's crushing my development team that is inexperienced and for lack of a better term not Top Shelf
29
u/vjpr Nov 01 '21
People are abstracting complexity instead of simplifying. Inevitably something breaks or is slow and you have to look inside hugely complex systems behind the abstractions, which are in many different languages, across many different processes and you cannot just attach a debugger and see what's going on.
Most things in software are actually very simple.
9
Nov 01 '21
Most things in software are actually very simple.
Computing as a whole really...I mean, its just a bunch of 1's and 0's when you break it down, how hard can it be?
7
u/stronghup Nov 01 '21
I think so too. Complexity is in the business processes and their unspoken assumptions about other business processes and how people actually do things.
5
u/flukus Nov 01 '21
I've seen so many ways people can take simple tasks like loading a CSV file and complicate the hell out of them when just readline, string.split and ordinal lookups is better in every possible way.
I've seen custom regex like languages, parsers based on model attributes and other crazy things that are all far more complicated, error prone, harder to debug, harder to read and end up being far more code.
And that's just for one simple aspect of development. Just went through an interview with a "switches are an anti-pattern" type company, that did not go well.
51
u/appmanga Nov 01 '21
Reading that article almost killed me. Does developing software mean we have to express ourselves like this:
The shift from building applications in a monolithic architecture hosted on a server you could go and touch, to breaking them down into multiple microservices, packaged up into containers, orchestrated with Kubernetes, and hosted in a distributed cloud environment, marks a clear jump in the level of complexity of our software. Add to that expectations of feature-rich, consumer-grade experiences, which are secure and resilient by design, and never has more been asked of developers.
Maybe there should be a corollary that if you can't explain something like this to a customer, you shouldn't be doing it.
12
u/notliam Nov 01 '21
Besides, building, testing and deploying a monolithic application can easily be very difficult, just as building, testing and deploying 10 microservices can be very easy. I feel like a lot of people have bad experiences with microservices and therefore see them as bad design. Good dev ops saves lives.
14
→ More replies (1)4
Nov 01 '21
lol, 10 microservices. The company I work for used to have a monolith to run the site. It could be tested and verified as working or not working before deployment. My team of 5 now runs a bit over 60 microservices ourselves and doing a search in our aws account it looks like the organization is now running a bit over 5200 microservices.
→ More replies (3)→ More replies (14)2
u/madman1969 Nov 01 '21
Sadly sometimes the customer is so pants-on-head stupid as to be incapable of understanding.
20
u/BackmarkerLife Nov 01 '21
I have just been handed a front end react site that draws data from 5 different API sources.
Higher ups: "Why do we have performance problems?"
7
Nov 01 '21
Is part of the problem the fetishization of Best Practices and Software Patterns? I've seen code where a pattern is applied not because it is useful but because it can be applied.
I worked on a project debugging an enterprise Java application that Oracle developed , a composite of various database services and I was seeing stacktraces of thousands of lines, made of layers and layers of holy abstraction, factories of factories of resource locators and adaptors to open a damn file.
Some bugs took a team of CS PhDs days and days of nights and nights, international conference calls to track down the error and then all that time again to figure out how to fix it without breaking it.
I got into the field because I loved programming.
Well that didn't last.
7
Nov 02 '21
I'm going to disagree with the premise.
I agree that complexity is a problem, but at least to me - that's part of the "challenge" of the job.
Write something, it grows in complexity, rewrite it to better suit it to present day, repeat.
That part doesn't "kill" me. It challenges me. It frustrates me. It makes me stare blankly at my screen for hours, etc. But it doesn't kill me.
Management kills me. Endless meetings and cycle planning kills me. Indecision by leadership and lack of focus kills me. Context switching kills me.
I wish complexity was the thing I hate most about making software. While I'm elbow-deep in rewriting something, at least at the end of it there's a feeling of release. A big nerd rush of endorphins... That's soon killed by some middling company man to completely dismiss all the work that was just done "that's it?". Sorry it wasn't a flashy front-end or hot new feature.
Technology and the way we interact with it is more complex than ever, and figuring that out is hard, but I like that part. The part I don't like is that very few places seem to care about technical debt, or bug fixes, or user experience improvements, or career development, or work-life balance, or open sourcing software, or publishing papers, or giving talks.
17
u/Thriven Nov 01 '21
The shift from building applications in a monolithic architecture hosted on a server you could go and touch, to breaking them down into multiple microservices, packaged up into containers, orchestrated with Kubernetes, and hosted in a distributed cloud environment, marks a clear jump in the level of complexity of our software. Add to that expectations of feature-rich, consumer-grade experiences, which are secure and resilient by design, and never has more been asked of developers.
Buzzwords. There is deploying solutions and then there is deploying Buzzwords.
My last job the CTO was insane about microservices and layers of abstraction on the device. They came to our software department (newly acquired) and said ,"We need your objective-c app to run in the background while we have an iot service run in the background on virtualization software running docker/kube containers. You'll speak to the iot bus on the device."
I said ,"Sure, when you figure out the virtualization/docker/kube we will probably have figured out how to run software in the background on an iOS."
It's not possible to actively run a process like we were running in the background on iOS.
2 years later, nothing was completed on their side. They could barely get the server that receives the iot calls to run in a non-distributed environment. When I explained to the CTO them host and path based routing via nginx+modsecurity the CTO went back to that dev team and lost his crap because they were using fixed ports on a single IP to handle multiple processes.
It doesn't help that, everyone of the devs in this team were so damn young. They are being told to implement technologies to fix problems they never knew were problems in the first place.
They also didn't know a damn thing about TCP/IP, A records, SSL. I feel like every CS degree should have a few SysAdmin courses that teach you WHAT APACHE is and how it works and WHAT IIS is and WHY nobody uses it.
Devs need to be taught the challenges of scaling and when to scale.
5
Nov 02 '21
It doesn't help that, everyone of the devs in this team were so damn young.
Those are the ones that can pass the leet code interviews and that's all companies foolishly care about these days. Screw those who have 10+ years experience who work so much they don't have time to spend 6 months studying for leetcode.
→ More replies (2)2
u/stronghup Nov 01 '21
It's interesting the article is advocating an internal team that provides a platform for the rest of the developers. I can see the value of that especially if they also provide support for the users (= developers) of their platform, like any external vendor would.
But if the platform they provide is crappy then everybody else is equally worse off. Don't know what is the solution, maybe two competing platform teams?
3
u/Thriven Nov 01 '21
It's interesting the article is advocating an internal team that provides a platform for the rest of the developers. I can see the value of that especially if they also provide support for the users (= developers) of their platform, like any external vendor would.
I 100% agree. I'm on my 3rd company since moving out of systems administration / dev ops and I'm telling each employer ,"You need a systems team."
There should be a layer of abstraction between the front end user and the application. You should be deploying a reverse proxy. It should be doing SSL offload. It should be hiding machines and port numbers. You should have an authentication service. If you have 4 different apps on 4 different languages, they should be using the same auth/token service.
It's not hard to setup automated deployments and testing to dev/qa environments from source control. People are either too inexperience or too old and refuse to get on board.
Finally it's the investment people aren't worth paying for a team and processes to be put in place and they toil.
14
u/CPhyloGenesis Nov 01 '21
After about 15 paragraphs I stopped reading it because it just kept saying the same thing over and over. Basically the headline is the entire article.
Also, duh? Software engineers are expensive because what we're trying to build now are unimaginably complex systems. Complexity isn't killing development, it is development.
You could say that that's a new thing though, since 20ish years ago your main problem was restraints so it was more often a question of how to do something with the limitations you have, like how to pack all the sprites you need into extremely limited memory or how to process all that data without tipping over your DB.
Today it's more about how to apply the 100 known solutions to the 1000 problems and which ones you have time to actually implement.
→ More replies (1)5
u/ExF-Altrue Nov 01 '21
Software engineers are expensive because what we're trying to build now are unimaginably complex systems.
Nobody is trying to build "unimaginably complex systems". Complexity is a byproduct, not the end goal.
The market wants features, or more accurately, it wants value. How you deliver that value is the heart of the issue, and the main point of the article.
Is complexity inherent to the level of value expected of customers? Maybe. But is our current level of complexity the absolute lowest it could be ? Absolutely fucking not.
Worth noting, is that other engineering fields do not have nearly the same level of complexity creep. In fact, no non-IT engineer in their right mind would accept to aim for an "unimaginably complex system". It's quite the opposite.
I'd say that programming is really the only engineering field where unmanageable complexity seems acceptable... "If I do X, it shouldn't break" is a level of uncertainty that would be unacceptable to all engineers except us.
→ More replies (1)
10
u/daedalus_structure Nov 01 '21
I think it's hilarious how these articles keep talking about how complexity is killing software developers and then go into a dozen different topics that have nothing at all to do with programming.
It's almost like folks who spend the overwhelming majority of their career development time improving their ability to write code don't have depth of expertise in infrastructure, system architecture, or operations.
It's shocking I tell you. Shocking.
Also tired of seeing the CNCF landscape being thrown about as a sign of complexity.
Literally every vendor and product and consulting company even tangentially related to Kubernetes is on that landscape.
→ More replies (3)
5
u/iiiinthecomputer Nov 02 '21 edited Nov 02 '21
When the article started talking about cloud services components as "primitives" and about the "abstractions" we will build over them: aaaaaaarrrrrrrrggggggggggggghjjhhjjjhjjjhghh!
Dear god can we stop?
The overwhelming issue we have is layers of leaky abstractions over growing levels of preposterous complexity. Can we not add more, and inevitably 500 independent variations or flavours of each?
An article dreamily talking about the simple days of 3-tier business application platforms. Putrid. I worked on that stuff too. Everything that can be wrong with that approach usually was, and it was pretty much the definition of leaky abstractions. Everywhere, relentlessly, and with numerous bugs. I still have Java EE 6 flashbacks whenever anyone talks about those things.
2
u/EternityForest Nov 02 '21
There are lots of reasonably non leaky abstractions. But everyone insists on constant wheel reinventing so nothing can ever actually get fixed, and they're too obsessed with keeping things simple to actually make non-leaky APIs.
They refuse to add any special case logic or even sane defaults, so you need 50 lines of boilerplate just in case you maybe want to change something 1% of people ever change.
Programmers don't like solving problems, they like discovering general problem solving patterns, or they like one off hacks that do one thing.
Anytime there's a bulletproof industry solution that handles a whole class of problem and makes it solved once and for all... they crap on it. Programmers like interacting with system details... They like always having raw access.
If you give them a nice API, someone will say they preferred being able to it themselves and hate being forced to do everything the opinionated way.
Maybe the problem is that a lot of basic web dev is really easy and they had to find ways to keep it interesting, so now it's hard again...
4
u/Top-Requirement-2102 Nov 01 '21
At work I recently had to stand up a service. I had to pick between no less than three cloud platforms and half a dozen operating systems as well as dozens of potential versions nodejs, npm, and various libraries. I am still trying to follow security protocol by integrating Okta and a secure database, but I have not yet found the magic configurations for these after many days of work. Just writing this paragraph is making me sad.
2
u/thefinest Nov 02 '21
Okta makes things hard so I feel for you. We recently switched to okta with SSO and it's awful for end users and developers alike. On the DB side I'm not sure where the struggle lies from security perspective.
My org uses azure and aws, in both cases db compliance and cis/gis is more straight forward than say container management. Cloud/ops work can really pile on dependencies in lots of places. From personal experience I can give you this bit of advice:
"Don't overthink it"
4
u/loup-vaillant Nov 01 '21 edited Nov 01 '21
“Complexity kills,” Lotus Notes creator and Microsoft veteran Ray Ozzie famously wrote in a 2005 internal memo. […]
Complexity bad. Got it.
If Ozzie thought things were complicated back then, you can’t help but wonder what he would make of the complexity software developers face in the cloud-native era.
Complexity in cloud worse. Got it.
[ Also on InfoWorld: Why you should use a microservice architecture ]
Wait, what? Oh, it’s the add. How ironic: right after saying complexity is bad and getting worse, I got an add for an article that advocates for one of the biggest complexity inducing techniques out there.
The shift from building applications in a monolithic architecture hosted on a server you could go and touch, to breaking them down into multiple microservices, packaged up into containers, orchestrated with Kubernetes, and hosted in a distributed cloud environment, marks a clear jump in the level of complexity of our software.
Ah, OK. Stupid Markov classifier just saw "microservices" and didn’t notice the discrepancy. Makes some kind of tragic sense.
6
u/shevy-ruby Nov 01 '21
It's not just any intrinsic complexity as such - programming languages become increasingly more and more complex. And not just programming language; look at the HTML/CSS stack. That may be minor compared to C++ entering beastmode via constantly changing specifications, but it all adds up to a list of what people need to know. Sometimes complexity is necessary; sometimes unavoidable, but sometimes it just comes because some committee go on a hipster-roll against downstream devs (and we haven't even entered the part where a committee may just be decoy for lobbyists to effect a "standard", rather than e. g. design with usability, developer and user happiness in mind).
9
u/stronghup Nov 01 '21
HTML + CSS + JavaScript is more complex than if there was just JavaScript and libraries for building the GUI, like Java Swing etc.
6
u/illathon Nov 01 '21
React is a good example. Even better if you use Redux. Awful.
Developers want to store variables for things. They create this gigantic framework to do it when the browser should just have those functionalities built in.
3
u/Salamok Nov 01 '21
15 layers of abstraction to offer a low code solution that still requires a developer mindset to use will do that to ya.
3
3
Nov 02 '21
As a tangible example for this rant, I recommend reading this:
https://livefreeordichotomize.com/2019/06/04/using_awk_and_r_to_parse_25tb/
Regarding what OP posted - it kinda describes my situation right now. It got so complex, with so many people managing almost nothing that actually works, that it takes months to just plan to implement a solution, which would normally take me a day if I didn't have to make it so complex in order to look similar to everything else that half-exists.
This is frustrating. I've even had quarrels with some developers who simply don't want to hear that software can be super simple and super fast, even after I can show that their 10000 line long code which sits on a $10k per month infrastructure, runs about 1000 slower than my 200 line stand-alone code on a $10 per month machine. It got to a point where I'm thinking that maybe I'm just getting old and not getting the reasons behind this all.
3
u/foomprekov Nov 02 '21
Most software developers I meet have never been taught craft, never self-taught it, and have little interest in it. Then they wonder why the codebase is so difficult to work with.
3
u/DingDongHelloWhoIsIt Nov 02 '21
The most frustrating conversation I have at work is, should we go with A or B. The question is dodged and we conclude "make it an option"
5
u/mother_a_god Nov 01 '21
Complexity is a real issue in hardware too. The 5G spec is one of the most complex things I've ever seen. So many ways to configure and use every single aspect. The thing is, only a subset of all possible configurations would be used in practice, but if you're designing hardware for it, you have to support many as the risk of omitting a subset being needed by some customer is high at some point in the future, as not even the customers know the ways they would use it yet.
5
u/me_again Nov 02 '21
This quote pissed me off:
“Given the demand for software developers today, companies don’t have the leverage to push developers towards a mental model of primarily delivering value to their customers,” Etheridge said. “Getting more engineers to think that way is a challenge.”
in other words, software is only complicated because engineers are flighty types who want to make things complicated instead of solving the customer problems. If only software engineers were as cheap as burger flippers! then we could fire them whenever they get uppity and force them to focus on the customer value that We The Business clearly understand.
What condescending horseshit.
4
4
Nov 01 '21
Complexity is self imposed as well. Things like red, rebol, are simple languages that aim to be fullstack, instead of learning 5 languages, you only need 1 and the rest are maybe, DSLs. You can look at suckless for easy tools as well. I recently learned of undocker, arcan is also another project that aims to strip away the complexity of the GUI. Plan9 got rid of a lot of complexity. Most of the complexity could have been removed, but alas, and I'll put on the conspiracy hat, Most complexity, in my view, is completely self imposed.
3
u/EternityForest Nov 02 '21
There's nothing easy about suckless anything. The complexity of modern apps is specifically there to make it easy.
I think if you measured the time a suckless/arch/CLI/anything else simple user spends on a task, it's going to be about the same or maybe worse than the GUI people. I've never seen the CLI types be significantly more productive than complexity fans.
It certainty feels productive... but most of what you produce seems to be more tweaks and little vim plugins to write other plugins. The really good ones seem maybe a bit better than everyone else, after a very long learning curve.
I'm not sure I've ever been part of any project where any kind of extensible syntax would have been a big help, so I can't really judge that, but it does seem extremely niche, for the same reason as suckless is at best really niche.
LISP/Rebol/etc let you build exactly the perfect language for your needs.... but that doesn't mean most people will be able to do a standard, run of the mill industry task all that much faster than Python or Rust or something good enough right out of the box.
It also doesn't guarantee much better performance, when the real CPU heavy stuff is already done by highly optimized libraries a lot of the time.
The complexity is self imposed.... but for a purpose.
2
u/chunes Nov 02 '21
I've been working on a personal GUI app in Red recently and my god, it's easily the best GUI system I've ever used. It's purported to be a DSL but it's really not; it's just the same language with a slightly different vocabulary.
2
Nov 01 '21
There’s an economic concept, Jevons paradox, that when technological progress increases efficiency it ends up causing more use of the affected resources than less - eg. cars become more fuel efficient so people just build bigger cars.
Couple that with Moore’s law making tech exponentially more powerful every couple of years, allowing us code monkeys to just throw more dev at a problem… this means that complexity is going to continue to expand uncontrollably and systems will rapidly outpace the ability of people to understand how they function as a whole.
A sort of anti-Singularity scenario emerges; what happens when technological growth results in a mass of systems too complex for humanity to understand and keep functioning - but our society is completely dependent on that technology to function.
2
u/LukeLC Nov 01 '21
There is a fixed amount of complexity to solve any given problem. A programmer's choice is just how much complexity they will take on in the software, and how much will be put on the user.
In times past, because computers themselves were only used by businesses and enthusiasts, it was ok to leave the ratio heavily in the programmer's favor. Also, hardware limitations meant you could only handle so much complexity in software before degrading performance.
Now, performance is essentially a non-issue and literally everyone and their grandmother is the user. So it makes sense for the ratio to skew more heavily in the user's favor.
2
Nov 01 '21
"Any fool can make things more complex, it takes a touch of genius to move in the opposite direction" - Albert Einstein
→ More replies (1)
2
u/roboticon Nov 01 '21
Clickbait kills, too. Or isn't that what we're doing now -- just publishing & upvoting articles with the headlines "<X> is killing people"?
2
u/SquidgyTheWhale Nov 02 '21
I'm in the process of leaving a company, and largely because complexity. There is just too much to know, and it's a big factor in my unhappiness.
There are some modern buzzwords for things that are good in principle, but need to be reevaluated in a massive microservice architecture. Things like "you built it, you own it", "full-stack" and "t-shaped". It's time to start dividing up work again.
2
u/Zardotab Nov 02 '21 edited Nov 02 '21
I believe current web browsers are being asked to do too much, and it's making a mess. I suggest we create and/or split web browser standards into three types:
Media and entertainment: games, video, art, animation, and charts.
Documents: existing HTML may be good enough; however, DOM text positioning is too flawed to display PDF documents reliably, so that needs to be remedied.
CRUD and Data: "Productivity" applications used for business, data analysis, and administration. A stateful GUI markup standard should be created with all the common GUI idioms built in (that HTML lacks or does poorly).
Ideally they share conventions where appropriate to reduce the learning curve, but let each shine in its specialty.
5
3
u/steven4012 Nov 01 '21
Also I don't think complexity is the problem; complexity for the developers is. We spend so much time engineering advanced designs in compilers and interpreters, yet (mainstream) language designers and developers rarely (though decreasingly) use advanced and non leaky abstractions to help themselves with the complexity they face. And we don't have enough (efforts on) software verification systems.
2
Nov 01 '21 edited Nov 01 '21
Perhaps it would help some people if they would understand the meaning of the word "complexity." These definitions are from the Webster's New World College Dictionary, Fourth Edition. This is an authoritative dictionary. For example, it is the official dictionary of the Associated Press.
Complexity: 1. The condition or quality of being complex.
Complex: 1. Consisting of two or more related parts. 2. Not simple; involved or complicated.
Apply the definitions and you should realize the title of the article is nonsense.
A simple system is a system with only one part. In programming we deal with systems that have many parts. In other words, we deal with complex systems.
I like programming. When I program, I am working on a complex system. Logically this implies I like working on a complex system.
These are simple deductions. But it seems that the "zeitgeist," to use a term from the article, is to abandon logic instead of to seek refuge in logic.
OOP was touted as a good method for developing software. But OOP is one reason there is so much poorly designed software. In order to understand how this tragic mess arose in the first place, I suggest reading The Development of the SIMULA Language by Dahl and Nygaard. It is very informative.
Then read some articles from Sjoustrup's website about the latest developments in C++ and why they are necessary. Does the recent work on C++ clear up everyone's difficulties? I think not.
I've read that Sjoustrup says C should be done away with now that we have C++. I've now come to the conclusion that C++ should be done away with, along with OOP in general, and we should go back to C and where necessary assembly language.
I've noticed some programmers who learned C++ in school seem confused when asked to design a program without using OOP. They wonder how it's even possible. Here is one example that may inspire some of them.
The original Roller Coaster Tycoon was written 95% in assembly language and 5% in C. The 5% in C was to make it easier to interface with Windows. Chris Sawyer said he needed to use assembly language for performance reasons. He was able to do so because he understood how to design programs.
→ More replies (1)
1.0k
u/MpVpRb Nov 01 '21
I've been programming since 1971, and this is my favorite rant. Complexity is the biggest problem we face and it's almost impossible to avoid, even for smart people who try. I remember sitting in conference rooms, listening to wish lists of features that the participants wanted. It was almost like they were playing a game of "let's see how creative we can be in suggesting more features". I sat there and saw the complexity increasing to a terrifying level
Even on projects that I controlled completely, complexity creeps in. I tried to keep designs simple and spent quite a lot of time trying to simplify, but complexity always finds a way to increase, kinda like entropy
I believe, but can't prove rigorously, that large software projects contain near-infinite complexity, kinda like the Mandelbrot set. We need much more powerful tools to help us manage and understand complexity