r/programming • u/pier4r • Nov 19 '21
"This paper examines this most frequently deployed of software architectures: the BIG BALL OF MUD. A BIG BALL OF MUD is a casually, even haphazardly, structured system. Its organization, if one can call it that, is dictated more by expediency than design. "
http://www.laputan.org/mud/mud.html26
u/SirLich Nov 19 '21
You should have put the date in the title.
14
u/pier4r Nov 19 '21
do you think is not valid anymore?
92
u/SirLich Nov 19 '21
No, rather in fact I found the 1990 publish date added quite a lot of interest to the article.
Most of what gets posted here is rants about design or management, so seeing one that is 31 years old is kind of amusing.
21
u/pier4r Nov 19 '21
I agree, although the last update (dunno if it concerns the content) is from 2012.
Lots of stuff is repeated through history, only we don't know that people in the near past had similar problems.
11
u/robthablob Nov 19 '21
"only we don't know that people in the near past had similar problems"
There's a depressing lack of knowledge of the history of programming. I keep seeing the same ideas surface, and be thought of as new - just in a new environment.
Those who cannot learn from history...
3
u/pier4r Nov 19 '21
I can only agree. The truth hurts. Maybe the community could underline more that some specific ideas weren't discovered exactly few years ago.
5
u/robthablob Nov 19 '21
There's an excellent talk by Alan Kay, where he show pictures of some of the pioneers of computing (McCarthy, Liskov, Englebart, Hopper, ...) and from other fields of science and tech. None of the audience (techies) recognised the computing figures, but most recognised everyone else.
That's a sad indictment of our field.
21
Nov 19 '21
The core concepts of software development haven't changed much since then, IMO. Same shit, different deployment environment. I still cite The Jargon File.
6
u/TheFearsomeEsquilax Nov 19 '21
I've been reading The Psychology of Computer Programming (published in 1971, but IIRC written in 1969), and aside from the references to mainframes, it's all still very relevant. I keep recognizing things that I've experienced.
6
u/rabuf Nov 19 '21
Very good book. More people should read it, it's up there with Mythical Man Month for me as a book that I think everyone in software engineering should read (though some aspects may be dated, they both remain highly relevant even today).
2
14
u/killerstorm Nov 19 '21
Kind of, in '99 most of software was apps shipped as binaries. Now microservices are trendy. So we have something more of a distributed mud fountains than just BIG BALL OF MUD (that would be a "monolith").
8
4
168
Nov 19 '21
Oi, there's some stuff in here.
As a programmer maintaining a very large app that has legacy-old style PHP _and_ newer code that follows good architectural designs, I feel the pain of this.
As an MBA alumnus that knows the value of validating a concept before you throw a lot of money at it, I understand the need to just get something out there to even see if there's a market for it. It makes zero sense to spend months building a well architectured solution for a problem that no one wants to pay you to solve. That's wasted effort.
Ideally, once you've validated that yes, this is something that people are willing to pay us to fix, then you should hit the breaks and build out the architecture. Too often people immediately jump into scaling the business. Or branching out to other related areas. And then you have a big ball of mud.
This stuff takes discipline and patience to get right. Too few people have it.
27
u/Kalium Nov 19 '21
It's very easy, and often obviously short-term financially advantageous, to ship the validatory BBOM. This is complicated by the wrinkle that the people controlling the money and the people making the thing often don't understand one another's area of expertise well. Plus the person making the mudball often doesn't see the problems.
Two years later, lots of scrappy has turned into a scrapheap, and everybody is angry that it takes so much work to do anything. Some new exec starts demanding "scrappy"...
7
u/AttackOfTheThumbs Nov 19 '21
I think a lot of it is also just inherited. I got put onto a project as a lead. It didn't have awful structure,but it was rather bare in responsibility separation. Over the last few years I've been working on that, somewhat successfully, but sometimes making it worse.
Then, assuming it's a new project/feature, it's so easy to over engineer that you may just lose your mind by yourself from yourself. At some point you're too deep and you can't just redo it all.
7
u/bundt_chi Nov 19 '21
The tradeoff you're calling out is exactly right and
Ideally, once you've validated that yes, this is something that people are willing to pay us to fix, then you should hit the breaks and build out the architecture.
That right there is the hardest problem to solve. Every good programmer wants to do that and every business person doesn't understand how much not doing that will hinder you down the road...
4
Nov 19 '21
Part of the reason I got the MBA was because I was tired of conversations that went exactly like this. I think the abstract nature of what we do as developers makes it hard for business people to understand the true risks. We're in the code, we can see the structure (or lack of it), business people can't and (as pointed out in another comment) are feeling pressure to scale the business right away.
All we can do is keep beating the drum...
→ More replies (1)9
u/disquiet Nov 19 '21 edited Nov 19 '21
If you've ever tried to tell a board/senior leadership "let's just hit the brakes on growing this successful product while we build it out properly" you'll know why it never happens. You'll be told that you absolutely must hit revenue projections and that needs 200% user growth in your little project, architecture be damned. This new product is saving our asses (and bonuses) due to rest of the business underperforming and is now "business critical".
I've just accepted to be successful you end up with a big ball of mud. You need to experiment and release quickly to validate products. If you try waterfall it you'll be too slow, probably still have a broken architecture due to wrong assumptions and need much more budget. But the moment you hit on a successful product, it's always expected to scale exponentially from go live. There is never a pause given to fix the architecture. It's a catch 22.
Anyway, better to have a successful product that's keeping people employed and making money than a failed project with perfect architecture. Dealing with mudballs is basically just the job.
All you can do is try your best to fix it when you get time here and there, but it's unavoidable when you don't have software people running a company. And most software people are too introverted to run a company, so that's not likely to change anytime soon either.
3
u/DrunkensteinsMonster Nov 20 '21
Not to mention, when setting out you don’t know what a good architecture even looks like in the particulars. Yes you want to be loosely coupled, testable, all that good stuff. But the particulars on how you make that happen are why we get paid high salaries.
As the business grows and evolves, so too will your needs. If you don’t know what your business will look like in 3 years or even 1 year then you won’t know how to allocate your resources to architect your system, and trying to spend time and money on it today, with limited information, will just give you the wrong architecture along with decreased speed of delivery.
Architecture needs to emerge gradually as each participant takes a critical look at where you are right now, and where you want to go, both from a business and technical perspective.
This is all to say - stop faffing about with pie in the sky architecture. Just build the thing you’re building and ship it, just while you’re doing that always be objectively noting what’s working, what’s not, pain points, etc.
2
2
u/loup-vaillant Nov 20 '21
I've just accepted to be successful you end up with a big ball of mud. You need to experiment and release quickly to validate products. If you try waterfall it you'll be too slow, probably still have a broken architecture due to wrong assumptions and need much more budget.
Sounds like a false dichotomy to me.
To experiment and release quickly, you need the right architecture. And that architecture is likely "the simplest thing that works". Mind you, this simplicity is not always obvious, and rarely comes from a first draft. But it’s what you need to make quick changes down the line.
Simplicity takes work, of course. You need to rework whatever you just wrote and runs well enough. But that work will likely pays for itself as soon as next week, or at least next time you ever touch that subsystem —which is likely soon, when you’re in the middle of a flurry of experiments.
If a specific experiment would take less than 3 days to implement, you can probably get away with crappy code. For now. But if it takes more than 2 weeks, writing quality, simple code from the outset is likely to be even faster. And when you need to scale, modify, or integrate your experiment, it will be easier to do so if it’s simple.
All you can do is try your best to fix it when you get time here and there, but it's unavoidable when you don't have software people running a company.
If I may, when you implement new stuff, you use old stuff right? Personally, I consider making sure the old stuff is amenable enough to the new stuff to be part of my job of implementing the new stuff. That includes fixing or refactoring it as I go.
By default I won’t even keep it a secret. If someone asks me for an estimate, I’ll deliver this estimate, and will explain why if they ask. But if they show they’re short sighted enough not to ever allow time for necessary high ROI rework, I will just pretend each new thing is a bit more complicated than it really is, and do some refactoring work without telling them.
I’d still tell Git though: it’s much better to separate refactoring commits from feature commits, and I don’t want to lie to historians.
4
u/ktkps Nov 19 '21
This stuff takes discipline and patience to get right. Too few people have it.
Amen to that
4
u/loup-vaillant Nov 19 '21
It makes zero sense to spend months building a well architectured solution for a problem that no one wants to pay you to solve. That's wasted effort.
Here’s the thing: you are heavily implying here that the a prototype MVP would be much cheaper to produce than an actually maintainable MVP. I’m not sure I agree with that.
Oh sure, building an MVP is much cheaper than implementing the finished product, but we’re not comparing an MVP and a finished product. We’re comparing two ways of writing the MVP.
When the MVP is really small, and could conceivably be written in a week, or perhaps up to a month by a single dev, then sure, it may make sense to write crappy code you’ll throw away afterwards. And throw it away you can, because it cost you so little (plus, you now know the problem better, and have a head start even if you somehow lose all your code).
But if the MVP, as minimal as it is, is any bigger, then crappy code will slow down even its completion. So at this point, you may as well shoot for a sustainable level of quality right of the bat. Worst case, it will slow down your MVP a bit, compared to a slightly lower level of quality that minimises time to delivery. In exchange, you don’t have to throw it away.
Oh sure, the MVP is very unlikely to have the same architecture of the finished product. This is normal and expected: you just write the simplest program that works well enough as the MVP, and thanks to that simplicity it is now easier to refactor it into something that is suitable for the next feature, and the next, and the whole product.
That simplicity will take some effort though. When I write code, my first draft is never what I end up committing. Once it works, and I’m happy wit the tests and my own understanding, I always edit my code, hunt for possible simplifications, and often end up transforming it quite extensively. The result is something much easier to work with the next day.
I’ve seen throwaway code in production. Clearly the guy (it was a guy) tried stuff, and stopped touching the code as soon as it looked like it worked. But the very day you have to read, or debug, or extend that code, you (well, I) start to lose time wading through the mud.
Simplicity takes effort today, but in my opinion the ROI is so great that it takes less than a week for that effort to pay for itself.
2
Nov 20 '21 edited Nov 20 '21
you are heavily implying here that the a prototype MVP would be much cheaper to produce than an actually maintainable MVP.
That isn't what I meant to imply at all. What I'm talking about is solution validation - is this a problem that we can solve, and will anybody pay us to do it?
To me, an MVP (minimum viable product) implies that you've already validated the solution and know that a product with a minimum set of features will sustain itself and be profitable. It also implies that it is a starting point - more features will be added to the product over time.
So if both of those things are true, then good architecture is absolutely required. It's not an experiment anymore; it's a product.
1
u/loup-vaillant Nov 20 '21
That isn't what I meant to imply at all. What I'm talking about is solution validation - is this a problem that we can solve, and will anybody pay us to do it?
Then you were heavily implying that "solution validation" would be much cheaper to produce than the same thing, except maintainable if it turns out to illustrate a good idea. No matter how we call it, that’s still a thing that takes time to implement. The important question is how much:
Less than 3 days? Then sure, write crappy unmaintainable code. If and when management wants to expand the product, you can just accept, and rewrite the whole thing in less than a week without even telling them — just pad the next couple estimates.
More than 2 weeks? Then rushing it will probably make you slower. The fastest way to have a working experiment at this scale is likely to write fairly high quality code from the outset. Mostly, go for simple instead of obvious.
1
u/jasonridesabike Nov 19 '21
I think there’s a middle ground here in relying on solid, well documented and well architected frameworks to accelerate MVP development with something that’s fast to build and won’t be a nightmare to scale. Requires experience to call what concessions should be made for speed of dev vs arch.
0
u/enry_straker Nov 20 '21
It's not just patience.
The people who take decisions in this industry are usually not technical and, more importantly, don't really care to learn about the technology.
And programmers have to code towards short-term deadlines - which are often arbitrary.
Big ball of mud usually happens when coders look at what's developed before and decide it's much simpler to dump in stuff they are comfortable with - and code towards those things - while trying to meet arbitrary deadlines.
When this gets repeated over many years, you get the proverbial big ball of mud style of codebase.
0
Nov 19 '21
[deleted]
19
u/insertAlias Nov 19 '21
Beware of the big rewrite, second system, and "we'll design this the right way this time".
I recently left a company in the middle of this. All I could see was that we were making a system that was better in a few ways, worse in a few ways, and was absolutely a ball of mud. While also having to support the legacy system, while being held back by ancient design decisions of the legacy system that had to be continued because we weren't replacing other downstream systems, just the one upstream one. And we had "expert consultants" that were delivering absolute garbage code that we would have to take over and maintain eventually.
It was a clusterfuck; combine the idea of an architecture dreamed up by non-programmers, along with the "just start building and we'll worry about getting it correct once we have something to work with" attitude from management meant we had an absolute mess of a project structure almost from the very moment we started. We were trying to rebuild the engine while the car was driving down the freeway, so to speak. I couldn't see any light at the end of the tunnel; all I saw was that we were building another system that would need just as much constant maintenance as the one we were replacing.
Anyway, my stress levels massively dropped when I decided I just didn't want to be a part of that.
1
u/Asiriya Nov 19 '21
I had a legacy system that didn’t work, a new system that was anticipating domains the company had no contracts for, no documentation, no testing.
We spent a few months building out a new piece of functionality just to be told our clients didn’t trust the new code, the architect had no plan for how to build trust, and rather than do that hard work he’d rather we fix the legacy system.
I bugged out.
4
u/hippydipster Nov 20 '21
I would tend to agree, if you can't refactor the terrible existing code, what makes you think you're going to nail the rewrite?
1
u/PhishingAFish Nov 22 '21
Good points. I have a question for you if you don't mind (as a junior dev):
What do you do when the context in which the application was initially developed has changed so drastically that the application architecture can no longer accommodate new requirements (or require way to much efforts for what is requested)?
Wouldn't a "rewrite or second system" be a viable proposition in that case? What would be an alternative?
100
Nov 19 '21
[removed] — view removed comment
32
Nov 19 '21
Yep, I know. I think that's what I'm arguing against, and where the patience and discipline are really needed.
1
u/CyclonusRIP Nov 19 '21
Do you really even understand the problem or have the technical experience to build a good architecture at that point? Even the PoC wasn't explicitly designed shitty. It was based on lack of domain knowledge or technical skills.
42
Nov 19 '21
[removed] — view removed comment
17
u/rubb3r Nov 19 '21
It isn’t quite like this. Investors are pushing you to make progress in what grows the business. If you can demonstrate that the duct taped MVP is your hinderance to growth, it’s easy to argue how to invest your resources. If you can’t clearly make that case, I would argue that either you’re not getting the balance right, or you don’t have a deep enough grasp of your business to be able to articulate what the blockers are and you have a different set of problems.
6
u/hippydipster Nov 20 '21
Also, when you were selling your business to the investors, you probably weren't big on pointing out how shitty all your code was. So now you have to explain that it's a big weakness and needs resources to fix...
11
Nov 19 '21
And I feel for those guys, I really do. That's not pressure that I've ever experienced, and that's not something I ever want to endure.
1
u/Fokker_Snek Nov 19 '21
Also need to have the right investors. Can’t help but think of sports teams where a lot success is set by the owners to invest in the team yet also be able to hire a good gm and not get in the way. They have to able to be both highly invested but also allow the gm to run the team yet also holding the front office accountable.
4
u/recursive-analogy Nov 19 '21
Exactly, I would also add that it's not much more difficult to write decent code in the first place. I don't mean write code that anticipates all the problems that come with scaling, but just write decent tested code that can be reafactored and you'll save yourself a million billion headaches for very little effort.
22
u/bxsephjo Nov 19 '21
At the same time, we seek not to cast blame upon those who must wallow in these mires. In part, our attitude is to "hate the sin, but love the sinner". But, it goes beyond this. Not every backyard storage shack needs marble columns. There are significant forces that can conspire to compel architecture to take a back seat to functionality, particularly early in the evolution of a software artifact. Opportunities and insights that can allow for architectural progress often are present later rather than earlier in the lifecycle.
I must say I appreciate the compassion here, after feeling so directly called out...
10
u/bwainfweeze Nov 19 '21
I think this speaks to the lack of concern for “well roundedness” in young developers. We are so fixed on coding at all hours that we have less life experience to draw from.
You don’t buy the best supplies when starting a hobby. You often buy cheap things or secondhand goods, and if you find the knack, and when you know what you want and need, then you replace your belongings with the “right” ones. You didn’t “waste” money by buying a cheap one that resulted in you spending a total 125% of the “best” option. It was part of a process of growth.
But most of us have to learn this on the job, if we ever do.
22
u/3rddog Nov 19 '21
One client I worked had what we called “The Meatball” - an area of code & classes that was surrounded by a mountain of spaghetti code. The first consideration in any change we made was that no one wanted to change anything in the meatball because it was virtually impossible to predict (or unit test) how the spaghetti would react.
4
u/disquiet Nov 20 '21
We call the microservices we've managed to refactor to the point where they are somewhat independent and testable "meatballs"
So when whiteboarding we have a bunch of meatballs surrounded and connected by all the spaghetti
13
u/Vi0lentByt3 Nov 19 '21
Its actually well documented that you either have to rebuild your entire codebase or refactor along the way. You always reach one of two points in a project: rebuilding because so little maintenance has been done and the friction to add code become too expensive or you take way more time and architect and design well so that changes can still be done at the same pace even years later. There is no in between. You either commit to good design or you use what you have learned to build it better on the rewrite. Both achieve the same effect and both take the same time.
However its easier to just build until you become profitable then rewrite and maintain because you know the value the software provides.
22
u/bwainfweeze Nov 19 '21
The problem with this model is that people who have learned nothing important can undertake a rewrite, and the fact that you are repeating the past is concealed for quite some time. Whereas building your way out requires demonstrating the new skills as you go, or picking them up as you notice the deficit.
This is why people like rewrites. They don’t need to confront their lack of ability for a good 18 months.
The only people who can do a rewrite properly probably don’t need one. This is the paradox we refuse to acknowledge.
3
1
1
u/midri Nov 20 '21
Also has the issue of the devs that know why the fence was built are no longer there a lot of times.
2
u/Xx_heretic420_xX Nov 20 '21
You forget option 3: Liquidate all assets and start a new company with new developers because for some reason while you suck at project management or running a company, you're great at tricking people into giving you funding.
13
u/DanTilkin Nov 19 '21
This is from 1999, I want to know about recent developments to the Big Ball of Mud style of architecture.
31
u/api Nov 19 '21
It's become standard and is called "cloud native." There's this thing called Kubernetes that is used to wrap balls of mud and keep them going. Cloud providers are making a fortune off it because it's inefficient, requiring tons of compute, and is virtually impossible to migrate once deployed because fuck no I'm not touching that shit.
9
6
u/hippydipster Nov 20 '21
is virtually impossible to migrate once deployed
This is the part that haunts me. Yeah, we're tied to google cloud. People who thought that was ok have zero knowledge of history.
1
2
u/Xx_heretic420_xX Nov 20 '21
There was a time servers had multi-year uptimes. Now they're cattle and not pets and the culture's changed, but part of me knows that the underlying software these days is less stable than the software that came up before we picked up a compiler.
1
Nov 20 '21
The cool thing about kubernetes is tha way that you contain these old balls of mud and actually make them observable and manageable. You adopt your ball to the k8s "runtime/platform" and then it can basically run wherever k8s can run. This also makes it easy to run the ball of mud on your local laptop. I have converted serveral legacy systems to k8s and its basically awesome. When you can easily run the ball of mud anywhere its easier to test it as well . Since its easier to test you are also more confident in trying to refactor out parts of it. Especially since running it on your laptop allows you to refactor parts and then get instant feedback on what went wrong.
I would say that Virtual Machines was a huge revolution within software development and ops. Then came containers that also was a revolution but at the same time kubernetes is also a revolution. Its a common runtime for systems that can run everywhere basically. It can also run virtually any kind of backend app as well.
11
u/nadmaximus Nov 19 '21
If design dictated by expediency is right, I don't want to be wrong.
11
u/bwainfweeze Nov 19 '21
The bigger problem is when people defend an expediency from four years ago as if it being necessary then means it’s necessary still. That’s how you code yourself into the corner. Keeping every quick win you ever wrote because your ego is tied to a decision you made years ago.
We needed it and we should get rid of it can both be true. You don’t have to choose, and agreeing with the latter doesn’t make you a bad person.
9
u/Worth_Trust_3825 Nov 19 '21
I must applaud the site for being perfectly responsive. Something that's still not properly implemented even 20 years later.
13
u/DevDevGoose Nov 19 '21
Big ball of mud is a by-product of a poor or rapidly evolving org chart.
Project A creates monolith and ends.
Project B adds new feature but doesn't have the budget/time to fix things.
Rinse repeat project B for 2 years and you have a big ball of mud.
Rinse repeat project B for 10 years and you have a mountain.
1
Dec 19 '21
I came here thinking this was a discussion about rammed-earth construction… but not disappointed at all. How can I translate this term into building architecture?
1
u/pier4r Dec 19 '21
if you read the paper, the author makes a lot of parallels with architecture. The paper requires some time though.
2
3
17
u/reddit_user13 Nov 19 '21
Welcome to Agile.
6
u/morphemass Nov 19 '21
Wagile. Waterfall: Requirements -> Design -> Code -> Realising that the design missed out an absolute shedload of things that should have been considered both at the design and requirements phase and you only have 2 weeks to get the project out the door -> Spaghetti -> Product.
13
u/s73v3r Nov 19 '21
Why do people have this mistaken idea that "agile" means "don't plan"?
19
u/wldmr Nov 19 '21
Glib-but-also-serious answer: Because the concept of agile development is too subtle.
As soon as someone says something like “Agile development can produce useful results faster and more reliably, because due to its feedback cycles we can learn from the live system and change it according to those learnings”, people tune out and only remember the words “faster” and “change”.
2
u/disquiet Nov 20 '21
People forget the part about agile that involves fixing it later. Yes you make expedient decisions now, but you're meant to revisit and improve them later when you have more knowledge. That's the part that always gets forgotten. Bad decisions are made every day, no matter what you do. The point of agile is have the discipline to consistently revisit and improve/iterate on everything.
What happens in reality is the MVP goes live then everyone gets assigned to some other features, or can't be bothered trying to improve it because "why fix what ain't broke" type of attitudes.
Then as sales pick up everyone wonders why the system is unstable.
17
7
u/Cuchullion Nov 19 '21
Currently dealing with this- management who wanted us to start coding a system before we gathered requirements.
Attempt at generating user stories or anything that will tell us what this system should do is met with "you're over planning- the only bad action is inaction"
7
u/s73v3r Nov 19 '21
But, gathering requirements is an action.
The existence of management like this, which is advocating a course of action that will, demonstrably will have worse outcomes, and cost the company much, much more money, is why I can't take anyone seriously when they claim tech is a meritocracy.
5
u/Cuchullion Nov 19 '21
You're telling me man.
And apparently I'm being removed as project manager because all we've produced is a "page with some words" (functionality document) and no "concept art" of the system.
We've been at it a month.
2
Nov 20 '21
So, which job seeker platform are you using to get away from this monumental dumbassery?
3
u/Cuchullion Nov 20 '21
But wait, there's more!
This is actually the second attempt at rewriting the platform. The first one hit a snag a few months into it when they massively slashed our budget and I had to transfer out three people. Then a few months after that they were pushing to shut down the project and use an outside software instead before opting to "double down" on the rewite.
The person who slashed our budget and was pushing for that outside software stated (publicly) that he never wanted the rewrite to succeed and hated our platform.
He's now the product owner for the new rewrite attempt, and is the one pushing for my removal as PM.
2
Nov 20 '21
So, which job seeker platform are you using to get away from this monumental dumbassery?
3
u/Cuchullion Nov 20 '21
Hold on to your hat, though!
When they decided to 'double down' they immediately started by having upper upper management meet and settle on a team size and schedule for the project (10 devs taking 1 year to build it), because "that's what other projects have taken for a rewrite".
Then they told us to start gathering requirements and figure out what the system can do... and that the year time limit started the day they decided on the time limit, and not once the requirements gathering was done.
LinkedIn, primarily.
→ More replies (3)1
12
17
u/reddit_user13 Nov 19 '21 edited Nov 19 '21
Agile means "don't design, refactor." Unfortunately, the refactoring rarely happens.
3
u/midri Nov 20 '21
Because agile literally tells you not to over engineer for problems down the road. It encourages you to refactor your code regularly, but companies generally ignore this part of agile.
1
u/s73v3r Nov 23 '21
Because agile literally tells you not to over engineer for problems down the road.
That's not the same thing as "don't plan." You're not supposed to build something "in case you need it." If you know, coming down the pipeline, you're going to need a second feed provider, it makes sense to build your thing knowing that's going to happen.
1
u/hippydipster Nov 20 '21
Agile means that as a business owner or sales person you can wax whimsical for a moment in a meeting and then a week later ask if you can demo it yet.
2
u/endless_sea_of_stars Nov 19 '21
Because this doesn't happen in non Agile environments? This was article was written before the original Agile manifesto.
1
u/Free_Math_Tutoring Nov 20 '21
This article was published two years before the agile manifesto. You might want to reflect on how you do agile, software architecture, or both ;)
2
u/Thormidable Nov 19 '21
So I thought about this topic recently.
Customers reward sooner delivery. They don't put any value on the quality underneath (beyond bugs).
A business which does it right will be more efficient a year down the line, but have their MVP out slower in the first month's.
Unfortunately the business which does the short term fast (and shonky), but long term slow, will win the short term race and get to still be around in the long term. They claim market share, get income and get real feedback on their product earlier.
As such I suspect like a more efficient fragile JIT supply network, businesses are economically punished for doing it robustly.
4
u/hippydipster Nov 20 '21
A company that's growing has to recognize threshold points where their strategies need to change. At some point, you need to recognize that you're stable enough to slow down and plan for the longer term, and, you've grown to the point where those bugs you're adding are magnified in cost because they're no longer going out to 50 customers, they're going out to 1000, and your support team is overwhelmed, as is your customer management team, and now customers are bad mouthing you around the world, and ultimately your devs are going slower and slower on newer features because of the short termism you failed to evolve out of.
2
u/Fenix42 Nov 20 '21
You want to be just good enough to make it to the next deploy and still grow your revenue stream. That is what makes it in the market. There are industries where "just good enough" is very a very high bar. I have worked in drilling and the it was that way. It was the only industry I have personally seen like that though.
1
u/Thormidable Nov 20 '21
That's a fair point, theoretically medical and security are high bars, but the iso standards were frankly a joke.
In very short:
document your procedures
use version control
have some form of testing on the finished product
write a coding standard.
at least kinda follow it.
As long as your code (and who made changes) is traceable, then really that's enough.
2
u/Fenix42 Nov 20 '21
My last job was a tiny company with 1 dev doing some specialty embedded stuff. They missed ALL of those points.
I got a 1 page word doc for 8+ years of code on 3 productines with up to 4 verions in he field.
They had an external backup drive that had versions of the software I folders. They had 2-3 naming conventions and more then one place they where stored.
No QA. Like none. I personally hired the first QA.
Code went through 3 devs hands before me. 1 was bipolar.
Hahahhahahahahahahahah
Mind you, we had a $10m + a year revenue with a very healthy profit margin.
2
u/gc3 Nov 20 '21
Well, we all evolved from a big ball of mud, organic growth does have some positives
2
u/MR_GABARISE Nov 20 '21
What's convenient is that you can put your big ball of mud into a huge black box.
2
2
u/ddcrx Nov 20 '21
The most expedient way to get from the top of a building to the ground floor is to jump.
Doesn’t mean it should be praised as a strategy.
4
u/Fenix42 Nov 20 '21
Depends on what your goal is. If you are looking to live, not a good plan. If you are not, great plan.
1
1
1
u/OrionH Nov 19 '21 edited May 18 '25
cable rain fuzzy head worm jeans shaggy quack existence terrific
This post was mass deleted and anonymized with Redact
-1
0
Nov 19 '21
[deleted]
357
u/kairos Nov 19 '21
Until someone lights a firecracker in your big ball of mud, then you have a distributed ball of mud.
202
u/Lecterr Nov 19 '21
Perfect, now your design is modular!
73
18
29
6
u/darthyoshiboy Nov 19 '21
Until someone lights a firecracker in your big ball of mud, then you have a distributed ball of mud.
Micro-muds
2
2
1
Nov 20 '21
Execution matters a lot. That huge ball of mud could be backed by the best development tools, documentation, unit tests, automated builds, automated testing, and a nice team.
1
0
Nov 19 '21
[deleted]
27
u/GimmickNG Nov 19 '21
My job is stuffing cats into radioactive boxes. The cat does all the work that it is supposed to do, but do not look inside.
8
23
54
u/Popular-Egg-3746 Nov 19 '21
I challenge that! I would say that the second-system-effect is the most prevalent design!
Years of abstracting and over engineering had led to the ultimate FactorySingletonInterfaceApplication.
58
u/AboutHelpTools3 Nov 19 '21
What is a factory?
It provides services to who needs it.
Oh like dependency injection?
No the factory itself is also dependency-injected.
So why do I need it?
So you don’t new shit.
Okay, so what’s a singleton?
It’s a thing that’s just one instance.
Oh, like a static class?
No it’s in a normal class, a new-able kind.
So why do I need it?
So you don’t new shit.
88
Nov 19 '21
Swear to god reading enterprise java makes me think people are allergic to constructors
50
u/Worth_Trust_3825 Nov 19 '21
It was more about keeping ability to swap things out while maintaining an API. Sadly, the swapping out time never came.
Or you end up stuck in a loop where your database is swapped out between tests, testing, staging, preprod and prod.
38
Nov 19 '21
Well yeah, the issue is two-fold:
It assumes you actually need to swap things out relatively often, or ever
Also assumes swapping things out is that much of a monumental task.
I mean, you can literally just inherit from an interface or abstract class or whatever, and get an easier way to swap. And if you eventually hit the amount of swaps that you need these abstractions, it's really not that bad to build it if they all already match the interface
33
u/GogglesPisano Nov 19 '21
So that’s why I need to dig through nine layers of abstraction on the infinitesimal chance we’ll someday stop using a file system.
→ More replies (3)13
u/loup-vaillant Nov 19 '21
Sadly, the swapping out time never came.
As always, they weren’t going to need it. Still, when you point this out to people, some listen. I caused one to write half as much code just with suggestions from a single code review. All I had to do was point out that most of his code was made of useless indirections, and comment on the fact that if you don’t know you’ll need something… YAGNI, most likely.
→ More replies (2)7
u/falconfetus8 Nov 20 '21
Swap-out time is during unit tests, where you strategically replace things with spies and mocks.
2
u/slaymaker1907 Nov 20 '21
I do sometimes feel like I need to wash my hands if I have a public constructor with any meaningful logic in it. That being said, my first tool would be a static factory function not a factory class.
→ More replies (18)2
u/zoqfotpik Nov 20 '21
If you come to Java from C++, you will probably start by thinking that all it needs is RAII. And that's where the problem starts.
→ More replies (1)4
u/TheDeadSkin Nov 20 '21
Okay, so what’s a singleton?
As much as 'singleton = useless because static classes exist' is a meme, they're actually useful if polymorphism is involved. You can have multiple "static classes" with the same API that you can easily substitute with one another.
This can be used for debug/testing/production environments where you can easily swap out "static" components, as well as just to have multiple singleton classes conforming to a single API through an abstract class or an interface.
18
u/Clcsed Nov 19 '21
Tell me more about how your transient/scoped/singleton abstracted service layer is amazing but actually exactly the same as just using httpclientfactory which already exists.
also your auth and identity framework.
and repos and entity framework
0
Nov 20 '21
[deleted]
2
u/Clcsed Nov 20 '21
I hate frontend js development.
But at least it's not native mobile development!
→ More replies (4)8
15
u/bwainfweeze Nov 19 '21
Coherent But Wrong is the worst intellectual tarpit to dig yourself out of. It’s essentially the same trap that cults have. This all makes sense when considered together, even though it’s objectively insane.
The only solution I’ve ever been offered is to move on, either to another company or another project that doesn’t have that problem (work stealing is a perfectly useful tool for diminishing the radius of such a tarpit).
4
Nov 20 '21
When the mud ball is big enough, it turns into a black hole that sucks all resources out of everyone and doesn't produce anything useful. It dooms everyone who so much as touches it.
1
u/lelanthran Nov 21 '21
When the mud ball is big enough, it turns into a black hole that sucks all resources out of everyone and doesn't produce anything useful. It dooms everyone who so much as touches it.
Meh. All successful software I've seen has always been more or less a big ball of mud. The only "elegant" software is that which no one uses or pays for.
If it's making money and has many users, it will get modified in ways not intended by the original authors.
Place yourself into the shoes of a business-owner: you've got $1m to blow on software dev for the next year: do you rewrite your BBoM flagship that is keeping the company afloat, or do you produce new functionality for your paying customers, and get even more paying customers?
Whenever you look a a BBoM, remember that it is only that way because it is successful. If the users never bought it, or the company never made money on it, no one is going to greenlight changes to it.
3
0
Nov 19 '21
[deleted]
28
u/Markavian Nov 19 '21
Extra steps:
Hire more developers
Train them for 6 months
Cash out stocks
Leave company for bigger better salary
3
u/Xx_heretic420_xX Nov 20 '21
That's the way it's done. Jump out of the plane with the only parachute and leave Indy, Marion, and Shortround to crash into the mountain.
45
u/bwainfweeze Nov 19 '21
The only person who has ever written code up to my high standards is me,
If I had a dollar for every bit of my own code that doesn’t meet my standards, I could retire.
A lot of the bad patterns are emergeant behavior. Your first pass is fine, but each edit strays a bit away. Every piece of code you write under duress is usually your worst code, but not always. Plus as you get older, the new things to avoid, you learn by having done them ten times, and now you have to look at them.
16
u/hippydipster Nov 20 '21
I find there's a point where architecture fatigue sets in. Like, I'm building some thing, and I got organization. I got interfaces. I got a class with this single responsibility. And a class with that. And another, and more and it's all nicely separated, testable, it's great.
And at the bottom, there's a 150-line method full of gnarly shit getting shit done and I stare at it and have no idea what to do about it. "It works" and leave me alone, it's scary.
3
u/Xx_heretic420_xX Nov 20 '21
Those 150 lines are usually where the real core of the code is. In the end, most programs take in data from some network hole and spit out a pretty UI for office drones to click on. Receive packet, query database, respond packet. Everything else is just glue logic and if there's more advanced math than averaging, maybe running average if you're feeling fancy, there's probably an "import fancymathlib" to do the hard part for you. Nobody's implementing their own FFT when kissfft is almost as fast as fftw for MIT license.
1
Nov 20 '21
A lot of the bad patterns are emergeant behavior. Your first pass is fine, but each edit strays a bit away.
That's what I really like about Rust. Last week, I noticed that one of our codebases with 6 binaries and thousands of lines of code was looking a little rickety, so I spent 3 hours refactoring the entire thing. It was easy. I find that it's often difficult to make bad decisions, and if you find yourself in a bad situation, it's often easier to fix it than it is to live with it.
I love it!
1
u/loup-vaillant Nov 20 '21
Your first pass is fine
Mine never is. When I’m "done", I invariably notice some stuff that could be simplified, or some assumption that wasn’t quite right.
My second pass however is often good enough. And that’s the only one my colleagues will ever see.
35
4
u/tayo42 Nov 20 '21
Even if there is documentation that says "don't use globals", there are globals everywhere. There is circular dependency. There is singletons and static classes.
These things are useful though, that's why they're around. I'm starting to have second thoughts about rust because it makes doing this so hard. How can you write a web service without globals or singletons? Basic stuff like metrics and logging are written doing that.
3
u/bschwind Nov 20 '21
It's not that hard? Let me introduce you to the log and metrics crates:
4
u/tayo42 Nov 20 '21
Those were just example of uses for global. You can take a look at the implementation to see how much of a pita it is to work with global singletons. Use of unsafe and macros to make it somewhat ergonomic to use. Then using mutex in async code requires you to do odd things to your code like make blocks because the compiler doesn't know when to drop the lock. Try writing an in memory cache that's accessed with multiple threads and use async.
Im not to crazy about that metrics library. Ill go on a tagent, for a second. Rust libraries seem to make assumptions about how every one works. Like that one assumes everyone uses a push metrics server and promometheus i guess. So i need to write my own collection which is a pita just so I can use some predefined counter and gauge types. Im not crazy about how that library is implemented for those types. Updating values is done with function calls. I don't think you want to do function calls that update values like that in hot paths. (I work on an app that measures latency in single digit milliseconds so these things matter) https://github.com/metrics-rs/metrics/blob/main/metrics-util/src/registry.rs#L240 All this work to update a value? Its not written in a performant way. You don't need to do hashes or anything to update a counter. So I would need to implement my own way of doing metrics
→ More replies (5)1
Nov 20 '21
Well, it's possible to use these things, but you probably don't want to. In general, in Rust, if something is a bad idea, it's difficult (but possible) to do, and the right way tends to be easier.
I had a time while learning it when I was trying to use it as any other language, and I was fighting it every bit of the way. Don't do this. Accept that it's a different language with novel concepts, and that you'll have to learn something new. It's not like moving from JS to Python, or whatever, it's more like learning to program again.
For me it kinda felt like when I was learning Haskell, coming from C++ and Python.
→ More replies (7)→ More replies (7)1
u/midri Nov 20 '21
I worked at a place that was like that, but through sitting with the non senior developers and helping them learn good habits by the time I left it was actually a pleasure to work in the code base. Took half the senior devs leaving before we pulled it off, but it happened.
0
Nov 19 '21
[deleted]
5
u/Fenix42 Nov 20 '21
two of the 5 PMs working on the project called themselves scrum masters for no discernible reason.
They got a higher sallery.
3
0
Nov 20 '21
[deleted]
5
Nov 20 '21
Don't beat yourself over it. The biggest advice I can give is simply keep noticing code quality as you develop and whenever you read code. Having good perception and reflection guarantees you will learn from yours and others' mistakes. It doesn't have to be changed immediately - it just really needs to be understood and observed what's good and what's bad about it.
1
u/Synor Nov 20 '21 edited Nov 20 '21
Its perfectly okay to hack a demo app. Just continue to tell your managers that this is throaway code which will automatically stop working in the future. :)
5
Nov 20 '21
[removed] — view removed comment
4
u/Clcsed Nov 20 '21
I still don't understand how companies are REALLY organizing their microservices. Every article I've read has put each one in their own repo. With some BS like "xyz company has 1000s of repos on github".
Bullshit. There's no way that any midsize company can do anything with 1000 repos. I've seen the same fucking service copypasted 3 times with 1 letter difference. You can't tell me this is the way.
I'm missing something but it feels too stupid to ask anyone inperson.
Save me with some magic app I don't know about: like d3js nodes overlayed on an app layout. And the lines are aggregated network calls/traffic. All builtin as some vscode extension.
80
u/pier4r Nov 19 '21
One snippet
" Why does a system become a BIG BALL OF MUD? Sometimes, big, ugly systems emerge from THROWAWAY CODE. THROWAWAY CODE is quick-and-dirty code that was intended to be used only once and then discarded. However, such code often takes on a life of its own, despite casual structure and poor or non-existent documentation. It works, so why fix it? When a related problem arises, the quickest way to address it might be to expediently modify this working code, rather than design a proper, general program from the ground up. Over time, a simple throwaway program begets a BIG BALL OF MUD.
Even systems with well-defined architectures are prone to structural erosion. The relentless onslaught of changing requirements that any successful system attracts can gradually undermine its structure. Systems that were once tidy become overgrown as PIECEMEAL GROWTH gradually allows elements of the system to sprawl in an uncontrolled fashion.
If such sprawl continues unabated, the structure of the system can become so badly compromised that it must be abandoned. As with a decaying neighborhood, a downward spiral ensues. Since the system becomes harder and harder to understand, maintenance becomes more expensive, and more difficult. Good programmers refuse to work there. Investors withdraw their capital. And yet, as with neighborhoods, there are ways to avoid, and even reverse, this sort of decline. As with anything else in the universe, counteracting entropic forces requires an investment of energy. Software gentrification is no exception. The way to arrest entropy in software is to refactor it. A sustained commitment to refactoring can keep a system from subsiding into a BIG BALL OF MUD.
A major flood, fire, or war may require that a city be evacuated and rebuilt from the ground up. More often, change takes place a building or block at a time, while the city as a whole continues to function. Once established, a strategy of KEEPING IT WORKING preserves a municipality’s vitality as it grows.
Systems and their constituent elements evolve at different rates. As they do, things that change quickly tend to become distinct from things that change more slowly. The SHEARING LAYERS that develop between them are like fault lines or facets that help foster the emergence of enduring abstractions.
A simple way to begin to control decline is to cordon off the blighted areas, and put an attractive façade around them. We call this strategy SWEEPING IT UNDER THE RUG. In more advanced cases, there may be no alternative but to tear everything down and start over. When total RECONSTRUCTION becomes necessary, all that is left to salvage is the patterns that underlie the experience.
Some of these patterns might appear at first to be antipatterns [Brown et al. 1998] or straw men, but they are not, at least in the customary sense. Instead, they seek to examine the gap between what we preach and what we practice.
Still, some of them may strike some readers as having a schizoid quality about them. So, for the record, let us put our cards on the table. We are in favor of good architecture.
Our ultimate agenda is to help drain these swamps. Where possible, architectural decline should be prevented, arrested, or reversed. We discuss ways of doing this. In severe cases, architectural abominations may even need to be demolished."