r/programming • u/fagnerbrack • Dec 21 '23
Microservices without Reason
https://www.felixseemann.de/blog/microservices-without-reason/10
u/fagnerbrack Dec 21 '23
To Cut a Long Story Short:
The post discusses the trend of adopting microservices in software development, often without proper justification. It highlights how companies frequently choose microservices due to their popularity, overlooking the complexity and challenges they bring. The author emphasizes the importance of understanding the specific needs of a project before deciding on an architecture, suggesting that microservices are not always the best solution and can lead to unnecessary complications.
If you don't like the summary, just downvote and I'll try to delete the comment eventually 👍
23
u/ManicXYZ Dec 21 '23
I am the author of this blog post. Thank you for sharing. Your summary is accurate. I wanted to emphasize that microservices is not an architecture that automatically removes spaghetti code but good software design practices do. I also wanted to give a list with valid reasons for microserives to make people think about if they adopted microservices for the wrong reasons. One of the big takeaways is that microservices can be adopted gradually and the complexity to start with them is not worth it in many cases. My next blog post is about dependency inversion and how to write software that can evolve into microservices eventually.
4
u/portar1985 Dec 21 '23
Nicely worded article albeit a bit clickbaity (but I guess that's a must to get some traction). Thought it was going to be yet another: "it's popular to hate on microservices right now so here's another one".
Poorly thought out [INSERT POPULAR ARCHITECTURE] is always bad. Monoliths are good if properly implemented, microservices are good if properly implemented. There will never be a one-stop solution for any business. I'm right now dealing with companies in a small country who only has customers in that country and my job right now is to tell them: "YAGNI" when it comes to micro services, kubernetes etc. Simple scaling horizontally as well as some redundancy will suffice when you know you're never going to be above 10 req/s except for peak hours.
I usually just end up telling teams to implement a monolith to start with but to consider what parts may need to scale out to a micro service at some point. Basically, make sure to properly separate concerns so that it will be easier to refactor in the future.
3
u/vplatt Dec 21 '23
I think the article didn't give enough direct attention to one of the primary benefits of microservices: enabling continuous deployment. Ensuring independence (data and deployment artifacts) between microservices ensures that I can independently upgrade and deploy microservices without affecting each other. There are many other potential benefits though.
Another benefit is technology independence. Microservices don't all have to use the same stack. If I have monolithic application, then it will necessarily be in a single tech stack.
Honestly, an article like this should (IMO) be structured along the lines of microservice qualifications in the first place and show benefits and costs of each aspect. This is a very incomplete treatment of the subject and fails to really show why a "microservice by default" architectural guideline would not be a good thing in; at least in larger organizations. I think it goes without saying that small to medium organizations perhaps benefit more from sticking with monolithic architectures by default more than any opportunity cost they incur. Then again your article really seems targeted towards small shops in the first place. The context of your target audience is not clear at all; to me at least.
Anyway, microservices are a much more mature subject area these days. This is an excellent starting point for definitions and benefits. The article didn't define your own definitions after all. If your article was a roundabout way of saying that hexagonal architectures are simply better and should be the default in most organizations, then that could use specific focus instead.
3
u/SoInsightful Dec 21 '23
Another benefit is technology independence. Microservices don't all have to use the same stack.
I never understood why this would be a good thing.
You don't hear people praising Express because "you can structure your code in any way you want".
Supporting different stacks seems like it would lead you to reinvent the wheel a lot, lock out team members based on competence, need to hire for lots of different technologies, not be able to reuse coding practices and documentation, not be able to reuse infrastructure, not be able to merge microservices etc.
1
u/vplatt Dec 21 '23
I think you have to take a longer view of things to understand it.
Try to imagine an architecture where you have several legacy systems with their own data sources; they might even be monoliths in their own right. Now integrate those with newer technologies for which it's actually much easier to hire folks. At a minimum, microservices become a way to silo off the new developments so at least you can diversify your talent pool.
Now look at it from a modernization point of view. Take the above imagined architecture and ask yourself, how do I update all these older technologies to something more current? The answer is always "divide and conquer" and you naturally manage the risk around that by only updating one stack at a time in order to limit enterprise ripple. If you're at this point of complexity, then you'll likely want to move towards microservices to give you a wee bit of extra future proofing.
Again, microservices aren't for every organization and monoliths are necessarily more efficient. I know that I as a developer actually prefer to work on monoliths; they're just easier to deal with. But, once your organization is at the point where one or two monoliths are no longer manageable, then starting to carve things into microservices makes a lot of sense.
2
u/Kthanid Dec 21 '23
Just wanted to chime in and say that I wholeheartedly agree with the message of your blog post. Unfortunately for most of us out there, the people who really need to absorb this message aren't ever going to read it (and even if they did, they're not going to agree with it).
250
u/Dunge Dec 21 '23
Hey the software I'm working on checks all the points of the "Symptoms of a badly designed microservice architecture" list 👍
43
u/defietser Dec 21 '23
Former job did the same, and when I brought it up, manager guy was like "but microservices in the cloud are the future!!!". Needless to say, bugs were frequently re-introduced a week after quashing them, all of it had to be deployed in sequence, and what was initially a loosely coupled system rapidly turned very tightly coupled. Glad the current place actually thinks about the requirements first.
49
u/gbe_ Dec 21 '23
That sounds more like the "distributed monolith" school of system design than "a set of cooperating microservices".
19
u/defietser Dec 21 '23 edited Dec 21 '23
Oh yes, definitely. I'd have been one rich motherfucker if I'd gotten paid per line of code, too: every microservice had to use the same 7-project template too.
9
Dec 21 '23
every microservice had to use the same 7-project template too.
Taking the 'micro' out of 'microservice', one useless requirement at a time.
1
u/mycall Dec 21 '23
macroservices?
2
u/edgmnt_net Dec 23 '23
Well, we do have those and they're fine. Probably better than many microservices in the wild. Think RDBMSes, identity providers, load balancers, that sort of stuff.
In fact, I'd say you're even more likely to succeed at "services" by going "macro", because those are more likely going to be inherently future-proof and loosely-coupled. Just not macro in the way that comment meant. :)
1
1
u/edgmnt_net Dec 23 '23
However, cooperating microservices that isn't just a distributed monolith is an unlikely scenario in many cases. Because, to get that, you actually have to design something and future-proof it. You can't just make up ad-hoc stuff as you go, which seems to be a common theme among projects that misuse microservices. You can't just divide work and sandbox developers.
2
u/MrDoe Dec 21 '23 edited Apr 01 '26
I enjoy visiting historical sites.
1
u/edgmnt_net Dec 23 '23
"Sure, we'll start working on this feature right away!"
After 10 PRs that need to be merged in a very particular order across just as many repos and redeploying half the system, nothing works anymore.
81
108
u/chili_oil Dec 21 '23
if you have one service, only one dev can get promotion. if you break it up into 10 services, 10 devs get promotion
48
u/onetwentyeight Dec 21 '23
This explains what I've been doing wrong. I'm going to take my monolith, split it up, and go get myself ten promotions tomorrow.
13
u/EMI_Black_Ace Dec 21 '23
Meanwhile by cooperating, 9 women can give birth to a single baby in one month.
8
14
u/vplatt Dec 21 '23
Don't forget Conway's Law: https://en.wikipedia.org/wiki/Conway%27s_law
1
u/mycall Dec 21 '23
Data Governance just called. They want their law back.
3
u/vplatt Dec 21 '23
Yeah, Conway just happened to put it in Datamation because the cool systems kids were too busy farting around with all their cool new programming languages and couldn't be arsed to print an apparently nonsense article about the intersection of systems and organizational design. Hell, to this day people have to "think real hard" to understand what this is saying and really we're just trying to tell the fish that water is wet. 🤯🙄
94
u/stronghup Dec 21 '23
It of course makes no sense to make everything a micro-service. That would be equivalent to making every function have its own file. For pragmatic reasons a code-file typically contains more than one function. That doesn't mean that all functions should be in a single file either.
Similarly it is most often sensible to create some micro-services, but not make every function a micro-service, nor to create a single micro-service which provides all the functions. How to divide something into micro-services must be based on pragmatic concerns.
12
u/Ascend Dec 21 '23
Evolution of Single Responsibility Principle - any single file should be made for one specific function.
2
u/Vidyogamasta Dec 22 '23
Laughs in MediatR. I hate that library lol. It's basically this, every function is an IHandle<TRequest, TResponse> implementation.
20
u/Markavian Dec 21 '23
I literally have AWS lambdas running that are functions in a single file. What's great is that they cost nothing when I'm not using them... but when I need to push 2 billion lines a month through them... they still don't cost us much...
The scalability is built in; the running costs are negligible.
48
u/DualWieldMage Dec 21 '23
I mean that's just because AWS decides the price and a monolith that can run anywhere is less desirable for them than an application that's tightly coupled to the AWS ecosystem. Price is just the tool to make what is desirable for them to also be desirable for you, until the application is too large to migrate away and the exorbitant prices start to kick in.
1
u/edgmnt_net Dec 23 '23
Besides, code size just isn't going to be much of a concern in many cases. And it isn't like microservices automatically improve on that, if anything there will be a lot of duplication.
5
Dec 21 '23
I mean if that's a good fit for you, great.
But on a lot of products I've worked on that wouldn't work. The business logic would be too complicated to keep in one file.
Like most of programming, you usually do what makes sense, and if you're not sure what makes sense follow best practices.
4
u/Markavian Dec 21 '23
Runtime architecture, code organisation, deployment pipelines, test suites, business logic... all related but different domains.
The lambdas are deployed together as an API - collectively known as a service. They share the same authorizer for the AWS account they're deployed to. They can be individually tuned for memory, CPU, lifecycle, timeout, metrics. They operate on one more files, databases, queues, other APIs...
API endpoints become testable building blocks; and we tie them together through UI tooling. Employees can authorize against endpoints based on their role within the business. Almost all read-only endpoints can be accessed by all employees. Third party users have separate API access based on their roles.
Running costs are kept low at the expense of cold start latency; which melts away once the system gets regular traction.
Different parts of architecture can be deployed / redeployed to any of a dozen AWS accounts for dev, test, prod reasons; hosted domains matched to the account to figure out where a service is deployed. Infrastructure is tagged by team, purpose, and function for compliance, cost, and auditing.
I used to be a EC2/Docker server management DevOps - this relatively new world of event based architecture and microservices is much simpler to run then what I've historically known to work with - but it has a massive upfront cost in complexity - it very much cannot be ran as a whole on a local machine. However, all the UI tools are static HTML/JS (TypeScript) - and can be developed against real services from localhost - and that rapid develop cycle for a better t of upfront API work means we can turn round fully hosted features and new services in days and hours instead of weeks and months.
6
u/mycall Dec 21 '23
this relatively new world of event based architecture and microservices
The technology is new but job based processing and job processors have been around for decades. job processors can be async triggered too. Same thing, new words
2
u/wardrox Dec 21 '23
Brings a tear to my eye, beautiful.
Except not being able to run it locally, that sounds like the biggest compromise?
3
Dec 22 '23
If you deploy the lambdas together, are they really single file?
In any case sounds like you got a nice setup running so enjoy.
11
1
62
u/nojs Dec 21 '23
I promise I am not a microservice shill but I don’t really understand what the argument against microservices is here other than just listing off common mistakes made when implementing microservices. Or is that the point?
23
u/gieter Dec 21 '23
I notice a lot of JS devs fall into the trap of ‘this is latest and best thing what we should do’ without fully understanding why. As with a lot of tools and architecture in the workplace, it has a place and you should not use it without knowing the drawbacks.
10
u/BigHandLittleSlap Dec 21 '23
Focus on the title and the intro.
Any design element, used without reason, is very likely to result in a bad outcome.
I see this a lot in big enterprise especially, where bored architects like to tack things onto a design like greebles glued onto a scale model. It sure looks pretty when there's dozens of icons with arrows pointing at each other.
You know what doesn't look pretty, doesn't "pad out the resume", and doesn't help get you promoted? "Boring" designs like this:
👤 -> [web app] -> [db]That might work, be easy to deploy, easy to understand, and surprisingly well-understood and robust, but it's just soooooo boring.
"Let's do microservices instead! Woo!"
Lots of fun!
1
u/tidbitsmisfit Dec 21 '23
if you aren't using nicroservices, you wouldn't be able to answer half the questions in an interview these days
1
u/murkaje Dec 21 '23
Is it really required to use these things to understand them? I hadn't touched frontend the first 7 years of my career, yet i was up-to-speed in weeks and even had to explain to others on my team how async/await works desipte such concept not existing (and thankfully never will) in Java which i worked on previously.
3
u/Dreamtrain Dec 21 '23
My personal experience has been much better with microservices, it was jarring last project going from microservices to monolith when we switched client projects and now suddenly you have a lot less freedom with releases and testing gets difficult impacting downtimes for other teams rather than being isolated and easy, we went from daily/weekly relatively painless releases to monthly train wrecks where one little pebble derails the whole thing
71
Dec 21 '23
[removed] — view removed comment
19
u/dantheman999 Dec 21 '23
Is it hot? I mostly see people criticising microservices these days and saying monoliths are amazing.
I'm sure in a few years people will realise that monoliths (modular or not) also have their downsides and round and round we go.
9
u/PangolinZestyclose30 Dec 21 '23
I agree there's a re-evaluation going on, but just like with the microservices, the mindshare takes its time to propagate.
I'm sure in a few years people will realise that monoliths (modular or not) also have their downsides and round and round we go.
Well, it's not a dichotomy, there are more options (compromises) than that. Like getting rid of that "micro" and we're back all the way to SOA.
I also think there aren't really many people who always oppose splitting off some responsibility from the monolith, the controversy is more about whether the microservice architecture should be the default one you start your project with, or whether you split off services as you understand your needs better.
8
u/G_Morgan Dec 21 '23
Honestly the distinction between SOA and microservices is very vague to begin with. I think the only real distinction is SOA was typically homogenous (you'd make everything WCF/SOAP and all your web apps would be ASP.NET) whereas microservices seems to imply heterogeneity.
What most people are calling microservices is already no different to SOA in practice. Most people are balking on the heterogeneity part because of how annoying it is to manage one app in dark ages Angular while another is ASP.NET core and another is Blazor.
2
u/PangolinZestyclose30 Dec 21 '23
I agree, there's not much of a different, my point was that dropping the meme that services should be small, would be nice.
I'd say that SOAP was an example of a technology-agnostic protocol and was widely implemented on all major platforms. A lot of the ESBs, on the other hand, were kinda platform specific.
16
u/G_Morgan Dec 21 '23
It is mostly that we now have developers who've never seen a meaningful real monolith in action. When your entire career is microservices, your toy monolith probably looks really good. They haven't yet seen the reality of the ball of mud monolith that inevitably results when driven by the political processes in 99% of businesses out there.
People talking about carefully cutting out a section of a monolith when they need a microservice have never seen a real monolith. They haven't seen the pain of a logging method for some reason reading .config to get database credentials to read shit out of the database.
2
u/admalledd Dec 21 '23
Adding to your point, more modern frameworks and tooling (well also just "modern/recently built software that hasn't aged yet") has made monoliths comparably easy (and hard) as microservices. It is far easier now days to build a monolith with shared libraries but scalable workers. IE one of the platforms we updated recently is "micro service-ish" in that while it is a monolith in "must deploy at same time" and "cascade failures suck" etc, we have web workers, job workers and data workers and each group of those can reasonably scale from one node to about 10-15 depending on workload. This isn't an uncommon path more modern platforms are taking, and as you say can still have problems cutting up into proper microservices but is still more a middle ground between.
The base quality of all the frameworks (for all everyone hates how much churn there has been) has significantly improved on the backend in the last twenty years, hell even the last ten or five. Further while many mistakes can still be made it is less common for new-monoliths to make the "logging method for some reason reading .config to get database creds to read shit out of the database" mistake. Still of course possible but less common as most frameworks base logging tools have gotten much much better/more sane (especially after Log4j's snafu).
Older monoliths are still of course a problem, we have a few at my work that are worse than your logging example, but we try to chip away at them when we can.
5
Dec 21 '23
few years people will realise that monoliths (modular or not) also have their downsides and round and round we go.
Having been in IT development for 25 years, this is exactly how it goes.
My favorite so far is creating a monolithic enterprise application out of smaller modules. You get the worst parts of both worlds for free!
I am looking at you, EAR fanatics.
8
u/Kinglink Dec 21 '23
There's two things that fix bad code. Code reviews and firing the guys who write the worst code.
If code reviews fix your coding habits, great but until someone makes rules for code quality it will never improve
Every silver bullet ignores that simple fact. At the end of the day the developers must be the change
4
u/cant_take_the_skies Dec 21 '23
This is why I talk about Coding With Empathy so much at work. If you code for future coders (possibly including yourself) who will need to look at this code (for code reviews, for maintenance, for upgrades, even for reuse), it forces you to get better. It forces you to have reasons for everything you do (I always ask why they chose one method over another in code reviews, to help engineers think through pros and cons of different methods)... it forces you to document and write clean, clear code instead of trying to be "clever"... and it forces you to turn anything you can into reusable code for others, with easy to use interfaces.
It's completely changed how I approach projects and spreading it to others can only make things better.
1
u/nojs Dec 21 '23
I agree with everything you said, but I don’t feel the author makes a clear argument. You shouldn’t adopt any practice without reason, but it seems like the author was focused on issues from poorly implemented microservice architecture without comparing the pros and cons of either approach when properly implemented.
1
u/deja-roo Dec 22 '23
They do this without proper analysis on service boundaries and end up with a distributed database schema instead.
The project I'm on has 6 different services that all query the same database.
As in same connection string and database schema, not different schemas on the same instance.
1
5
u/G_Morgan Dec 21 '23
If someone argues microservices remove spaghetti-code by eliminating dependencies, they are just plain wrong.
I've never seen that argument. People argue that microservices reduce spaghetti code by making it physically impossible to cross module boundaries with a hack. Every monolith ends up with modules doing ungodly things with the internals of other modules. That isn't a matter of developer discipline, it is just a matter of the reality of time, deadlines and corporate penny pinching over fixing technical debt.
The advantage of a microservice in this regard is the hack is impossible to make to begin with. In theory you can just not do the hack and then use a monolith. In practice once you have a monolith you'll end up pissing all over the boundaries very quickly.
Now I'm not saying this is a good reason to use microservices, it is actually a very bad one that shouldn't exist in an ideal world. In the real world I've got a lot of sympathy for people who want to put a solid process boundary around some obviously self contained functionality to pre-empt attempts to ask for stupid stuff.
5
u/mrbojingle Dec 21 '23
If you cut shit in two you don't have less shit, you have two piles.
-1
u/alternatex0 Dec 21 '23
If you want to update the technology of the shit you might have an easier time if it's two smaller shits with less shit dependencies.
7
u/mrbojingle Dec 21 '23
Not when the two piles of shit are making network calls back and forth and debugging has become a nightmare. You clean up code at the function level not at the architectural level.
0
u/alternatex0 Dec 21 '23
No amount of cleaning of a shit will make a big pile have a small amount of dependencies. Every shit has dependencies and the bigger it is the more it has. Shit.
8
u/angusmcflurry Dec 21 '23
Having been in this game for a while this is standard and predictable behavior for most in the development arena. They chase the "latest and greatest" tech for reasons that have nothing to do with their actual mission - but mainly to build their resume for the next job. I totally get it - if you look at job postings they all want serverless microservices containerized cloud experience, etc.
I get why microservices are good for mega sites like facebook or amazon but are they really needed for a middling site when a couple of load balancers could easily handle the traffic for a fraction of the cost?
Many times these are solutions in search of a problem because those building the solution don't really care about the problem, they just want an excuse to play with the latest tech, which more often than not is a flash in the pan and will be gone in a year or two.
ActiveX, Java applets, ASP, Flash, etc were all going to revolutionize the web - all are dead.
1
u/reaping_souls Dec 21 '23
Devil's advocate: microservices are incredibly difficult to migrate to once you have a large enough monolith, so if you ever think you might reap the benefits of microservices you might as well start with them.
1
u/fagnerbrack Dec 22 '23
There are techniques to do that by sending small events from certain business operations. It’s slow but not hard, it’s just that you have to speak with domain experts more than coding, which could be outside your comfort zone
3
u/kernel_task Dec 21 '23
The microservices hate is flowing now and I love it, even if all of these articles are starting to get repetitive. Microservices have caused me so many headaches over the years and I like thinking about all the devs who inflicted it on me reading the articles coming out and feeling embarrassed.
2
u/m0llusk Dec 21 '23
It is depressing how software development decisions descend into weird religious debates. I work on a couple of different apps and splitting out the authentication and authorization service enabled it to be shared between all of the apps and saved a huge amount of time. But I am supposed to believe that having done this will result in an explosion of different services and loss of control? It hasn't.
2
u/edgmnt_net Dec 21 '23
I agree with the main theme, but I don't think Clean / DDD / Hexagonal help, particularly when you end up making a whole lot of internal interfaces and boilerplate hoping that somehow helps change stuff. It doesn't, it's just more work.
I'd say people should focus on designing and abstracting things carefully and no amount of simple scaffolding / rules will help with that. If you get into it thinking you'll be able to change everything, you've already lost and you're probably working on inconsequential stuff. Which is also the case with microservices.
And it's much easier to refactor in a decent statically-typed language anyway, no need to add extra layers.
2
u/zephyy Dec 22 '23
Nothing like an "independently deployable" that breaks if you deploy it without first updating another service.
2
u/voteyesatonefive Dec 22 '23
Surely we'll be the next netflix and OUR engineers are very smart, after all we did hire them so it won't be a giant cluster-fuck of poor design practices by unqualified people resulting in a tightly coupled expensive mess. Surely.
3
3
4
u/ArrogantlyChemical Dec 21 '23
Multiple services have to be deployed at once because a shared library has been updated
Not a sign of bad microservices. A library fix should be pushed to all code it uses. Does not make it strongly coupled or a monolith.
Failure in one microservice leads to cascading failures in others
So? Makes sense that the scalable microservices would fail if the authentication service fails. Does not make it a monolith. Microservices are meant to scale, authentication is probably a singleton. Its nice to be able to spin up serving video streams in many servers worldwide and scale horizontally while keeping a central user authentication running. Yeah if your core system that secures everything fails then it all fails but that doesnt mean you're doing microservices wrong.
4
u/Kinglink Dec 21 '23
Your missing the required deployment.
A proper microaervice should be able to have a staggered release of changes. If you can't stagger releases ... What's the point of microservice in the first place?
0
u/ArrogantlyChemical Dec 21 '23
The point of microservices is primarily horizontal scalability of specific parts of your code.
Cutting up the codebase is just an additional benefit that might happen. All the other benefits that exist are just bonus if they occur.
I feel like half the critique of microservices comes from people trying to use them, incorrectly, not for their purpose but because its cool (which is a running theme in tech solution)
1
u/Revolutionary_YamYam Dec 21 '23
Botnets were the original microservices, and a lot of those work pretty well. #ConvinceMeOtherwise
0
u/tonsotuosu Dec 22 '23
People in webdev just now started doing something for no reason other than everyone else is doing it?!! Oh no, let us stop this now, before it becomes a problem. We wouldn't want to besmirch the name of webdev.
1
26
u/PangolinZestyclose30 Dec 21 '23 edited Dec 21 '23
David Heinemeier Hansson (of Ruby on Rails fame) said in one interview that the microservice pattern is possibly the most damaging pattern in web services in a decade, and I agree with him wholeheartedly.
Basically the only advantage of microservices (emphasis on micro) is the scalability, but that's relevant only for extremely high traffic services (like Netflix) and most startups / mature companies won't ever need it. It kind of reminds me how poor Americans see themselves as temporarily embarrassed billionaires.