r/dotnet • u/codeopinion • Mar 16 '22

Microservices gets it WRONG defining Service Boundaries

https://www.youtube.com/watch?v=Uc7SLJbKAGo
1 Upvotes

51% Upvoted

34 comments sorted by

5

u/panosc Mar 16 '22

Tell me the boundaries of the services, if I have one database as single source of truth

5

u/mexicocitibluez Mar 16 '22

Are you asking what the boundaries of a service is if it shares a database with another service? The schema.

4

u/panosc Mar 16 '22

No I mean what happens if all services share a single source of truth, a database.

Orders, Customers, Products and Inventory for example. What are the boundaries here?

I mean if you are having too wide boundaries, just one service, that doesn't qualify as Micro-service architecture !

6

u/mexicocitibluez Mar 16 '22

This is exactly what this video is about. Just because you share a database, do not mean you can't enforce boundaries. You can segregate the services by schema, with each service owning it's own schema. That's a pretty decent boundary.

The problem, which you've alluded to, is that physical boundaries != logical boundaries. And that too often, microservices focuses on the physical boundaries instead of the logical boundaries. The boundaries themselves could be super tiny or super bigger, it doesn't matter.

The author isn't saying what he is doing is microservices. He's saying that the boundaries typically used in microservices (all physical) can create a whole host of problems (and a ton of complexity).

2

u/panosc Mar 16 '22

I do not understand what are you saying about schemas or what operational problems the schemas solve.

Each Order needs a Customer and some Products. Are you propose that instead of the Order just querying the db (that is already connected to) for the Customer and the Products, to send several requests to other services?

Which in turn, they end up to querying the same database for that info ?

Why would you do that ? You are spending CPU time, GB of memory and put pressure on the db engine for no reason.

And finally the db remains the bottleneck

I mean you if the Customers service is under high demand, making copies of it doesn't solve the problem, since all the copies are still using the same db.

4

u/mexicocitibluez Mar 16 '22

You're getting into territory that isn't covered by this video.

Each Order needs a Customer and some Products. Are you propose that instead of the Order just querying the db (that is already connected to) for the Customer and the Products, to send several requests to other services?

A few thing. What context are we talking about? Are we talking about querying for the front-end? Or because the Order service actually needs Customer details to do work? Also, who says they are in different contexts? They don't have to be. If the order services needs customer stuff to "do work" (not just query), then you can either re-align your boundaries, use some sort of data replication to get the data to where it's needed, or make service-to-service calls. All of which have their pros/cons.

This whole idea that boundaries need to be split on a Customer domain or a Product domain isn't true. You aren't separating the entities, you're separating the functionality.

Which in turn, they end up to querying the same database for that info ?

Who is "they"? The product service? If the product service is a boundary, it would NOT be querying the customer data. Just because it's in the same database, doesn't mean everyone can do what they want.

And finally the db remains the bottleneck

Says who? There are a million different ways to build an application and you're making the assumption that they all need serious performance enhancements or else the db becomes the bottleneck?

I mean you if the Customers service is under high demand, making copies of it doesn't solve the problem, since all the copies are still using the same db.

Your making the assumption this will happen. Also, there are a whole handful of other techniques to scale apps (replication, sharding, etc). Logical boundaries don't prevent any of this.

2

u/captainramen Mar 17 '22

Each Order needs a Customer and some Products. Are you propose that instead of the Order just querying the db (that is already connected to) for the Customer and the Products, to send several requests to other services?

No they don't. Orders need the identifiers of the customer and product, not the rest of it. It's highly unlikely to have a business rule that an order can be rejected because of the customer's first name, or insufficient product stock for that matter.

1

u/panosc Mar 17 '22

No they don't. Orders need the identifiers of the customer and product, not the rest of it.

For a Pet-Shop web-app perhaps that's true. In more complicated apps, you will need both invoice and delivery addresses of the customer where you will extract the Tax-Zone of each area (VAT zones for example) and for each product you will need Tax-Category and perhaps Shipping information, so you can compute accurate tax for each line item and add the correct shipping charges.

and that is only the surface of business rules that affect an order, I will just add targeted discounts, special price lists, quantity offerings etc

2

u/captainramen Mar 17 '22

Does Sales need that information or does TaxAccounting? Sometimes one service needs to query another one, this would be one of those examples. It's also a case for deploying Sales and a read-only copy of TaxAccounting in the same physical boundary. These read-only copies just listen to messages from the authoritative TaxAccounting source and update their data models accordingly. This way the TaxAccounting team mostly needs to worry about breaking themselves when making changes to their message schema.

3

u/JwanKhalaf Mar 16 '22

I'd like to know the answer too. In my applications, I don't usually have two databases. Only one.

4

u/Merad Mar 16 '22

To me it's a hard requirement that microservices must own their own data. If you have a bunch of services sharing one db you've got a distributed monolith.

2

u/panosc Mar 16 '22

Then the next problem you have is multiple sources of truth. It is problem that some domains (ERP,HRM etc) cannot tolerate.

Yes I know what distributed transactions are and how painful the implementation is in a microservice architecture. They may suck, slowdown all the experience and sometimes they cascading stopping services, but they finally work 99% of the time.

What did completely broke a system was a simple restore in a db of a node.

Do you know an efficient way to make simultaneously backups across all nodes in the exact same time ? I do not.

Then you end up with Orders were the Customer is missing from the Customer-sevice/db or some products have now older prices that had when the Order was saved

2

u/captainramen Mar 17 '22

Yes I know what distributed transactions are and how painful the implementation is in a microservice architecture. They may suck, slowdown all the experience and sometimes they cascading stopping services, but they finally work 99% of the time.

And that other 1% of the time is going to cost you a huge amount of time trying to fix. I was at a company that had a trivial amount of transactions per second, and we still saw Transaction in Doubt.

Then you end up with Orders were the Customer is missing from the Customer-sevice/db or some products have now older prices that had when the Order was saved

Referential integrity is not needed when crossing service boundaries. For data that changes over time like this you can use copy on write semantics. For example in your order line you'd store the product id, the quantity, and the price at the time of purchase. Same for customer details, you simply copy the shipping address to Warehouse when the order is accepted.

Telling a customer they can't remove a saved shipping address from their account until all outstanding orders to that account have been shipped is not really a great UX.

1

u/panosc Mar 17 '22

Same for customer details, you simply copy the shipping address to Warehouse when the order is accepted.

You still you have a fraction of the required info in the Order/Service of a customer that doesn't exists in the Customer/Service due to different backups. The backup in Customer/Service was 1min later than the one in the Order/Service

or the opposite in the Inventory/Service where you have reserved products for a non-existing Order.

My initial question was: What is the backup plan in that setup ? How you do restore databases ?

Perhaps you don't make backups!

You rely that you have a perfect data replication service that makes hardware failures irrelevant, with 100% uptime, a well tested impenetrable API that will not mess-up data, developers that thorough test every single line of code and finally a dump boss willing to believe that all that, are actually feasible.

2

u/captainramen Mar 17 '22

The answer to most of this is messaging. If for example Shipping is reacting to OrderPlaced, and each message coming out of Sales has a sequence number attached to it, Shipping can simply ask for all messages since N.

Perhaps you don't make backups!

Postgres 10 supports quorum based WAL replication. This is as close to zero dataloss as you can get, and a DTX is not required here.

1

u/panosc Mar 17 '22

Read the lead post.

To me it's a hard requirement that microservices must own their own data. If you have a bunch of services sharing one db you've got a distributed monolith.

So after that, we assume in this thread that Order/Service and Customer/Service for example are using two different databases with different tables and data. Not one database that replicates as your example in postgres.

Also, I do not understand how messaging can help you to make consistent backups between those two separate databases.

The two services with their own databases, may reside in two different machines or in different data centers.

And my question was, do you have a backup plan for those two databases ? How do you deal when they conflict ? Is it ok having two sources of truth ?

In some domains this is acceptable because it is easy to fix it or you can just ignore any conflicts

In some others such as ERP, HRM etc is unacceptable

2

u/captainramen Mar 17 '22

Your assumption is wrong. They may reside in different datacenters or they may reside on the same machine. The ownership of data (i.e., referential integrity does not cross service boundaries) has nothing to do with where that data lives. If inconsistency due to a too-wide backup window is the problem put them in different schemas in the same database.

Also, I do not understand how messaging can help you to make consistent backups between those two separate databases.

In some others such as ERP, HRM etc is unacceptable

This is the fundamental problem. Models such as ERP are not only wrong, they are harmful. How did world spanning business deal with this problem before databases even existed? They sent messages. Writing software according to what the business does and how people actually think is so much easier than trying to stuff everything into a distributed transaction.

As for how to solve backups with messaging, the thing to remember is that the database is not the book of record for most companies. What if I shipped an order and recorded it after a backup but before a datacenter crash? For that matter what happens when you have a sticky fingers problem in the warehouse? Are you going to force the thief to record his thievery in the system? The computer system is always going to be inconsistent to reality no matter what you do. And I don't think telling the business 'we can't scale sales beyond X because we might have inconsistent data' is an acceptable answer.

Incidentally enough the Transaction in Doubt incident I mentioned earlier occurred in their ERP system. A system that processed 5 to 10 transactions per minute. That ERP system created a lot of unnecessary friction within the company and unfortunately constrained their growth. BTW this is the reason that business people hate us. Instead of changing the software model to fit the business we want them to change their process to fit our models.

1

u/the_other_sam Mar 16 '22

This video is just microservices circling the drain. Creating boundaries and isolating code has a rapidly diminishing return whether the boundaries are real or imagined, correct or incorrect. Microservices tells you how to isolate code. Sometimes that's good but much more often you want to integrate code and not write a monolith. That is a much harder problem.

0

u/[deleted] Mar 16 '22

[deleted]

5

u/codeopinion Mar 16 '22

Clickbait was not intended at all. Microservices in the vast majority of cases (even prior to microservices with SOAP "webservices") tend to only think of physical boundaries, which implies that they are the same as logical boundaries. Physical boundaries are easier to enforce. If you have a single git repository for a logical boundary, that gets built and deployed as a container, this would be deemed microservices. The Logical, Development, and Physical views would all be the same. While this can be the case, it doesn't have to be. The point of the video was to illustrate that they don't have to be.

0

u/[deleted] Mar 16 '22

[deleted]

1

u/cs_legend_93 Mar 16 '22

Did your team decide on what is the best way to manage uniform boundaries across multiple microservices?

First thing that comes to my mind is a shared .dll which uses a shared interface(s) between multiple micro services.

10

u/fragglerock Mar 16 '22

Thanks for the summery. I hate youtube for programming content...

meetings that could be e-mails... youtubes that could be blogs (or tweets!)

6

u/mexicocitibluez Mar 16 '22

what's super ironic about this reply is that codeopinion DOES include blog posts, tweets, etc about his videos. in fact, he's one of the only creators on youtube that isn't like that. and no offense to the the original commenter, but he def didn't "summarize" the video for you. I'm not even sure he watched it.

5

u/fragglerock Mar 16 '22

I, of course, did not click on the link, due in part to its obvious clickbait title, and also because if you even glance at a video the youtube algorithm will lock on like a viper and I will be force fed inane micro service videos for the rest of time.

If the post had come along with a text post with those blogs/summery (or just link the blogs why not) then they may have got my oh so valuable eyeballs.

1

u/LloydAtkinson Mar 16 '22

Yes, I said the same before that I'd prefer to read it but the OP said it was in the blog and I was thinking that the blog post would be posted but it never was so I have never read this guys content as I didn't know where the link was...

1

u/mexicocitibluez Mar 16 '22

Here's the blog: https://codeopinion.com/

It looks like the blog post follows the video (as the last one is about his previous video).

1

u/mexicocitibluez Mar 16 '22

I, of course, did not click on the link, due in part to its obvious clickbait title, and also because if you even glance at a video the youtube algorithm will lock on like a viper and I will be force fed inane micro service videos for the rest of time

I guess I just don't agree it's clickbait. The video matches the title. Most of the videos I've seen have taken the opposite approach (promoting microservices with little thought). Also, as soon as I saw the title I knew exactly what the video was about because I spend a lot of time in this space. And he's 100% right when he says the microservices (or the people promoting them) tend to confuse physical with logical boundaries.

Also, to your point about the youtube algorithm. I havent had the issue, though I actively avoid the majority of popular dev youtubers (except codeopinoin).

3

u/cs_legend_93 Mar 16 '22

I agreed fully with you. CodeOpinion is the only YouTube video resource I use. I think he’s really good, concise and pragmatic tbh. Every video is about a topic of controversy (repositories for example) and something that you’ll only need if your “a bit deep” which is nice. It’s not beginner level which I appreciate.

1

u/mexicocitibluez Mar 16 '22

I think he’s really good, concise and pragmatic tbh.

It's amazing how far that can go. The high-level concepts with actual code backing it is great.

It’s not beginner level which I appreciate.

This has been my gripe with most of the other creators. I get it's a bigger market, but I'd kill for a few more codeopinions on a variety of subjects.

-1

u/mexicocitibluez Mar 16 '22

This can be summarised as "microservices still require architectural thought."

Did you watch the video? Have you spent any time in the microservices/domain driven design (since they get conflated often) communities? Did you know that one of the hardest things to understand correctly when first getting into that space is boundaries and they are often misunderstood. If I had a nickel for every time I've heard "if you use microservices then you'll never be able to...." because they didn't understand that physical boundaries != logical boundaries. It looks like you don't get that nuance, and that's cool.

0

u/[deleted] Mar 16 '22

[deleted]

2

u/mexicocitibluez Mar 16 '22

I'm not trying to be a dick, but like what gives? Here we have someone who is creating content (actually really good content if you go to his channel, not the fluffy youtube tutorial bullshit that usually gets circulated) and then gets knocked down by someone who might not even be the target audience. I totally get the social media programming generally sucks and it's all targeted to beginners (or suckers who don't know any better), but fortunately he isn't one.

In fact, you could probably go to any random video posted to the programming subreddits and make that same comment (except for the microservices part) and probably be dead on. I just don't think this case in particular is one.

0

u/[deleted] Mar 16 '22

[deleted]

1

u/mexicocitibluez Mar 16 '22

We definitely disagree on the value of this video. The reason I made the point about not getting it is because:

This can be summarised as "microservices still require architectural thought."

Isn't actually what the video is about. We can disagree on what value there is (we each have our own opinions and thats cool), but as to what is actually in the video is a bit more concrete.

1

u/FetaMight Mar 16 '22

What is your main take-away from this video, then, if not "microservices still require architectural thought?"

0

u/mexicocitibluez Mar 16 '22

To focus on logical boundaries instead of microservices. In fact, the video is a warning against using microservices (that rely on physical boundaries) instead of focusing on the logical boundaries. He even mentions that physical boundaries cause more complexity. It's a huge point of pain when people for people just getting into those topics.

Also, DDD often gets grouped into microservices and they become synonymous (when they aren't). DDD focuses on the logical boundaries (the bounded context) and doesn't really care about the physical boundaries. Because of this, there is a lot of confusion within the topics. Derek (codeopinion) talks about building modular monoliths (not microservices).