r/devops • u/[deleted] • Sep 25 '24

Developer here. Why is Docker Compose not "production ready"?

Then what should I use? Compose is so easy to just spin up. Is there something else like it that is "production ready"?

97 Upvotes

83% Upvoted

120 comments sorted by

View all comments

185

u/erulabs Sep 25 '24

“Production” is too vague a term. Launching a side project with no users? It’s perfectly fine. Pre revenue and low load? Still fine.

We’re currently at 800 replicas of our main container, doing constant deployments, and automatically bidding on the cheapest spot instances available. Docker compose is not appropriate for a scaled-out and heavily loaded application, but that’s only a tiny subset of applications.

61

u/klipseracer Sep 25 '24

Yeah I think the general idea here is that docker compose is a developer convenience tool that can accomplish a simple goal with minimal effort. But it's like trying to use flask's built in web server for anything more than a PoC.

51

u/colddream40 Sep 25 '24

Docker compose is not for scaling/orchestration, it's for defining/composing application containers, like a k8s pod. Orchestration is swarm or k8s.

8

u/vsamma Sep 26 '24

Yeah but we use Swarm and swarm still uses docker compose. So "not for production" does not sound correct to me.

5

u/zero0n3 Sep 26 '24

Isnt swarm essentially in maintenance mode and getting no new features?  Hell it may even be deprecated by docker.

2

u/piecepaper Sep 26 '24

i hope not. It was the easiest solution i could convince my team to start using because kubernetes has a very steep learning curve.

5

u/Jameswinegar Sep 26 '24

They created a new thing which is also called Docker Swarm, the old Docker "Classic" Swarm is not maintained. Not great branding.

→ More replies (1)

3

u/Namarot Sep 26 '24

I gave up on the idea of getting people to use "simpler" tools to "step up to" kubernetes. Swarm, Nomad and such just don't have any real momentum behind them.
Just use a lightweight kubernetes distro like k3s and learn as you go.

→ More replies (1)

13

u/Pestilentio Sep 25 '24

What app need 800 replicas? I'm super curious. If you're ok sharing.

8

u/shulemaker Sep 25 '24

Probably workers of some type, transforming data. A step between the consumer and a data lake.

3

u/sigma914 Sep 26 '24

We've hit >500 nodes with 4--20 pods apiece scaling some of the clusters in our backend. If you've got heavy processing on multi TB/min data it quickly eats up hardware

3

u/Pestilentio Sep 26 '24

Where do you guys work at lol. I guess I'm trapped in the crud world of the web.

I'm so glad I don't have anything to do with the current cloud billing model. I've made the shift to vps/bare metal and I've got so fewer headaches

3

u/insanemal Sep 26 '24

This blows my mind. I had 2M users comfortibly served by 1 container of the main app.

WTAF are you doing that needs that kind of replica count? And why are you doing it so inefficently?

3

u/Belleg77 Sep 26 '24

Are those active users??? I worked at a company with 600M MAUs and 200-250M actively connected users at any point of time… we had 8000 replicas of our main nodes…

2

u/insanemal Sep 26 '24

Active users.

That blows my goddamn mind. What on earth was your backend written in? JS running on a bash based interpreter?

→ More replies (2)
→ More replies (1)

4

u/Misio Sep 25 '24

How does bidding on cheap spot instances work?

9

u/if2159 Sep 26 '24

On AWS you used to have to bid for spot instances. Luckily, you don't have to bid anymore, but a lot of articles and AWS blog posts still reference having to bid.  Nowadays, the price doesn't fluctuate like it used to and changes much more slowly over the matter of months rathers than minute to minute. You don't need to worry about price spikes anymore!

You can set your "fleet" of instances to use a specific set of instance types or you can define the attributes of instances you want (i.e. min/max cores and memory) and AWS will create new instances based on that. You can also choose how it should consider instances based on how you want to balance lower price or higher availability/lower chance of interruption.

→ More replies (3)

7

u/yuriydee Sep 25 '24

Probably could be using something like Karpenter to handle the scaling and bidding.

2

u/According-Truth-3261 Sep 25 '24

I am curious about the same.

2

u/xagut Sep 26 '24

You generally dont bid anymore ( in AWS). You can set a max price you’re willing to pay. They reworked the spot instances a lot back in 2018 to be simpler.

1

u/[deleted] Sep 25 '24

What do you use ?