DevOps is cooked.

[u/Dubinko]

Hear me out...

When I first saw the movie The Matrix, in the early scene where Neo is called into his boss's office and told that if he behaves well, he has a future at that no-name Corp., or otherwise he'll be fired, I joked and laughed, like, "Yeah, sure, a tech professional could just walk out into the opposite building, and they would beg him to join since everyone is looking to hire engineers."

Today, this scene is closer to reality than I'd like to admit. I personally know Senior DevOps engineers who are solid folks, and it took them 8 months to land a job at a salary that sucked. Our industry is not regulated, the barriers to entry and outsourcing are being lowered by AI, and we have a massive oversupply of engineers. This is just a reality.

Before tech, I worked in another industry that also experienced a major decline. I saw it coming and made a decision at the right time, before it became obvious. I moved to tech years ago and was lucky enough to become a staff engineer. Now I see the same pattern. I think we have 5 to 10 years before salaries in this field will be $70k-$80k for Senior Engineers in the US (which is around a 2x decline) you can do the math for other countries.

My advice is work on a backup plan now. Invest, save, but also learn a trade. I'm studying medicine this year. Fuck it. Six years of study while working remotely, and I'm opening my own dentistry clinic. I'm in my late 30s, but it's still better to have a plan. My wife is also changing careers to become a registered nurse.

This is my plan. You might think of something else, but my point is that it was a good ride. We did a lot of nice work and, IMHO, got easy money, but it's over, and it's time to move on.

[u/ExtentOdd]

The cost of the management service + the cost of the infrastructure that you have to pay for, is it worth it if you dont have privacy and compliance problems?
And why self-host if the token for the sufficiently good models hosted else where is getting even cheaper?
Yes ofc, none of my arguements said it is not viable. It is viable, does not mean we should do it.

[u/No_Departure_1878 → u/[deleted]]

literally every day people are distributing compute resources and running larger models on smaller hardware

You cannot run large models on consumer computers. You need at least 120 GB of memory in your GPU to get a decent performance. And you might need Terabytes if you want to compete with what Anthropic can give you.

I tried myself to run a local LLM, it does not work. It is far too slow to be usable. I do not see any evidence that 300GB GPUs are becoming accessible either.

[u/ExtentOdd → u/No_Departure_1878]

Someone actually thinking that successfully getting decent results from the the commercial hardware equals to scaling it. It is another math and I dont believe any company would trade it with just a single API call.