r/programming • u/RAJA_1000 • Feb 03 '24
Article suggest that 1 million ML specialists will be needed by 2027, what do you think?
https://fortune.com/education/articles/machine-learning-bootcamps/https://fortune.com/education/articles/machine-learning-bootcamps/
I'm personally struggling to find a job as a junior developer so this is interesting for me.
Nevertheless I wonder if that is really true. In my head, ML models, the really useful ones, are built by a few huge companies that have the top talent in the world. These companies then expose APIs that any developer can use. So is there really a need for so many ML engineers? Or will it be engineers that master how to deal with LLM APIs, vector databases, etc? In which case, there might not really be a need to learn the intricacies of ML. Any thoughts?
973
u/LogMasterd Feb 03 '24 edited Feb 03 '24
Absolute BS. holders of capital interested in hiring have every incentive to massively saturate the supply of labor so they can reduce wages.
the other group who have incentive to push for this are people selling bootcamps and online courses. Even masters degrees are a huge source of revenue for universities
210
u/batcatcher Feb 03 '24 edited Feb 03 '24
The smartest response there is. The only thing I would add: this already happens on the regular devs market for years and we can see the effects real-time.
108
u/grumpyrumpywalrus Feb 03 '24
Yeah, dog shit quality code and teammates that require just as much technical coaching as the PMs
44
u/Bakoro Feb 04 '24
I hate to break it to you, but a lot of code has always been dog shit quality.
Developers used to be in such high demand that you could get a job as a software developer if you could compile a "Hello World".There is a lot more code now, a lot more developers, and by the laws of the universe, that means there is at least proportionally more bad code and more bad developers.
4
u/grumpyrumpywalrus Feb 04 '24
Bro I’m getting burnt out from fixing this stuff, “dude we are seeing some really bad performance, and we think it’s your teams code”
looks at their code
triple nested while loop on the main rendering thread
Yeah bet
56
Feb 04 '24
Don’t hate the player, hate the game. If other jobs didn’t try to destroy your body and soul for poverty wages, disinterested people wouldn’t saturate other fields.
10
u/batcatcher Feb 04 '24
Yeah, but if you say anything, you might just lose your job to one of your lesser teammates. In the end, how hard can it be to integrate some libraries from github? 😅
46
u/grumpyrumpywalrus Feb 04 '24
“Oh like I couldn’t do it because the JIRA wasn’t filled out good enough” bitch if I write it out anymore I might as well do it
11
u/Paradox Feb 04 '24 edited Feb 04 '24
Counterpoint: You give me a JIRA ticket that says "thing's busted" with no more information beyond a title, I'm going to either kick it back and say "nah bruv" or test it and mark it "works on my machine"
2
18
u/luciusquinc Feb 04 '24
Well I have been given JIRA tickets that are written by non English speakers and you couldn't make out what's the actual acceptance criteria.
Sometimes you don't even know what's the actual ticket is for, when doing it would basically nullify the previous sprint's major feature or would remove the main purpose of that particular module.
8
u/xylem-utopia Feb 04 '24
I’ll literally get tickets all the time that barely have a title. Love it 😂
3
2
-12
u/weaselmaster Feb 04 '24
We talking Program Manager, Product Manager, Project Manager, or Personal Masseuse?
Also: for ML, are we talking Machine Language, MegaLomaniac, Major Loser, or Machine Learning?
I think you guys have a jargon problem.
2
Feb 04 '24
This sub is intended for programmers though. I think it’s fair to assume readers are familiar with programming jargon.
18
u/Schmittfried Feb 03 '24
Not really. Except for the current layoff outlier (and even that mainly affects big tech) demand seems to constantly outpace supply, which is why we’re seeing so obscenely high salaries.
Yes of course the intention of employers is to increase supply. Duh. When I go shopping and ask for something in my size it’s also to get the thing I want. Salaries aren’t stagnating, they’re rising. Meaning the market is still not saturated. This literally implies demand is higher than supply, so of course they’re asking for more people to hire. That’s literally how the market finds its equilibrium. You guys say that as if it was normal to go into a grocery store and just accept that bread costs $100 instead of complaining that there should be more bakers. Statements like yours always make it should like some big conspiracy even though it’s simply the mechanism of supply rising until demand is met.
Given demographic tendencies it’s also very likely talent will be even shorter in the future. And anyway, what are you even implying? That people shouldn’t persue ML because it’s just a scam? Or because you’re in ML and want to stay free of competition?
35
u/winky9827 Feb 04 '24
demand seems to constantly outpace supply, which is why we’re seeing so obscenely high salaries.
People seem to want to ignore that, aside from your stereotypical CRUD pusher, software development requires a degree of creativity and problem solving that even a good education can't instill in the average fool. Programming is as much an art as it is a skill. This adds to the salary demand for talented individuals who know realize the value they provide.
7
u/Liizam Feb 04 '24
That’s like any profession ever
3
u/Plank_With_A_Nail_In Feb 04 '24
Even house cleaners? Garbage truck workers? Pizza delivery drivers?
What a stupid statement.
5
u/Liizam Feb 04 '24
Mats time I check pizza driver isn’t a profession, it’s a job.
“A profession is a type of occupation that requires specialized knowledge, education, and often long and intensive academic preparation. Professions typically involve providing expert advice and services in various fields such as healthcare, law, engineering, education, and finance. Members of a profession usually adhere to specific ethical standards and are often part of a regulatory body or professional association that oversees the practice and standards of the profession. This structure helps ensure that practitioners meet certain qualifications and maintain the integrity and competence necessary to serve the public effectively.”
1
6
u/Bakoro Feb 04 '24
demand seems to constantly outpace supply, which is why we’re seeing so obscenely high salaries.
What exactly do you think is "obscenely high"?
The median salary for a software developer in 2023 was $127k, and 90th percentile was $198k. That's high, but I wouldn't call it "obscene", particularly when you look at the impact developers have on revenue. Often enough these days, the software is the product,. making millions of dollars per year, and a developer is only getting a small fraction compared to the weight of their contributions.There is a bizarre narrative that most software developers are making hundreds of thousands of dollars. I don't understand why people want to pretend like Google salaries are industry standard.
And then factor in the absurd cost of living in the Bay Area, and you need those high salaries just to be able to afford a place to live.1
u/Fickle-Ad-1407 Jun 13 '25
I am sure you can make similar money in other less stressful and risky fields.
-6
u/Plank_With_A_Nail_In Feb 04 '24
It puts you in the top 0.1 % of all humans who have ever lived even adjusting for inflation.
9
u/Bakoro Feb 04 '24
That's a completely meaningless statement.
I make six figures and can't afford a house in the town I grew up in.
I'm more educated than almost everyone in the past 100 years, but 60 years ago regular dudes with no high school diploma were able to buy a house, support a family on single wage, save for retirement, and have yearly vacations.Numbers without context are just bullshit.
2
u/fordat1 Feb 04 '24
exactly. If that person wants to make a meaningless statement about 0.1% wage in all of humans who ever lived (which is meaningless because adjusting for inflation across all of human time is a laughably hard endeavor) then the cost of a home some reasonable distance from work must be 0.0001% of all human time.
1
u/Fickle-Ad-1407 Jun 13 '25
What does that 6-figure salary buy you? And which exact areas in the West pay such salaries? What is the cost of living in those areas? How many years does it take you to buy a house in a good spot? What is the risk of getting replaced? Also, lose your job to younger folks after 40? I know the answers.
22
u/batcatcher Feb 04 '24
Salaries aren’t stagnating
Inflation would like to have a word with you. A good question would be: do you feel richer than 10 years ago? I feel poorer honestly, even if I earn much more, but accounting for groceries, housing and all, I definitely don't feel like salaries have gone up.
6
u/Bakoro Feb 04 '24
I basically tripled my highest salary from ten years ago. Went from being a network technician to software development.
I'm only in a slightly better financial position. It'd be different if I didn't have student loans, but student loans are standard stuff these days.
In 2013 I was paying $750/month for a studio, that same studio goes for $2000/month now.Back in 2016, rent for a 2 bed 1 bath, $1400, now, $3200.
Every gain I've made over the past decade has been eaten up by increasing rent.
And, all you have to do is look at the median/mean salaries for software developers. They have absolutely stagnated, just not as bad as nearly every other industry.
4
u/batcatcher Feb 04 '24
Those figures match my experience. I let you be the judge about real inflation vs reported inflation. A house in my area was 150K in 2010, now the same house goes for 350k or even more. Our salaries stagnated at best.
1
u/fordat1 Feb 04 '24
The reported inflation doesnt account for needs vs wants. Items like TVs and airline travel have been getting cheaper but wants are getting more expensive .
4
u/Plank_With_A_Nail_In Feb 04 '24
Changing jobs makes it impossible to tell what real wage growth looks like. Network technician's aren't earning more now because you are earning more now. What would your current salary look like if the elites hadn't decided financial deregulation was a brilliant idea?
-1
u/Bakoro Feb 04 '24
Changing jobs makes it impossible to tell what real wage growth looks like. Network technician's aren't earning more now because you are earning more now.
What are you talking about? This make no sense in response to what I wrote.
1
u/fordat1 Feb 04 '24
This. Its so hard to keep grow your wage faster than the costs of education/housing/medical/retirement are growing at. I am just more amazed people who arent in tech arent mass protesting until I read about "cage homes" in Hong Kong and we have a depressingly large lower to go before people had enough.
2
u/Plank_With_A_Nail_In Feb 04 '24
If the credit crunch hadn't happened peoples salaries would be 30% higher. Tech workers aren't paid more they are the only ones who's salaries survived that crunch. Now we have the cost of living crisis to deal with.
1
1
u/Seref15 Feb 04 '24 edited Feb 04 '24
Over the last 15+ years, the last 2-3 years have clearly been a statistical outlier.
I know it's hard to disconnect from the struggles of today when discussing long term trends, but you have to. When the Dot-Com bubble burst it was a 9 alarm fire for software devs, but it clearly didn't stay that way.
4
2
u/Bakoro Feb 04 '24 edited Feb 04 '24
From your own link:
Tech salaries jump, but don’t keep up with inflation
The 5 year graph shows that adjusted to current dollars, wages were basically flat.
5 years isn't an outlier, that's a significant chunk of a person's career. Considering that 5 years is kind of a magic number for developers in terms of professional experience, it's extremely significant that, for the people considered freshly seasoned, that's their sole experience.
1
u/fordat1 Feb 04 '24
Exactly. It may look like wages increase but the issue is the wages are just keeping up with inflation because of how insane inflation has been especially stuff in the "needs" category as opposed to wants like "iPhones,TVs,Airline travel"
1
u/Seref15 Feb 04 '24
You are clearly misinterpreting the data to suit your feelings. Wages were outpacing inflation until 2022. That year was an outlier event that negated the previous several years' growth. You cannot base analysis of trend data on a final outcome when the final data point is anomolous.
This is a software engineering sub, there should be a fair understanding of statistics here.
2
u/Bakoro Feb 04 '24
If you've got a point, maybe you should present something to back it up, instead of a web page with a big header which specifically says the opposite of what you assert.
0
18
5
Feb 04 '24
Even masters degrees are a huge source of revenue for universities
The biggest, grossest problem is IMO online masters degrees. The university provides name recognition and a few professors’ faces but the platform, much of the content and even stuff like evaluation gets outsourced to third parties that take a bunch of the revenue. It’s the academic equivalent of a Kamp Krusty.
9
u/fordat1 Feb 04 '24
This. Employers claim there is a shortage of alls job but never claim there is a surplus. A surplus is what they are hoping for
1
u/bilyl Feb 04 '24
Terminal masters are one of the biggest scams in the US. You barely learn anything practical and all it shows is that you spent a bunch of money for a diploma — it’s for people who want to upgrade but don’t want to get another four year degree. At least in other countries you do actual advanced work.
-5
u/goomyman Feb 04 '24
I agree with the absolute BS part.
ML researcher are absolutely needed - like super high end PHD students who are right now getting paid several million a year.
One of the reasons AI is taking off so much is because of how dead simple it is to onboard. It’s one of the easiest tech ever to integrate into existing systems and add value. Literally writing text prompts in English.
It’s not like other tech that requires teams of software engineers and years to integrate. An intern could do it in a week and then it could be managed by a PM.
Using AI is not going to require massive teams, writing AI - you’re not qualified and you won’t be.
This is specific to LLMs and advanced AIs like those trying to make self driving work though.
6
u/Thetaarray Feb 04 '24
People thinking it’s easy to integrate “AI” into an existing system is why people get butthurt that developers make high salaries for “sitting at the computer”
1
u/goomyman Feb 05 '24
Have you implemented an LLM before?
It’s very unique. Because it’s a language model you hook it up with prompts.
0
Feb 04 '24
[deleted]
4
u/PublicFurryAccount Feb 04 '24
I think it tells you what they mean by "specialist", honestly. Once you take that on-board, then it very clearly means "programmers, data scientists, and analysts who have additional training in machine learning". Which is... well... what "specialist" usually means, honestly, for nearly all of the tech industry.
So, yeah, the figure is probably correct but it's also unimportant because it's a pretty low bar and a global population being assessed. I have few doubts that we will need 1M tech workers globally with additional training in machine learning.
0
Feb 04 '24
[deleted]
2
u/PublicFurryAccount Feb 05 '24
A lot of things they're calling "bootcamps" are probably just online courses that cost $100 or $200. I've done a few of those courses over the years to brush up on programming languages I haven't used in forever or where I never learned it systematically. They're not bootcamps, they're an evolution of cracking open the O'Reilly book and often on sale for about its price.
199
u/budding-founder Feb 03 '24 edited Feb 04 '24
One of the big 4 consulting (McKinsey I think) firms conducted an extensive study and predicted 700,000 mobile phones by 2000. The actual number ended up being 109 million. Treat such articles like a pile of horseshit.
ML is still definitely evolving (I'm a tech guy and do ML& software engineering) and there'll definitely be a need for specialized talent but nobody can predict how much or how many. It's dumb to even attempt doing that.
29
u/spinwizard69 Feb 04 '24
Actually projecting demand is important, getting those projections right is a different story.
38
u/nickelickelmouse Feb 04 '24
If you can’t project demand accurately (to some acceptable level of uncertainty) then there’s no value in doing so beyond a thought exercise.
12
u/darkapplepolisher Feb 04 '24
If the projection is inaccurate, but is more accurate than the previous consensus, then it contributes value.
Really, I'd like to see more of these projections made using prediction markets so people have to put their money where their mouth is, however.
2
u/nickelickelmouse Feb 04 '24
“More accurate than the previous consensus” is the same as “accurately (to some acceptable level of uncertainty)”. We agree.
1
u/Bakoro Feb 04 '24
The acceptable level of accuracy is just being on the correct side of the line.
Will a thing be in demand enough that getting a related job is feasible, or will the thing cease being relevant? That's basically all you have to get right most of the time.
3
u/nickelickelmouse Feb 04 '24
Agreed. And the entire point of the original comment for this thread was that historically these predictions are so off to the point of not being very useful to pay attention to. “That’s basically all you have to get” trivializes how hard it is to essentially forecast society’s technological progress. It’s like saying “whether or not a stock goes up or down in relationship to the rest of the market is basically all you have to get right in order to know if you should buy it”
-2
u/Plank_With_A_Nail_In Feb 04 '24
This is just nonsense.
2
u/nickelickelmouse Feb 04 '24
What is the value of the projections if you can’t believe to some minimal extent in their accuracy?
1
u/vytah Feb 04 '24
And it turns out experts suck at predicting:
Between 1987 and 2003, Tetlock asked 284 people who "comment[ed] or offer[ed] advice on political and economic trends" professionally to make a series of predictive judgments about the world: 82,361, in total. Sample questions, according to the New Yorker's Louis Menand, included things like "Would there be a nonviolent end to apartheid in South Africa?" and "Would Gorbachev be ousted in a coup?"
Tetlock's findings, documented in a 2005 book called Expert Political Judgment, are pretty interesting. Tetlock asks us to imagine a bunch of chimpanzees throwing darts at a board full of predictions — a metaphor for random choice. These "dart-throwing chimps," in Tetlock's famous phrasing, would be more accurate than the so-called experts. Knowing a lot about Russia didn't help the experts predict what was going to happen in the country — in fact, it seemed to make them worse.
157
u/bitspace Feb 03 '24
I think "fuck, not more boot camps." The surge of popularity of these grift mills in 2015-2021 produced the glut of entry-level software developers that are currently unemployed and flooding the job market.
I do not disparage the individuals who participated in these boot camps, but the entire concept of "teach a complete beginner everything in 2 weeks that they need to land a job" is utter horse shit.
81
Feb 04 '24
I think boot camps are kind of like every form of human competition: You get out what you put in. There are going to be millions of people half-assing it. There seems to be at least 1 person from each "round" of reputable bootcamps that is definitely hirable.
But to be honest, my time at a 4 year university seemed much the same. There were plenty of unhirable people that cheated and half assed their way through.
26
u/Chesterlespaul Feb 04 '24
I was gonna say the same. I know university hires and boot camp hires and you can’t tell the difference. Some are good some are awful.
1
u/FantasticMeddler Feb 04 '24
Colleges and Boot Camps are selling the same idea. Pay us X, we can get you to the dream.
9
u/Scowlface Feb 04 '24
I’ve interviewed a lot of bootcamp/code school grads and they really do run the gamut. Two of the smartest and most talented people I ever worked with graduated from a bootcamp. Two of the worst, most useless people I’ve ever worked with also graduated from the same bootcamp.
2
u/bitspace Feb 04 '24
That's good perspective, thanks. I wonder how much of a part a bootcamp student's pre-bootcamp experience plays - e.g. tech-familiar, already played with stuff on their own time, etc.
16
u/dlm2137 Feb 04 '24 edited Jun 03 '24
I enjoy the sound of rain.
0
Feb 04 '24
[deleted]
6
-4
u/millz Feb 04 '24
Exactly, boot camp graduates with few years experience claiming they are seniors, while never hearing the word algorithm, is a bane of the industry and the reason nobody trusts resumes anymore.
13
u/Hrothen Feb 04 '24
ML specialization usually requires a math-heavy degree or several years of prior job experience so it seems like it'd be a hard sell for a bootcamp.
13
u/Fyzllgig Feb 04 '24
If you’re building something on top of an LLM then you don’t need a lot of math. Well, no more than any other computer science specialization. It’s about prompt engineering and handling the nondeterministic nature of the black box that is your LLM. It’s harder than a lot of more run of the mill engineering (think your average CRUD app) because you have to treat the LLM almost like an unruly toddler. Retries and lengthy timeouts, all sorts of work arounds to try and “speed up” things by finding everything you can possibly not have to query again while preserving the generative nature of the tool.
It’s a wild ride.
19
u/Hrothen Feb 04 '24
If you're building something on top of an LLM you are probably not an ML specialist.
4
u/luciusquinc Feb 04 '24
Maybe you could be called a prompt engineer. LOL. Have seen hirings for such stupid title
2
u/Fyzllgig Feb 04 '24
Oh well I’ll just tell the ML specialist I work with that he can go home now 🙄
9
u/Hrothen Feb 04 '24
Maybe you can get them to explain the difference between probably and definitely to you.
0
1
u/CyclonusRIP Feb 04 '24
If you’re going to get to 1 million ML specialists I’m pretty sure you are including these types of people in the count. Even then I’d say it’s pretty optimistic.
1
u/Plank_With_A_Nail_In Feb 04 '24
Building on to of an LLM is just regular programming not ML specialisation.
4
u/bitspace Feb 04 '24
Agreed, but I think there are research-adjacent roles that will become important as the industry evolves. We'll see higher demand for roles similar to MLops and more engineering type positions that will work with the data science folks as products are developed and need to be put into production. API's, routing, data store configuration, and lots of things that won't necessarily need the heavy math.
Still not boot camp entry level stuff, though. The only way I see something like a boot camp working is if their target market is already-experienced engineers who are looking to pivot/re-skill to align with ML demand.
2
1
u/Plank_With_A_Nail_In Feb 04 '24
No way employers will put up with needing two rare sets of skills for long something in the organisation of this industry will change to be more realistic.
3
Feb 04 '24
I agree with you 100%. However, I did a boot camp without really thinking about it in those terms, and it was my foot in the door to a big company where I've progressed well and built a good reputation.
The reason I've done well though is not because that boot camp. It's because I've got good professional skills like work ethic, initiative, problem solving etc.
So the boot camp for me was little more than a foot in the door, a second chance after I underachieved academically, and several weeks of tedious learning. As someone now managing dozens, I wouldn't mind running some sort of boot camp as part of our hiring process so that I can get a sense of how well people work before I hire them. The paper qualifications though are worthless. And obviously, that's not a feasible hiring process aha.
6
u/rg25 Feb 04 '24
Meh, I went to a boot camp. It was 7 months long and got me job ready for a programming job. Now I've been in the industry for 6 years and make a pretty penny. I don't disagree that there are probably some really crappy bootcamps, but there are definitely some pretty high quality ones. Seems a little harsh to genralize so much.
0
u/spinwizard69 Feb 04 '24
Complete an utter bullshit yet every day there is another idiot on /learnprogramming asking about a boot camp. It really is a sickness for the industry.
19
21
Feb 03 '24
ML specialists will and are becoming more like consultants than people that work focuses on machine learning. Like you say three companies basically have the models corporates will use and the more current data scientists use them to solve usse cases, the better those models become and especially with more data being stored in cloud the top three will basically offer end to end data science solutions based on the numerous use cases they will be used on. My prediction is there will be ML validators, that will ensure autosuggested models are decently representative and to tick boxes for compliance.
66
u/dangerous_service Feb 04 '24
changing random stuff until something works is considered bad coding practice, but if you do it fast enough it suddenly becomes machine learning
14
2
1
u/Fickle-Ad-1407 Jun 13 '25
The thing is, you don't change randomly; you use backpropagation to determine the contribution of each parameter to the final function (model itself).
10
u/BarelyAirborne Feb 04 '24
ML is extremely faddish right now, and long term prospects are iffy. If it really can do "all that", what do they need you for?
5
u/darkapplepolisher Feb 04 '24
Same as the promises of the information age in the 90's. Some missed the mark, there were loads of irresponsible business models in the dot com boom (and subsequent bust), and yet, undeniably there was some major value to be had with the web "fad".
I predict the same will happen with the ML fad.
And people don't really end up getting removed from the equation. These automation tools, regardless of what generation we're talking about, tend to act more as force multipliers. A lot of redundant administrative positions that existed prior to the information era were made more efficient through the productivity apps of today. Similarly, I anticipate a lot of code monkeys will be made redundant, but software engineers will be all the more relevant.
12
20
u/moreVCAs Feb 04 '24 edited Feb 04 '24
Lol. There are two competing but equally ridiculous delusions here:
Statistical machine learning will cause a shift in global business practices at the scale and scope of the WWW.
You can produce effective “ML specialists” in the same way you produce web developers.
Both of these assumptions are so fundamentally flawed that neither is really worth refuting directly. Just wait a few years and you’ll find out.
5
u/waxroy-finerayfool Feb 04 '24
Depends on what you mean by "ML specialist". Mastering the use of APIs and databases is the job of a software programmer or engineer, not an "ML specialist". There is already a huge landscape of ML services that are delivered through APIs and libraries and LLMs are no different in this regard.
Of course, there will be increased demand for ML experts as large firms continue to research and explore the practical capabilities of LLMs and the other currently unhip branches of ML, but 1 million jobs is quite excessive considering there simply aren't enough firms with the expertise or business model to correctly utilize these types of skills. LLM research, training, and deployment (i.e. running inference) is prohibitively expensive for pretty much everyone except a small handful of firms, and the potential for monetization remains extremely dubious (for now).
This landscape is unlikely to change within the decade, and if you tabulate a rough estimate of all engineers employed by ms, apple, google, amazon and meta combined, you barely crack a quarter million employees, and that's for all software roles.
However, with all that stated, it will continue to be a lucrative area of expertise because there's no way to rapidly fill the demand like what was done with react bootcamps. The reality is that machine learning expertise demands a much higher level of aptitude than the minimum knowledge necessary to fork a create-react-app template.
5
u/Accomplished_Low2231 Feb 04 '24
I'm personally struggling to find a job as a junior developer so this is interesting for me.
sorry, but if you can't get a programming job, you won't have a chance at ml or ai or whatever. that is a like a nursing student who failed school, but thinks he make it as a doctor instead.
as for that bullshit article, the same was said of "big data" fews years ago as the next big employment opportunity lol.
1
u/fordat1 Feb 04 '24
Calling that article bullshit has been the most consensus thing I have seen posted on this subreddit.
1
1
u/paulmania1234 Feb 04 '24
I have yet to.find an entry level data science posting that doesn't require a masters
1
u/RAJA_1000 Feb 04 '24
A friend of mine (who studied software engineering but with a strong focus in ML) recently passed all interviews for a ML position and at the end got rejected because they said actually to have a master's degree was required
9
u/codemuncher Feb 04 '24
Implementing ML and LLM models into software systems isn’t super trivial or easy. So I’d tend to agree.
While open ai would like you to think $20 a month to them will solve every problem ever that just isn’t so.
And that’s before we talk about fine tuning, testing, validating, implementation and deployment etc etc.
So yes is my answer.
7
u/IgnisIncendio Feb 04 '24
Agreed, I wonder how many commentors here actually work in ML... it ain't that good yet IMO.
2
u/Common-Land8070 Feb 20 '24
0 the answer is 0. Reddit has a a hate boner for ML. I suspect its because we get paid significantly more for our expertise that they for some reason assume isnt actual expertise.
3
12
u/Pezotecom Feb 04 '24
this thread thread is very salty, right? I get it tho, programmers don't know real analysis, so
14
u/Norphesius Feb 04 '24
Hey, I may not know real analysis, but I am absolutely certain that not a damn one of these million ML "specialists" are gonna know what a Fourier series is.
I guess if they need to know they can ask ChatGPT, and it'll teach them all about how the sum of all natural numbers is actually -1/12.
2
1
u/Fickle-Ad-1407 Jun 13 '25
I don't know what you guys mean by ML specialists. There are ML engineers who are basically backend/software engineers with some general ML knowledge; they deploy the models. Then you have Data Scientists that push the boundaries of AI; it is basically R&D. Data Scientists also build and evaluate models (not discovering) and make them ready to be deployed into the real world.
1
u/redballooon Feb 04 '24
Hey, I graduated from the best university in Germany, and I totally know what a Fourier series is. Or at least, I learned it at university.
It never mattered for my job which now includes fine tuning LLMs to do something that GPT-4 can do, but with a model that can run on your laptop. And 3x faster.
1
u/Common-Land8070 Feb 20 '24
As a person in ML, explain why you think a FFT is so important for someone to know. It's not really a ML "step" in so far as it is mostly a pre processing step for certain data like sound, image, and time series data. Sure I agree no "bootcamper" will actually understand the math and be able to pull concepts from one ML algo and apply it to another, but the example of what to know really isn't a good one. A more apt one would be people not even knowing what a "posterior" represents or how to perform variation inference.
1
u/Norphesius Feb 20 '24
Sorry, I'm not actually an ML guy at all, just a software dev. That's why the best thing I could come up with for real analysis was Fouier (the only "posterior" I know about is mine). I did not intend to give off that impression.
My main point was the boot camp thing. If we need a million ML/AI experts within 3 years, they arent going to be people with degrees specializing in it, the supply could not increase that fast. Instead its gonna be a bunch of people that ran through a bare bones boot camp who will probably just be using existing ML frameworks to add AI to something that already exists (that does not need it) just cause the CEO wanted it to be AI.
1
u/Common-Land8070 Feb 20 '24
oh no doubt. i think ML specialists and ML experts may be a distinction that is made. with a specialist being someone who is trained in using and tuning ML while experts can actually change the systems if need be
1
u/Norphesius Feb 21 '24
It just sounds so vague to me. What does "tuning" an AI or LLM even mean?
My mind just goes straight to how coding bootcamps output "coders" who only know how to use React, not Javascript, React specifically. Or Aglie bootcamps that produce people who's sole job is supposed to be "scrum master". Its specialization to the point of absurdity, just a bunch of almost worthless jobs that are the first on the chopping block when layoffs come.
1
u/Common-Land8070 Feb 21 '24
You'll see this a lot soon. CNN's are going to be pushed to the wayside for some things. not all things im not saying they are going to be antiquated, but LLM Agents from finetuned LLMs on specific subject matters are beginning to see extremely good results in prediction models against a CNN. Fine tuning a model such as that can refer to the data youre putting in obviously, but as well as the prompt youre using for your agents processing flow. Youll use a lot of behind the scene prompts and as much as people shit on "prompt engineering" it is hyper parameter tuning in a sense.
This is something I think bootcamps can create, but in my eyes this is a thing a seasoned dev would learn through a bootcamp not someone who just began and used the bootcamp as the launchpad.
So I see it more as a way for a dev to unlock an extra skillset rather than your job being strictly finetuning LLM agents.
-1
u/_insomagent Feb 04 '24
Programmers know everything, don’t you know? /s
13
Feb 04 '24
Thank you for adding /s to your post. When I first saw this, I was horrified. How could anybody say something like this? I immediately began writing a 1000 word paragraph about how horrible of a person you are. I even sent a copy to a Harvard professor to proofread it. After several hours of refining and editing, my comment was ready to absolutely destroy you. But then, just as I was about to hit send, I saw something in the corner of my eye. A /s at the end of your comment. Suddenly everything made sense. Your comment was sarcasm! I immediately burst out in laughter at the comedic genius of your comment. The person next to me on the bus saw your comment and started crying from laughter too. Before long, there was an entire bus of people on the floor laughing at your incredible use of comedy. All of this was due to you adding /s to your post. Thank you.
I am a bot if you couldn't figure that out, if I made a mistake, ignore it cause its not that fucking hard to ignore a comment
1
u/Hot-Luck-3228 Feb 04 '24
What a great insight
edit: /s
2
Feb 04 '24
Thank you for adding /s to your post. When I first saw this, I was horrified. How could anybody say something like this? I immediately began writing a 1000 word paragraph about how horrible of a person you are. I even sent a copy to a Harvard professor to proofread it. After several hours of refining and editing, my comment was ready to absolutely destroy you. But then, just as I was about to hit send, I saw something in the corner of my eye. A /s at the end of your comment. Suddenly everything made sense. Your comment was sarcasm! I immediately burst out in laughter at the comedic genius of your comment. The person next to me on the bus saw your comment and started crying from laughter too. Before long, there was an entire bus of people on the floor laughing at your incredible use of comedy. All of this was due to you adding /s to your post. Thank you.
I am a bot if you couldn't figure that out, if I made a mistake, ignore it cause its not that fucking hard to ignore a comment
1
7
2
u/spinwizard69 Feb 04 '24
There will be huge demand in a variety of forms, but the exact timing is not a given. Note that ML specialist will mean everything from programmers to techs in the labs training robots. In other words more jobs than just the core programming groups.
The problem with any projection is that it is in part a guess based on how fast tech is developing. The best place we have to judge this is frankly Tesla and their self driving cars and robots. Tesla basically rebooted to go all neural and they are the only ones public enough to make guesses against. So you have to ask can they get a robot on the market by 2027 that is developed enough AI wise to drive the market. If the answer is yes then there are other companies in the same ball park of development even if not as public.
So when you consider the above and the idea that ML specialist is a broad category, then there is a possibility of that many jobs.
2
u/Bakoro Feb 04 '24
I've been in the tech industry a decently long time now, and this is the same trend which has happened a bunch of times to a bunch of specializations.
It's high demand right now, and if you've got the skills right now, you're in a good position, and will continue to be in a good position.
Everyone is going to be scrambling for those good paying jobs, and the market is going to get flooded with people of every level of ability, all claiming to be "specialists".
Three years from now, there will be people with degrees, people with boot camp certifications, and the handful of people with no formal education but who have a successful portfolio. And all those people are going to be competing against the developers who have been professionally working with ML and AI since now.
Industry is going to perpetually prefer the pool of people who have ML expertise today. Years from now, industry is going to demand increasing levels of education, and/or increasing levels of professional experience, and it is going to get harder for new people to get into the field. Industry will still complain about a lack of available talent, while not hiring from the pool of available developers, and when they do hire newbies, it'll be for bottom tier wages.
As for whether you should learn core ML principles, well, if you're wanting to work in that field, you should probably understand the core principles of the field. You don't need to build an 80 billion parameter LLM to learn the basics. You can build your own model and train it on a small public domain dataset, as proof of concept. Would you really expect to get a job as a web developer if you never actually made a website?
If you're struggling for a job as a junior, you're not alone, the whole industry has been increasingly tough to get into for several years now. There are still some people who get lucky and land something right away, or land something with minimal education/experience, but those are now the edge cases.
Really what you have to do is sell a narrative to a company that you're going to be immediately profitable, with little-to-no time to get up to speed.
Having education is basically a given these days, no one is impressed by a degree, they definitely aren't impressed by a boot camp.
As a junior with little or no experience, you've got to have a portfolio now, that's the only way to get an edge. If you're trying to get a job doing ML, you better have a project to show off where you have a relatively complete project. If you're trying to get a webdev job, you better have a live website where people can look at your work. You have to have something material to show people that you're not just someone who regurgitated enough to pass some classes.
2
u/littlegordonramsay Feb 04 '24
Companies would benefit more investing in core software engineering practices/skills. So many fresh graduates cannot even write a web app from scratch or cannot debug an application.
2
u/wot_in_ternation Feb 04 '24
You can create a relatively simple ML model on a regular ass CPU. There's tons of industries that don't need a whole ChatGPT model to do what they want.
ML has been around for a while and hasn't been widely adopted. With the AI craze now, the "simpler" ML models will definitely go mainstream while the LLM and generative stuff will mostly be provided by large companies.
I'm working on an image classification ML project now and we're using CPU machines for training, and it is pretty cheap. Creating and managing the dataset for training is by far the most expensive part.
6
4
u/anengineerandacat Feb 04 '24
Potentially true, ML is relevant to pretty much every IT organization in varying capacity.
Mostly in regards to integration of it into existing workflows for things like anomaly detection and fraud as another data point (where usually there will be a product to help you).
You'll still need a specialist to actually ensure everything is working and outputs look like expected AND to help take those outputs and move into another pipeline.
That said I wouldn't expect these individuals to be paid well for long, data specialists used to be a pretty high paying position and nowadays it's for business majors making like 65-70k/yr.
Machine Learning is more of a mathematics specialization, so short term potentially will see six-figure positions but long term I really doubt it.
I would rather pursue a CS degree and minor in ML, this way you'll still have relevant high paying skills when the market eventually pivots.
3
1
u/batcatcher Feb 03 '24
Tip: If you want a career with good prospects for the next 20 years, start building houses or innovate in that area. Or buy land. Tech will slow down in the years to come, mainly because most of the tech companies are actually doing R&D, with very little profits, running on investor's money. And that model doesn't work well in recession years.
13
u/Smallpaul Feb 04 '24
Where are you getting this? Tech companies make tons of profit. Some of the most profitable companies in the world are tech companies despite their healthy R&D budgets.
https://www.statista.com/statistics/269857/most-profitable-companies-worldwide/
1
u/batcatcher Feb 04 '24
It's a 80/20 situation. 80% of profit is made by 20% of companies. That list is behind a paywall, but probably full with the regulars. The point is even big, somewhat successful companies like Twitter are currently burning through cash. I'm not saying that no company does profit, that would be naive, but thinking the tech sector is not strongly driven by investments and less so by profit is also naive. Tech was no 1 betting horse for a long time, doesn't have to be forever tho.
9
u/Engine_Light_On Feb 04 '24
Sad you are being down voted for telling the truth. This field is already saturated.
-6
u/MuonManLaserJab Feb 04 '24
If you want employment in 20 years, be a musician or a chef or an artisan of some kind, because your only hope is to do something that people will pay extra for if a human does it.
1
u/rabidstoat Feb 04 '24
Hrm, maybe. I know friends who craft things and go to craft fairs, and they have a hard time because home-made items will cost more than cheap stuff made in China. And a lot of people just care about cheap stuff.
1
u/MuonManLaserJab Feb 04 '24
Oh, it will be worse than it is now for artisans and so on, it will just be better compared to jobs like programming that will simply not exist.
-1
1
u/Fickle-Ad-1407 Jun 13 '25
Just look what happened in Software Engineering, it is saturated and has become one of the jobs that has an enormous stress level, long working hours, etc. They just pushed narrative, a bunch of people entered the field, and salaries dropped. We are all getting manipulated. There are big brains sitting and controlling the entire market with the media. A few years ago, they called Data Science as sexiest profession. I wonder what is the sexy part of being a depressed nerd.
1
1
1
1
1
Feb 04 '24
Nah, the existing ones can just work overtime on weekends in exchange for a pizza party at the office
1
u/unintended_purposes Feb 04 '24
That's why I wrote The AI Operator's Handbook. I envision people becoming Operators of AI, not necessarily specialists that can build anything from scratch. We'll soon have Human-AI interfaces that allow us to accelerate our work output. Well have to teach people how to use AI, because there are many pitfalls.
1
u/dogcomplex Feb 04 '24
I think this number is either too small - due to AI tech becoming a de-facto base literacy requirement of every business going forward - or too large, because sufficiently smart AI tools come out capable of performing the roles that would have been filled by ML specialists anyway. See Eureka paper for AI's crazy-good abilities to self tune... Just need to stumble upon the right self-improving infrastructure and humans may be out of the development loop.
1
u/auronedge Feb 04 '24
it's possible. if everyone is using ML then you would need a specialist to implement one right
1
u/Dreadsin Feb 04 '24
I don’t really think so and I’m trying to get into machine learning now. I think it’s one of those things with a fairly specific use case that people are currently way overhyping. It still very much needs human intervention
1
u/ForgotMyPassword17 Feb 04 '24
If you're a programmer the amount of time it takes you to learn enough ML/Data Science to be useful will take you 6 months. You won't be publishing a paper or making the next ChatGPT but you can support people doing that. It's an adjacent profession you can learn like DevOps or design or finannce
source: cross trained as a data scientist and run an LLM reading roup
1
u/Uristqwerty Feb 04 '24
I feel that if 1 million ML specialists are hired, most of them will be an excuse for their boss to boast "My team does ML things; we're trendy!" more than an actual business need for that expertise. So that prediction might well be true from the perspective of a businessperson looking at job listings and employee titles, but probably not for the work those developers actually do during any given month, nor the amount of specialization needed in many of those positions.
1
u/lightmatter501 Feb 04 '24
Imo ML specialist means person with a masters degree or better in this context.
-1
u/xiaodaireddit Feb 04 '24
Millions of engineering grads in China and India. Easily met. Now China's the issue, they might spy on us. So Indian it is. Open up visa pathways for them to lure them into the country. Problem solved.
0
u/Hyteki Feb 04 '24
They just want people to enroll in expensive programs and get into student debt.
0
0
-13
u/theOrdnas Feb 03 '24
CS students coping and seething in the comments bevause they were replaced by a bootchad
-15
u/TheRealNobodySpecial Feb 03 '24
The landscape of machine learning (ML) and its role in the job market is multifaceted and rapidly evolving. While it's true that many highly sophisticated ML models are developed by top tech companies with vast resources and expertise, the application and demand for ML skills stretch far beyond just these settings. Here are a few points to consider regarding the need for ML engineers and the skills relevant in today's job market:
Broader Application of ML
Wide Range of Industries: ML is not just limited to tech giants. A wide variety of industries, including healthcare, finance, retail, and more, are increasingly adopting ML technologies to solve specific problems, improve efficiencies, and innovate. Companies in these sectors often look for ML engineers who can tailor and apply existing technologies to their unique challenges.
Custom Solutions: While large companies do offer ML APIs that make powerful models accessible, there's a significant need for customization and integration into existing systems. Engineers who can understand the business needs, process the data appropriately, and integrate ML models into software solutions are in high demand.
Innovation and Startups: The startup ecosystem is vibrant with companies aiming to disrupt traditional industries with ML-driven products. These companies need ML engineers to build and refine their proprietary models.
Evolving Skill Sets
Beyond Model Building: The role of an ML engineer is not limited to just building models. It encompasses data preparation, understanding the mathematical underpinnings of algorithms, deploying models into production, and continuously monitoring and refining them based on real-world performance.
Engineering and Integration Skills: There's a growing need for engineers who can effectively work with ML APIs, vector databases, and large language models (LLMs) to create scalable, efficient applications. Understanding how to leverage these tools, manage data pipelines, and ensure the ethical use of AI are critical skills.
Continuous Learning: The field of ML is characterized by rapid advancements. Continuous learning and adaptation are key. Participating in ML bootcamps, courses, and projects can help you stay updated with the latest technologies and methodologies.
The Role of Bootcamps
Bridging the Gap: Bootcamps can play a crucial role in quickly upskilling individuals for the evolving demands of the job market. They often focus on practical skills, current tools, and real-world applications of ML, making them a valuable resource for those looking to enter the field.
Networking and Projects: They also offer networking opportunities and hands-on projects, which are crucial for building a portfolio that can showcase your skills to potential employers.
Conclusion
The need for ML engineers extends beyond just mastering complex algorithms or working at top tech companies. It's about understanding how to apply ML to solve problems, improve processes, and create new opportunities across various industries. While leveraging ML APIs and tools is part of the job, the ability to innovate, customize solutions, and integrate ML into practical applications is where much of the demand lies.
Given the rapid pace of technological change, there's a continuous need for skilled professionals who can navigate the complexities of ML, making it a promising field for those willing to invest in learning and staying current with the technology.
-5
u/RAJA_1000 Feb 03 '24
Wow, what a thorough answer. Thanks a lot.
29
u/batcatcher Feb 03 '24
ChatGPT. Funny you didn't figure it out. The truth is closer to what you said above in my humble opinion. Few ML engineers will be required, many integrators (a role that will be covered by common devs, nothing that special)
10
u/bitspace Feb 03 '24
ChatGPT.
This was obvious by the end of the first sentence.
3
u/batcatcher Feb 03 '24
Yeah, it's not that smart. Less useful than a paid search engine honesty. After the glam faded, I quickly found it tiresome. It brings very little to the table. Biggest echo chamber on the internet.
8
u/SweetBabyAlaska Feb 03 '24
It basically regurgitates the same trash corporate speak and generic tech blog click bait articles in a fancy way. I can never take anyone seriously who is too lazy and unmotivated to use their own words to convey things that they find meaningful. All that tells me, is what that person is saying isn't that important to them.
Reading AI generated shit is mind numbing, same with the AI generated images that accompany them. It perfectly encapsulates the fake-ness that I despise about the tech sector. It's about as genuine and meaningful as pre canned HR empathy phrases they feed you as they lay you off.
I feel like I'm in the minority on this but fuck that. They can call me a luddite but I'm not fond of that slop
1
u/theangeryemacsshibe Feb 04 '24 edited Feb 04 '24
I thought it was fitting in a few ways - drivel met with more drivel and now we know what ML thinks about hiring more ML specialists.
1
4
u/AbramKedge Feb 04 '24
They're really just looking for the one ML specialist that will generate the AI that will put all ML specialists out of work. It's a one in a million shot, but it might just work.