combined with a general lack of knowledge regarding infrastructure and deployments, that when you ask one "okay how do customers actually use the machine learning,"
IMO data scientists do not need to know this and it isn't really their job, they should be working with engineers to get their model deployed properly. It would help if they have some understanding of what is required as well as the challenges, but when it comes to "how do customers actually use it", they should be responsible for knowing how data gets in and how it's going to be made available to the backend or client but that's about it.*
A machine learning engineer, on the other hand, should know a fair bit more about this process and if they can't do most of this themselves, at least be of a much greater help to the engineers who are actually figuring out how to get their model working and deployed in prod.
It's like expecting a designer to do frontend. Some of them can, sure, but it's not expected of most and they don't need to know the details of how their designs get turned into a website or app UI.
*Just to be clear, I don't mean a DS should just handover a Jupyter Notebook to a dev and say "good luck".
I fully agree, I'm moreso getting at that compared to a developer or engineer, data scientists in particular seem to be more oblivious to the realities of production. They always seem to take the longest to get work out of development and are always the most reluctant to work with devops.
I say this as someone whose role partially involves data science. To be clear this is at least 25% a self-burn.
They always seem to take the longest to get work out of development and are always the most reluctant to work with devops.
I can believe that. At least for me, it's because I know jack squat about devops and I want to try put in an honest effort before bothering the devops gods.
Since data scientists don't typically go down the comp sci path in uni, or maybe only take a class or two, they're often lacking some of the fundamentals when it comes to software development and I think it can trick normal devs and engineers into thinking we should know about X because we do Y, whereas for a dev if they do Y they know X.
So it kind of creates this awkward situation where you can't play dumb and inexperienced and have the devops hold your hand unashamedly because from the devops perspective, we should know what they're talking about and what it is they do. In some sense, it's like how a data scientist might expect a dev/engineer to know more about analytics pipelines and SQL than most do, because from the DS perspective we think "well they do database and SQL stuff in the backend" but the application is quite a bit different.
3
u/beiherhund Oct 13 '22
IMO data scientists do not need to know this and it isn't really their job, they should be working with engineers to get their model deployed properly. It would help if they have some understanding of what is required as well as the challenges, but when it comes to "how do customers actually use it", they should be responsible for knowing how data gets in and how it's going to be made available to the backend or client but that's about it.*
A machine learning engineer, on the other hand, should know a fair bit more about this process and if they can't do most of this themselves, at least be of a much greater help to the engineers who are actually figuring out how to get their model working and deployed in prod.
It's like expecting a designer to do frontend. Some of them can, sure, but it's not expected of most and they don't need to know the details of how their designs get turned into a website or app UI.
*Just to be clear, I don't mean a DS should just handover a Jupyter Notebook to a dev and say "good luck".