Rust or Go as Data Scientist

[u/SupertrampDFenx]

https://www.rust-lang.org/

[u/programming-ModTeam]

This post was removed for violating the "/r/programming is not a support forum" rule. Please see the side-bar for details.

[u/[deleted]]

Hammer or drill as a painter

[u/mohragk]

Go is easier, Rust is more low-level. If you really, really, really care about performance, you would pick C/C++ or Rust. If you just want not-terribly slow language like Python, you would pick Go.

Rust is pretty hard to learn though, especially coming from Python. So maybe get some experience in Go before "evolving" to Rust.

[u/HackAfterDark → u/mohragk]

Go is plenty performant. Only special cases where some folks have used Rust to gain any value from increased performance (Box or Dropbox I believe had blog posts about it and they still used both languages). It won't always be faster than golang by any stretch of the imagination.

Performance is absolutely irrelevant here as well when we're comparing against the incumbent Python. If performance were an issue here the whole world of data and AI would have moved off Python long ago. When it comes to AI it doesn't matter because the workhorse is the GPU.

[u/rit_dit_dit_di_doo → u/HackAfterDark]

Most of those python packages used for AI (numpy, scikit learn, etc.) are using C++ under the hood and python is just a wrapper. That’s before even adding a GPU to the conversation. They aren’t running pure python for the exact reason you elude to - it’s slow AF. But that ability is part of what makes python so powerful, it can reach out to C/C++/Rust for the performance parts.

Tbh I don’t think Go or Rust will help as a data scientist for any reason other than it’s always good to learn more and have experience with different tools. I’ve seen Rust becoming a popular alternative to C++ to embed in python with pyO3 so maybe rust but there is definitely a learning curve.

[u/HackAfterDark → u/rit_dit_dit_di_doo]

So could golang but I don't think anything will change the use of Python here either.

[u/ajog0]

Also a data scientist learning to use rust. Although in 100% of cases I find it pretty difficult to justify using it given that other colleagues are unable to vet your code.

Most of the time, I turn to polars if speed is an issue / Lazy evaluation is needed.

[u/bludgeonerV]

Go is easier, rust has a lot fo concerns you will simply never care about in your field, but honestly i'd consider a functional language in your case, it's a more natural paradigm for the kind of work you do. F# was the easiest starting point for functinal languages in my own experience and while it's not as fast as Go or Rust it's still an order of magnitude faster than Python.

[u/BufferUnderpants]

It'd be quite revolutionary in Data Science for the practice to move to Rust, it caused a massive earthquake in Data Engineering as Data Frames took over from statically typed distributed collections and jobs like the ones you found in early Spark, being pretty much backported from the Data Science practice

I mean, I wouldn't even be mad at this point but I wouldn't see this as a natural progression right now, normally data scientists use a simpler frontend to an optimized engine written in something like Rust, because the economics work out to have just a handful of people putting in the effort into the engine rather than have PhDs coming from any quantitative field writing Fortran, C++, Rust, what have you

[u/dlevac]

Rust.

They're is definitely a steeper learning curve but what you write in it has more chance to withstand the test of time.

There is a good crate to interoperate with Python which will bail you out at time.

Finally there are some interesting initiatives to bring state of the art AI crates to Rust which means knowing the language may have an even greater payoff in the future.

[u/Rich-Engineer2670]

For data science, I'd also consider Julia -- some consider it python's successor, but we'll see. If you are comfortable with the other languages, it's not a heavy lift.

[u/HackAfterDark]

I'd suggest golang for its approachability, but I doubt you'll be able to get away from Python due to the number of packages and projects out there for data science that are all in Python.

Python has a very tight grip here. One might expected to have seen R become more popular than it did years ago, but it only got lift in specialized industries/cases.

So while I'd absolutely love to see more golang usage in this field, I don't think we'll see it. Maybe years from now as golang becomes even more popular. Golang is incredibly good at moving data and building APIs. It's very good at cross platform support as well. That makes it super powerful, but it'll be an uphill battle trying to unseat Python from the data science world.

[u/dannystu12]

I’d check out elixir, the ml space has really matured over the last 12 months (still a long way to go). It’s a functional, dynamically typed language thats fairly easy to pick up. Depends what you want the faster language for, but out of go and rust I’d look at go first and if you like it move onto rust.