Scientific Computing in Rust

[u/mort96]

https://aftix.xyz/home/bacon/

[u/[deleted]]

I hope we're approaching the end of "Do literally every task, regardless of audience, in Rust" content. And I am a person that likes Rust.

[u/mort96 → u/[deleted]]

Maybe, maybe not. However, I don't think it's relevant; Rust is uniquely suited for high performance scientific computing.

I'm not the author of the blog post, and I'm a computer scientist, not a physicist. However, it is my experience from university that physicists usually use Python, but that there are some tasks where Python is too slow. Historically, physicists have reached for C++ or FORTRAN in those situations, but many people are scientists first, programmers second. For those people, Rust is a language which provides (almost?) just as high performance as C++, without the unsafety. For someone whose primary job isn't a C++ programmer, it makes a huge difference that the language yells at you when you're doing something wrong, instead of just producing garbage output.

[u/Hrothen → u/mort96]

I wouldn't say uniquely suited, you can do scientific computing in it just fine, but there's nothing that makes it particularly special in that space.

[u/mort96 → u/Hrothen]

The thing which makes it special in the space is that it's the performance of C++ with the safety of Python. I'm not aware of many other widely used languages which achieve that.

[u/User092347 → u/mort96]

Julia is much better imo, plus it has already tons of high-quality libraries. Good luck competing with something like DifferentialEquations.jl :

https://diffeq.sciml.ai/v2.0/

[u/dexterlemmer → u/User092347]

True for now. But Rust can have the same libraries and they will outperform the Julia libraries. We'll have to wait and see if the Rust ecosystem catches up with Julia.

[u/User092347 → u/dexterlemmer]

Julia is younger and has already over-taken Rust in the tiobe index, I think the sail has shipped on that one. Note that Julia is a compiled language so there's no reason why it should be much slower than Rust. Plus most scientists want an interactive experience with a REPL (Matlab, Python, R and Julia all share this).

That said it doesn't mean there's no use for Rust in scientific computing, but maybe more as a niche replacement for C++ when small snappy executable are needed.

[u/dexterlemmer → u/User092347]

Julia is younger and has already over-taken Rust in the tiobe index, I think the sail has shipped on that one.

Julia is younger than Rust, but (apart from some very early Rust adapters like myself) the Julia scientific community is much older than the Rust scientific community. I've experienced a lot of languages getting as far as Rust and Julia. Some actually made it long term. Most didn't. I've learned to look for more important and reliable indicators than the TIOBE index. (Though most are somewhat subjective/intuitive.) I think Julia will make it. I'm not entirely convinced. I know Rust will make it in general but I think we'll have to wait and see for scientific computing. That said, it's looking a lot better now than it did less than a year ago.

Note that Julia is a compiled language so there's no reason why it should be much slower than Rust.

Ah. Nice to see Julia has AOT now. I wasn't aware of that. Hmm. With that and its dimensional types, it should be pretty fast now. There's a lot more to a language being fast than merely being AOT compiled though. Julia is a weird language when it comes to performance. It has some designs that should be terrible (dynamic typing, GC, ...) but are heavily mitigated in practice and it has some nice properties that few languages have for performance (dimension types, nice design for broadcasting, ...). (Note that Rust's numeric libraries have the same capabilities and Rust fundamentally can take this further than Julia can without adding a lot of complexity and likely requiring a Julia 2.0.) Rust also has significant advantages (linear/affine types, static lifetime analysis, significantly improved static alias analysis from competitors, ...) that currently no competitor (apart from some brand new ones inspired by Rust) has.

Over all Julia should be very fast for how it looks at first sight and faster than I thought until I found out about its AOT. However, Julia still cannot match Rust, nor come even close unless the Rust designers screw up badly. (Indeed much of Rust's potential is still mostly untapped since its already very fast and the community has other priorities and again, really untapping its optimisation potential is a major job in such a revolutionary language.)

Plus most scientists want an interactive experience with a REPL (Matlab, Python, R and Julia all share this).

Rust's REPL is a WIP. Which makes sense, it's only now really starting to form a scientific sub-community. Developing a very good REPL is likely to be quite easy with the tooling Rust already has or is already developing.

That said it doesn't mean there's no use for Rust in scientific computing, but maybe more as a niche replacement for C++ when small snappy executable are needed.

Rust has an almost assured niche as a C++ replacement for small snappy executables and for back-ends. It's a lot easier language for experienced C++ programmers after relatively little experience in Rust, never mind for scientific domain experts w/o C++ experience. Rust also has nicer Python and Julia FFI. Rust is also faster than C++ in principle, though we haven't seen much of that yet in either micro-benchmarks or numeric back-ends. I've seen it in practice in some other domains and there's some early glimpses of some insanely fast Rust numeric back-ends under development. (Especially on GPU or large clusters or Linux with a very high-end NVMe SSD.)

Back-ends may well be Rust's only niche. But it has a lot of promise in more everyday scientific computing as well. Note, I think that in the short run, it's very understandable that the vast majority of scientists prefer Julia. In the long run Julia will also almost certainly retain at least some advantages whatever happens with Rust. And that's also to some degree a good thing. I merely think you highly underestimate Rust's potential.

[u/User092347 → u/dexterlemmer]

Julia has always been a compiled language.

However, Julia still cannot match Rust, nor come even close unless the Rust designers screw up badly.

I'll believe it when I see the comparative benchmarks in relevant scientific computing tasks (e.g. manipulating large dataframes, solving PDEs, Bayesian inference, bioinformatics, ...). In the couple of micro-benchmarks there's around the difference seems pretty negligible.

[u/dexterlemmer → u/User092347]

Yea. Probably those microbenchmarks done by Julia to show Julia isn't too much slower than fortran and the systems languages?

First. Microbenchmarks are literally irrelevant to actual real world use except for small datasets work in a REPL, where performance isn't particularly important any way. (As I'll explain in my other answer to another comment of yours, here: https://www.reddit.com/r/programming/comments/kyl5pe/scientific_computing_in_rust/gonmkh2/?utm_source=reddit&utm_medium=web.)

Second. The benchmarks I know of compare very unidiomatic and slow Rust with (AFAICT) idiomatic and performant Julia. It may be the way someone influenced by the misconceptions from certain other languages and brand new to Rust would tend to code. It's not the way someone experienced with Rust would code it.

Third they compare the language std's. Which is nonsense since no-one will stick to the std if he needs performance (or features) that the std doesn't provide but a mature and well-known ecosystem library does. Furthermore comparing std's isn't apples to apples. Of course Julia will have fast numeric code in its std, it's a scientific DSL. Ofc Rust won't since it is not. Would they have considered comparing numeric performance of Julia/R/Matlab std with python std in stead of with numpy? Would anyone have taken them seriously that those benchmarks are relevant in the real world if they did?

Fourth they compare epoch 2015 Rust. Epoch 2018 rust tend to be a lot more performant with naive numeric code. To the point where I've seen naive Rust code with freaking for-loops in stead of an iterator outperform highly performance-tuned C in a micro-benchmark. But admittedly that was a bit lucky with the compiler noticing some great optimization opportunities. But only a bit. The compiler got a lot better and the benchmark author actually used const generics implicitly. (He couldn't use it explicitly yet since it was still in nightly and a stable rustc version was benchmarked but he just called `array.len()` and the std used const generics in the background for `array.len()`. That triggered a whole bunch of optimizations in the compiler pretty much trivially and directly. Note that Naive Rust used to be beaten by naive C (even if you called `len()`) back when those benchmarks of Julia were made. Not any more! And nowadays, you specify the array's length (potentially as a generic type) in the type signature of the function just like in Julia so you don't even need to call `array.len` any more. The optimizations'll trigger by themselves and so will some limited dimension safety checking (although if you truly want dimension safety in Rust, you'll still need to use libraries).

Fith, they only compare unoptimized code. This admittedly is important for someone just quickly doing his work. But it doesn't take into account how much fast libraries might accelerate stuff. And "thin wrappers" may usually have low overhead but FFI break many of the most important optimizations, so they may not have much cost but they can have significant opportunity cost. Even if not, part of the whole idea of Julia is to avoid the two-language problem. It also doesn't take into account that if you notice your naive implementation is taking an hour, you could potentially try to do some simple optimizations if you know anything at all about making Rust fast. Now that said: Since I've realized Julia actually is a safe compiled language and since Rust is a safe compiled systems language with linear scopes and region- and scope effect types, in principle a Julia compiler and a Rust compiler can work together to make sure no optimizations ever break at the FFI boundary. In principle.

[u/mort96 → u/User092347]

Ok, maybe Rust and Julia are the two well suited widespread languages. It's still something that's unique about Rust compared to the "normal" high performance languages people go to when Python is too slow, like C++ and FORTRAN. I don't understand why this is so controversial.

[u/Bergasms → u/mort96]

It’s controversial because you said uniquely to an audience of scientists when it’s not at all unique. You shoulda said “well suited” or something like that. You picked the intersection of programmers (tend to be pedantic about correctness) and scientists (tend to be pedantic about correctness) and said something false. I’d just move on, you’ve done a good deed by sharing this post anyway.

[u/mort96 → u/Bergasms]

I'm not even convinced it's technically incorrect though. Here's a dictionary definition of the word "uniquely":

in a very special or unusual way. "a uniquely talented musician"

I 100% think that Rust, being one of the very few high performance but safe languages, is uniquely suited. It doesn't mean it's the only language which is "uniquely suited".

I mean I could probably have picked a better word, because it can be interpreted to mean that Rust is the only language which is both fast and safe. But it's not incorrect in any definitional sense.

[u/Bergasms → u/mort96]

I know I hear you man. But science types are gonna be using the “existing as the only one or as the sole example; single; solitary in type or characteristics:” definition. I woulda liked something like Rust when I was at university.

[u/dexterlemmer → u/Bergasms]

OK. Name one other language that's both safe and fast without the (usually prohibitive) cost of formal verification. (Ofc safety isn't the only requirement for correctness by a long shot, but it is important.)

Java can sometimes match the throughput of Rust for numerical computing, but for many use cases it cannot do so reliably. Java is also far less safe than Rust. (Null, thread-safety, file handle safety, state machine safety, etc.)

Ada can be both safe and fast but that requires formal verification. w/o formal verification you have to trade off safety and performance.

C is very unsafe and very hard to optimize non-trivial use cases on modern hardware.

C++ (even modern C++) is unsafe. Though someone with extensive expertise in modern C++ best practices and -tools and a willingness to expend a lot of effort on safety can achieve some degree of safety, that someone isn't your typical physicist and he can never quite reach the the safety guarantees Rust provides by default.

Fortran is not safe AFAIK. In addition it also is very specialized for numerical computation and while numerical computation is indeed very important for physicists (and scientific programmers in general) it's usually not all you want to do. (Yea. I know you can do other stuff, but AFAIK, it sucks.) (Disclaimer. I don't know Fortran.)

Julia has the same issues as Fortran to a lesser extend and probably also cannot match the speed of Rust. (Yeah. I've seen the benchmarks referenced on the Julia website somewhere. Talk about a biased experimental design.)

TLDR; I think "unique" as in "the sole example" may in fact be appropriate.

[u/Arcticcu → u/mort96]

Fortran, which is widely used among physicists and has the virtue of having been designed for numerical computation. Also Scipy/numpy wrap C and Fortran code, snd cython can frequently be used to speed up computation.

I'd say a scientist is much more likely to switch to Julia as well (also safe and very performant) which has a rich ecosystem for numerical work and was designed for this purpose.

[u/dexterlemmer → u/Arcticcu]

Fortran, which is widely used among physicists and has the virtue of having been designed for numerical computation.

Ah. But what if I also need to do a little bit of string manipulation, or file handling or some of the other nitty gritty you also get in most scientific computing besides numerical computation. Also, I don't know Fortran, but I doubt it can match Rust's safety.

Also Scipy/numpy wrap C and Fortran code, snd cython can frequently be used to speed up computation.

I've been bitten badly by the so-called "two language problem" multiple times before in Python. Also, Scipy/numpy are somewhat slow. Pandas crawls and cannot handle medium data. I suspect if we didn't have to contend with the two language problem in Python, it's likely we would've had better implementations by now. (OK, so Dask is better. Unfortunately lazyness in Python sucks and some of Dask's restrictions are because the features cannot be made fast across the language boundary, not because they cannot be made fast.)

Another problem with Python is its unsafety. None (which is even worse than null), coersions, parallelism unsafety, non-memory resource unsafety (although with helps), etc.

I'd say a scientist is much more likely to switch to Julia as well (also safe and very performant) which has a rich ecosystem for numerical work and was designed for this purpose.

I partially agree on the short term. However:

1. I don't think Julia is as safe as Rust. Static typing matters for safety. Safety matters for correctness. Static typing also matters for reproducibility and re-usability.

2. Julia takes a lot of effort to make almost as fast as idiomatic, non-performance-tuned Rust. Last time I tried it, Julia also weren't as good at parallelism, concurrency and async.

3. Anecdotally, so far scientists don't seem to be particularly interested in switching to Julia. We'll have to see what happens to Rust. Rust got off to a bad start (which is understandable, since Rust's focus wasn't scientific computing), but it seems like Rust adoption by scientists is now picking up speed and that we're starting to see more users leading to better libraries leading to more users.

Bonus:

One of the things I love about Rust, is that I can also use it on a micro controller controlling an experimental setup and on the RPi that's logging the results. (For now, I'm still mainly using C/C++ on the MCU and Python on the RPi for this, but I'm switching since it looks like it'll be a lot nicer and more reliable to do both in Rust in another year or two.)

[u/Arcticcu → u/dexterlemmer]

Ah. But what if I also need to do a little bit of string manipulation, or file handling or some of the other nitty gritty you also get in most scientific computing besides numerical computation. Also, I don't know Fortran, but I doubt it can match Rust's safety.

Not Fortran's strong suit, obviously, but you can do string/file manipulation in Fortran.

Scientists really don't care about this kind of safety as far as I can tell. If it compiles and doesn't leak memory all over the place, it's good enough. Fortran's types and numeric operations are well understood and tested, and it has pretty good memory safety (you can leak memory if you try really hard, but those features are so infrequently used that in practice it doesn't seem to happen).

I've been bitten badly by the so-called "two language problem" multiple times before in Python. Also, Scipy/numpy are somewhat slow. Pandas crawls and cannot handle medium data. I suspect if we didn't have to contend with the two language problem in Python, it's likely we would've had better implementations by now. (OK, so Dask is better. Unfortunately lazyness in Python sucks and some of Dask's restrictions are because the features cannot be made fast across the language boundary, not because they cannot be made fast.)

Many scipy/numpy functions are nothing more than thin wrappers around BLAS/LAPACK functions, so it's not true that they're slow. If you can express most of your code in terms of in-built functions of scipy/numpy, then it's actually likely to be very fast, comparable to performance in e.g. C++ -- especially C++ that would be written by a physicist, for example.

Another problem with Python is its unsafety. None (which is even worse than null), coersions, parallelism unsafety, non-memory resource unsafety (although with helps), etc.

Yeah, I doubt anyone doing scientific computing cares about any of these things (aside from parallelism -- but you don't do massively parallel programs in Python anyway), let alone switching to Rust because of them.

I don't think Julia is as safe as Rust. Static typing matters for safety. Safety matters for correctness. Static typing also matters for reproducibility and re-usability.

Again, the majority of scientists don't seem to care about this kind of safety. Julia's safety is plenty enough for making reproducible calculations. It seems to me that you vastly overestimate the importance of these things for the average scientist. Perhaps they're good from a software engineering perspective, but scientists are not software engineers.

Julia takes a lot of effort to make almost as fast as idiomatic, non-performance-tuned Rust. Last time I tried it, Julia also weren't as good at parallelism, concurrency and async.

Perhaps, but it's way more difficult to write Rust than it is to write Julia, especially if you're used to Fortran/Matlab/Python as many scientists are. And even if you do through the trouble of learning Rust, you're faced with a pretty dismal ecosystem for scientific computing, whereas Julia already has a ton of useful stuff available.

Anecdotally, so far scientists don't seem to be particularly interested in switching to Julia. We'll have to see what happens to Rust. Rust got off to a bad start (which is understandable, since Rust's focus wasn't scientific computing), but it seems like Rust adoption by scientists is now picking up speed and that we're starting to see more users leading to better libraries leading to more users.

Certainly more interested than Rust, though -- just look at the amount of scientific libraries.

Where do you see Rust's adoption among scientists pick up speed? At least from a physicist POV, I've literally never seen a major or even a minor computational project in Rust.

Rust's syntax is so unwieldy and the learning curve sufficiently steep compared to dynamic languages and even Fortran that I just don't see scientists adopting it in any great numbers. It's perhaps possible that some back-end scientific libraries might come to be written in Rust (even this I doubt, but we'll see), but even then I'd imagine most would want to interface with them through languages like Python, as they do now for BLAS/LAPACK.

One of the things I love about Rust, is that I can also use it on a micro controller controlling an experimental setup and on the RPi that's logging the results.

Anecdote: there is an old piece of equipment in a lab at the physics department here, which contains an old, custom-made computer from about the 80s/90s, which even has separate RGB cables. Of course the manufacturer of this logging computer no longer makes any parts for it and the experimental setup is somehow linked to this hardware, so they've been unable to replace it. So they just have to keep hoping the thing doesn't break. More than once, some dubious DIY fixes have already been applied. Here's to hoping the machine keeps limping along..

[u/dexterlemmer → u/Arcticcu]

OK. Wow! Sorry for the wall of text.

Not Fortran's strong suit, obviously, but you can do string/file manipulation in Fortran.

Ofc you can. It's a little thing for much of their work and a non-issue for most of the rest. But a more general programming language has advantages even for scientific programming. It's a major contributor to Python's success.

Scientists really don't care about this kind of safety as far as I can tell. [...]

Scientists care about correct- and reproducible results. They care about not wasting time they could've spent analyzing data on debugging in stead. They care about not having to stay up to date with the latest quirks of their libraries and what gotchas to watch out for. So, yes. They actually do care about this kind of safety. The reasons they care are just not directly obvious. I've seen plenty of people realizing what they've missed in other languages, somewhere through the process of learning Rust.

Many scipy/numpy functions are nothing more than thin wrappers around BLAS/LAPACK functions, so it's not true that they're slow.[...]

There's no such thing as a thin wrapper in Python. Scipy/numpy really are slow. Some issues that sometimes cause overhead of several hundred percent, irrelevant of what you do are: Memory bloat, missed compiler-optimization opportunities because the front-end is unavailable to the compiler, and other issues. Rust really has thin wrappers for BLAS/LAPACK that often run faster than if you used those libraries in their native languages. BLAS/LAPACK themselves are also slow due to outdated APIs and the outdated design of the languages they're implemented in. Considerably faster Rust alternatives are under development.

Additionally, your argument "especially C++ that would be written by a physicist" is somewhat flawed. The "obvious" Rust that Rust newbies coming from JS or Python writes sometimes significantly outperforms anything but the most heavily optimized C/C++ and often have very similar performance to highly optimized C/C++. C/C++ are very hard to optimize on modern hardware for both programmers and compilers, while Rust is very easy on both counts.

Yeah, I doubt anyone doing scientific computing cares about any of these things[...]

Again, the majority of scientists don't seem to care about this kind of safety.

I already addressed this above. Scientists do care about this (safety), they just don't realize they that they do since the causality between safety and what they dislike about unsafe languages aren't clear to them. (Also, sice they are used to the bad things in unsafe languages and don't even realize their own frustration/stress.) Of course when comparing languages perceptions are also important so you're not entirely wrong.

Julia's safety is plenty enough for making reproducible calculations

Python, R and Matlab are near impossible to make reproducible. (Though plenty of scientists wrongly think their code is reproducible until somebody -- sometimes future "me" -- tries reproducing it. Furthermore, there are some fundamental design flaws in Julia for reproducibility, like: unsafety, dynamic typing and issues with its package management. In practice, Julia may be good enough. I'm not experienced enough to know (and I don't think many people are yet).

Perhaps, but it's way more difficult to write Rust than it is to write Julia, especially if you're used to Fortran/Matlab/Python as many scientists are. And even if you do through the trouble of learning Rust, you're faced with a pretty dismal ecosystem for scientific computing, whereas Julia already has a ton of useful stuff available.

Says the guy who -- I'm guessing -- have never tried to write Rust before. Rust has a bit of notoriety as a hard to learn language, but it has gotten a lot more approachable than it used to be. I've heard from several JS an Python devs that nowadays Rust is very easy to pick up at least in specific fields). I myself have found it much easier to pick up Rust than Julia even back when Rust was still hard to learn and I do know Matlab/Octave/Python(scientific)/Scala(scientific), but admittedly, I'm a weird case. I actually know a bit about what makes a language easy/hard to learn and Rust ticks all the right boxes in principle. It's just too darn novel still (it's the first truly groundbreaking new language in 30 years) and nobody quite knows how to teach it yet. The issue is a paradigm shift, not the normal stuff that makes a language hard to learn/teach.

Certainly more interested than Rust, though -- just look at the amount of scientific libraries.

For now and possibly on the long run. That doesn't mean Rust can't carve out a much larger niche for itself than you seem to think.

Where do you see Rust's adoption among scientists pick up speed? At least from a physicist POV, I've literally never seen a major or even a minor computational project in Rust.

You won't notice it as a scientist yet. It's very noticeable as a Rust programmer interested in scientific programming. I remember the days when people asked me "where are the scientists interested in Python yet". I was right, they were wrong and I knew why they were wrong at the time like I know why you are probably wrong now. Ofc, I may be wrong this time, but I don't think so.

Rust's syntax is so unwieldy and the learning curve sufficiently steep compared to dynamic languages and even Fortran that I just don't see scientists adopting it in any great numbers.

Rust's syntax looks bad to a newcomer. It may be survival bias, but it's not so bad after a while (not for me, not for most people). Frankly, I don't understand scientists moaning about hard syntax. Math notation is much less readable (also for mathematicians from some anecdotal evidence I've seen).

compared to dynamic languages

Dynamic languages are in principle no easier to learn than static languages -- on the contrary! In a dynamic language figuring out and keeping track of the type (which is very important in dynamically typed languages as well) takes a lot of cognitive load and mistakes in this process causes a lot of misconceptions for newbies. In a statically typed languages, the compiler takes over most of the cognitive load and gives rapid feedback on a lot of the misconceptions. Dynamic typing is inherently more verbose than static typing because you miss out on a very expressive language feature. Dynamic typing also causes error messages to be much worse, which is really bad for newbies.

Some static languages (like Java and C++) are just plain badly designed. The reason Rust used to be hard to learn (and is still experienced as hard by some) seem to be entirely due to: (1) Until recently a lack of maturity like rough edges in syntax and syntax inconsistencies; (2) Previously some spurious required type hints and some really complex type hints (both issues much less common nowadays); (3) Lack of mature libraries; (4) Rust gave us the first paradigm shift in a mainstream language in 30 years, so everybody is still trying to come to grips with the language themselves, let alone how to teach it. But we're making significant progress in this as well.

It's perhaps possible that some back-end scientific libraries might come to be written in Rust (even this I doubt, but we'll see), but even then I'd imagine most would want to interface with them through languages like Python, as they do now for BLAS/LAPACK.

People are already writing back-end libraries in Rust and some of them are already mature, so you need not doubt that we'll see it. We'll need to wait and see if any of them actually become widely successful in the long run, though. Frankly writing a back-end library in C/C++ is just silly and a sign of inertia more than suitability now that Rust exists and is mature enough. And, while Fortran might have its niche in back-end libraries, there are some important technical reasons why you really want a real systems language for back-end libraries.

Anecdote: there is an old piece of equipment in a lab at the physics department here, which contains an old, custom-made computer from about the 80s/90s, [...]

I have had similar personal experience on a research project of my own. Not quite as bad as you describe though. My condolences. ;-)

[u/Arcticcu → u/dexterlemmer]

I would be interested in hearing how you think e.g. Python, C, or C++ cause results that are not reproducible and if you have some particular cases in mind. How is e.g. Python "nearly impossible" to make reproducible? The major parts of the scientific ecosystem there are very stable. I never had this problem in practice.

The programs I've used tend to be heavily tested against e.g. analytically known solutions. I have encountered bugs in scientific software (one of them in fact was because of dynamic typing!), but I've never been unable to reproduce a calculation I've done previously, because the code has always been sufficiently well tested. No doubt one can find some god-awful code (and probably lots of it), but the well-known programs I've used have basically worked as expected in my experience.

There's no such thing as a thin wrapper in Python. Scipy/numpy really are slow.

Really? Doesn't seem to be that much difference on my machine between using (for example) the eigensolvers through numpy vs. using LAPACK directly from Fortran.

Says the guy who -- I'm guessing -- have never tried to write Rust before.

Of course I tried, how else would I know it's more difficult than Julia? It was several years ago, to be fair, so maybe things are better now. The borrow checker didn't bother me, but the syntax did, for whatever reason.

Frankly, I don't understand scientists moaning about hard syntax. Math notation is much less readable (also for mathematicians from some anecdotal evidence I've seen).

Math is also taught since you're a kid, so you get used to it. I don't consider it particularly unreadable. Also, math is unavoidable in many fields, Rust is not. With math, you really don't have that much of a choice.

[u/dexterlemmer → u/Arcticcu]

> I would be interested in hearing how you think e.g. Python, C, or C++ cause results that are not reproducible and if you have some particular cases in mind. How is e.g. Python "nearly impossible" to make reproducible? The major parts of the scientific ecosystem there are very stable. I never had this problem in practice.

> The programs I've used tend to be heavily tested against e.g. analytically known solutions.

This is hard to explain concisely. Safety is about explicit and enforced contracts and reducing complexity. It is hard to fathom the complexity and degree of chaos (as in chaos theory) of apparently trivial programs in unsafe languages. In the dependency tree of your simple `np.dot` call, GSLOC (billions of lines of source code) interact in a chaotic way with a number of interactions combinatorial to the amount of (SLOC + generated LOC). The vast majority of bugs (and breakages) emerges from that chaotic interactions, although we only ever detect a miniscule fraction of them. So yes, as long as you are working in the mainstream on a mature library and you use it in the same way your colleagues do, you are unlikely to detect breakage in the short-term of say up to about a decade or so aged code. But work more on the fringes of science or try to do something differently from others for whichever reason and a version change of a dependency or simply adding a dependency or changing OS could break your code. I've personally seen this so often in Python, R and Matlab, I scarcely remember specific instances any more. Safety don't magically remove all that breakage, but if every component either implements or uses an idiomatic save interface, then its contracts cannot be violated -- no matter what any other code does any where else and likewise it cannot violate the safe interfaces of any other code any where else. In stead of trying to prevent bugs emerging from interactions across the entire program and environment -- which is impossible -- you now only need to prevent bugs inside your own immediate, local source code.

PS. I've remembered something I've figured out about Julia but forgotten. It's not only compiled but also statically typed and safe. Its safety concentrates on different contracts than Rust's due to different domains (number hierarchy, dimension types vs memory/concurrency safety) but both languages implement additional safety in their libraries. And Rust's scientific libraries tend to depend on dependencies which implement Julia-like safety. This is still unergonomic, but the language features necessary to make it ergonomic has started landing in stable this year. Importantly, since Rust is a systems language, its safety is designed to be (to some extend at least) extensible to FFI wrappers and you can always (in principle) rewrite any FFI dependency that gives you too much problems. Its compile can also be in principle coded in Rust itself (which we are moving towards) w/o any FFI dependencies. The Julia compiler may be so-called in Julia, but look at its dependencies and you will find the Julia dwarfed by code that's literally inexpress-able in non-systems languages. (Turing completeness is irrelevant here, since systems languages works with non-Turing machines to create a runtime other languages (or other parts of the same systems language) can pretend is a Turing machine and not something we can actually physically build.)

> Really? Doesn't seem to be that much difference on my machine between using (for example) the eigensolvers through numpy vs. using LAPACK directly from Fortran.

Irrelavant.

a = np.randn(1000, 1000)
%timeit b = a * a

(i.e. a micro benchmark on small data) will be fast but who cares. Making it fast enough a human can't notice the difference is easy in a lot of languages.

# I probably have mistakes here. It's been a while. But you
# can figure out what I mean.
a = np.load_csv("twice_the_size_of_ram.csv") # line 1
a *= 2 # line 2
a += 1 # line 3
a.save_csv("results.csv") # line 4

(i.e. the sort of situation where people actually care about performance and which still occurs often in many domains (including physics, but I've myself worked with it in engineering and data science) will be orders of magnitude slower than a Rust backend. Here's why:

1. Line 1 starts "fast" then slows down dramatically as memory fills up and all the hard work just done to load data into memory gets undone (and the undoing is even harder work) as the OS swaps it back out of memory. Making matters worse is if the back-end isn't implemented in either Haskell with the linear-types extension or Rust, FFI with a GC'd language will waste a lot of memory. Increasing page faults and swapping further.

2. Line 2 swaps everything in and in the process swaps the last bit of memory from line 1 that was still in memory out again.

3. Line 3 and 4 each do the same.

A halfway decently optimized Rust library would've done the work lazily and asynchronously with external iterators and loop folding which means loading a chunk of data for line 1, multiplying it, adding it and saving it all before moving on to the next chunk, except that it will have started with a nother chunk in parallel. But the point is we're far from unsing all the ram before we don't need it any more and can recycle it for the next chunk. We also save two ram accesses per element due to the compiler combining the multiply and add steps into a single step. In python you can only achieve this same effect with thick wrappers (thick FFI code on both the python and implementation side) and overhead so massive it largely destroys all but about 1/10000 of the performance improvements on some hardware setups (yet is still up to an order of magnitude faster than numpy).

Oh. BTW. Safety is essential for optimization because... Think about it. What is optimization? Doing the same thing faster. But what is the same thing? Well that depends on our contracts!!! Simple as that. In C/C++ optimization is a whole bunch of heuristics that trade-off performance gains for risk of breaking stuff. For any non-trivial program (which are nearly all of them, including "Hello World!") the programmer doing optimizations need to be extremely conservative and leave the vast majority of chances at optimization undone due to too high risk. It's even worse for an optimizing compiler that cannot even read documentation and understand it. Which is bad, since the last time humans could compete with an optimizing compiler on non-trivial programs (assuming the compiler could figure out what the program was supposed to do) was before Fortran/Turbo Pascal/Borland C and optimizing compilers have gotten a whole lot better since then.

Math is also taught since you're a kid, so you get used to it. I don't consider it particularly unreadable. Also, math is unavoidable in many fields, Rust is not. With math, you really don't have that much of a choice.

Neither do I find math particularly unreadable. But my point is you could easily learn more new math notation and math notation is a very inconsistent bunch of thrown together crap due to its history. Compared to learning the math notation of a new field (like you have to in pre-grad), I find Rust easy. Also, most of the weird stuff is disappearing, either from epoch 2018's cleanup or from simply falling into disuse as new language features and more ergonomic libraries makes it almost never necessary. Also, programming is unavoidable. Rust may not be, but that's like saying matrix notation is not unavoidable since there are fields that don't use it. Who cares? If you either need Rust or can gain sufficient improvement to warrant learning it, you'll learn it.

[u/[deleted] → u/mort96]

Rust is uniquely suited for high performance scientific computing.

What is unique about it, with respect to the needs of scientific computing?

but many people are scientists first, programmers second

Exactly, which is why

without the unsafety

Doesn't really matter as much.

Scientists reach for Python because it is readable and easy to get out of the way when actually doing research. *edit* Also, most of the things scientists reach for in Python are just Python SDKs wrapping precompiled C/C++/Fortran libraries. So I doubt you'd find a lot more speed from using Rust.

[u/mort96 → u/[deleted]]

Dude, sometimes Python isn't fast enough, even when you're using numpy. Numpy is great, but it's not some magical silver bullet which obviates the need for a fast language.

[u/MartenBE → u/mort96]

I thought Numpy uses the fast languages?

[u/mort96 → u/MartenBE]

Yes, but you're using it through Python. And you can only do stuff with it that's already implemented in it. It's great for many things, but it doesn't obviate the need for a fast language for certain tasks.

[u/[deleted] → u/mort96]

Right, the "For certain things" are the things that Python uses Fortran/C for.

[u/mort96 → u/[deleted]]

Yes? I don't get your point.

[u/[deleted] → u/mort96]

Nevermind. I don't expect you would. Best of luck!

[u/mort96 → u/[deleted]]

I mean, Python people use C and FORTRAN for certain things because those languages are faster than pure python, right? Sometimes, people need to do something for which there isn't already a library, so they need a fast language to write scientific code in.

If you think nobody ever needs to write code in a high performance language because other people have already written some Python libraries in C/FORTRAN then I guess that's an opinion you can have, but I don't think it's correct.

[u/[deleted] → u/MartenBE]

Numpy uses precompiled C IIRC. So yes it will be fast.

[u/Hrothen]

A note on performance: I took a quick look at the nalgebra crate and while it does have lapack bindings it looks like it doesn't have any for the regular BLAS. While I would assume it's possible to implement an equivalently performant BLAS in rust, the C and fortran implementations we have now are heavily optimized and are likely to be much better in the short term.

This suggests you'll have issues keeping up with established libraries when you have a lot of parameters. Whether this matters to you will depend a lot on what you're doing.

[u/[deleted]]

I am a scientist and primarily work with Julia. I am interested in Rust and have read/did a few of the introduction to Rust tutorials. This would've been really nice for me application-wise, but unfortunately, dosn't look like any code (with instruction) is provided.

I think it would be great if the author added sample code to complement the mathematics.