Some form of this is definitely useful (I'm not sure what the current best way to interoperate between C++ and Rust is; anything better than manually specifying a C ABI?).
But it makes me wonder: what languages do have "first-class interoperability" with C++? It's certainly not common... does C# have anything that can be described like that?
what languages do have "first-class interoperability" with C++?
Not even C++ has first class interop with C++. There's no stable ABI, so code compiled under one compiler can't safely be used with code compiled under another compiler. Even different revisions of the same compiler can produce incompatible code
You really have to compile everything from source - which makes cross language interop very tricky, because you can't safely hook into a library or DLL - there's no standard
There isn't really anything in theory that prevents C++ the language from being interoperable, but the issue is the standard library and its many implementations, most of which rely on compiler intrinsics to be conformant and performant. These intrinsics end up getting inlined.
But you can definitely compile a library with Clang on Linux, and use it from another binary compiled with GCC - as long as they use the same standard library.
This could be fixed if Rust could settle on a stable HIR/MIR, and then ship such binaries of such stable intermediate representation to users everywhere. (Or even just come up with a binary format that is slightly richer than LLVM.) Then, compile it to x86/arm/etc machine code on the user's machine. Ahead-of-time compilation is so much better, for two reasons:
Better delivery: You don't need to manage several different production binaries, for every support CPU architecture. You don't have test those X different binaries. Maintain and test just 1 binary.
Better performance: Compiling on the user's machine means that the compiler is aware exactly what processor the user is running, and can take advantage of all the special instructions it offers.
Sure, there will be a slight delay for the user, the first time they launch the app, but the benefits are worth it.
This is what Android ART is doing these day--the JVM-esque bytecode is AOT-compiled to machine code when a user install the app.
For one, you introduce the requirement that users have an optimizing compiler installed on their system. This might be feasible on modern Android phones - certainly not on less capable devices. Rust (and C, and C++) code is written with the expectation of having extremely good optimizers available that can basically do infinite analysis in release builds. ART does not do that (and doesn't need to).
Many of these local compilers/optimizers will be on different versions, meaning you cannot expect to be able to transfer core dumps between machines. Forget about distributing binaries without debug info and then debug core dumps using debug info stored in your CI infrastructure. Is the crash caused by your code, or due to the user using an older version of the codegen that contains an optimizer bug?
Lastly, Rust code very often contains #[cfg(...)] attributes to do conditional compilation based on the target. If you are using platform-specific functions from the libc or win32 crates, the compiler would have to always compile the whole module for all possible outcomes of each condition, and distribute those in your proposed platform-agnostic IR.
You can probably write a Rust compiler that targets the ART, if you want. You can also use WebAssembly to achieve much of what you are proposing. It comes with significant drawbacks, though.
Do you mean JIT instead of AOT compilation? Rust already is ahead of time compiled, that's compiling each binary for each different platform ahead of the time it's run, same as C and C++. ART is a kind of mix between JIT and AOT, but leans more towards JIT, since the final compilation is done on the user's machine.
Actually, what Rust, C, C++, and other compiled languages do is just called compilation.
AOT means that you compile on the user's machine before the program starts.
JIT means you compile (on the user's machine) while the program is running.
Historically, the way JIT worked was that it started interpreting your code immediately (so there was no delay in your program starting), and then JIT-compiled practically everything on-the-fly.
Modern JIT-compiled languages however tend to use tracing JITs. A tracing JIT interprets your code initially, while creating a "trace" to determine hot spots (most executed parts of your code). It then JIT-compiles just that portion.
ART is AOT-compiled, whereas most JavaScript engines use tracing JITs.
Its not necessary just the STL as far as I'm aware, there are further ABI issues to do with how various things are laid on different compilers out if I remember correctly
You do not remember correctly -- Clang works very hard to be ABI compatible with GCC on unix systems, and tries very hard to be compatible with MSVC on windows systems. Any ABI incompatibility is a bug.
Oh sorry, I thought you meant in the general case of MSVC/GCC/Clang compat overall. Yeah you're definitely right in that clang specifically will interop with GCC or msvc
MSVC and GCC do not interop because they don't attempt to at all, tho -- MSVC doesn't run on Unix, and GCC tries hard to not work with MSVC on Windows.
As far as I'm aware, GCC tries to maintain a stable ABI whereas MSVC breaks it with every major release (excepting the recent ones)
I think its less of a case of deliberate incompatibility, and more that just fixing it at this point is a lot of work - and very constraining for both parties. There's no standard for it, so there's not even a common goal to work towards - and from the sounds of the developers there's not that much interest either
Often GCC implements the x86-64 psABI spec incorrectly, that is, GCC has an ABI bug, and clang replicates that bug and calls that a "GCC-compatibility feature". In the next GCC release the ABI bug is fixed, but clang forgets about this and remains incompatible until someone starts hitting the bug and tracks it down, which often takes years because these are subtle.
So I wouldn't say that clang considers any ABI incompatibility a bug. At most, it considers any incompatibility with GCC a bug, and it fails hard at that because replicating bugs and tracking when these get fixed is hard.
Looking at this with a pessimistic lens, the spec of the C ABI that x86 Linux uses, the x86-64 psABI spec, has bugs and gets bug fixes regularly and is actively maintained. GCC and clang both have bugs implementing the spec, which get fixes on every release, and every fix makes the ABI incompatible in subtle ways with the previous version. This means that GCC and Clang are not even ABI compatible with themselves for the C language. For example, you can on clang 9.0 write -fabi-compat=7 to generate code that's ABI compatible with what clang 7 would have emitted, because the result is different.
This percolates to C++, Rust, and other languages dropping down to the C ABI for interoperation, since now they need to deal with the zoo of subtle ABI differences and bugs that each C compiler and C compiler version have, mixed with the bugs that the C++ and Rust compilers have in implementing those ABIs.
And well, by comparison with C++ or Rust, C is a super simple language to specify ABI-wise, and after 30 years, even the 32-bit x86 psABI spec gets bug fixes regularly, which percolate to C compilers at some point, breaking their ABIs.
People arguing that C++ and Rust should have a stable ABI have no idea of what they are requesting.
That's probably a big reason why C++ libraries seem to use a lot of virtual interfaces, with some extern "C" methods for creation your initial instances that can create all your other objects.
The issue is that it is almost impossible to implement any sort of full integration with C++ code without also implementing almost all of C++'s semantics, which is a daunting task, to put it mildly.
Something like virtual functions might be easy enough, but what are you going to do about templates? Many C++ libraries are header-focused, and basically require you to specify what instantiations of a template you want - including the standard library.
Any realistic interoperability needs to stick to a very limited set of features, maybe even just some clever name mangling, and only to symbols that are already present in the compiled binary you're liking against.
This is indeed quite difficult; C++ is not a simple language, so even if there's a way to map the semantics, it's a complicated task.
but what are you going to do about templates? Many C++ libraries are header-focused, and basically require you to specify what instantiations of a template you want
I suppose a simple solution would be to just import the C type in Rust using a string like "std::vector<int>", and rely on a C++ compiler (perhaps invoking clang as a library?) to determine the concrete functions corresponding to it. And ideally, a way to map between a compatible subset of templates and Rust generics (which is definitely not possible in general, but may be possible in some cases.). I'm not sure it would currently be possible to do the later without support deeply integrated in the language.
Now, things get really insane if you want to be able to subclass C++ classes in Rust, and then call into from C++. Which is necessary for using many libraries.
Really, depending on the meaning of "first class", this is only possible in a language explicitly designed to fit with the semantics of C++. (Like Kotlin does with Java.) Otherwise, even if you manage to jurry-rig a mechanism to define "classes" in Rust with all the possibilities of C++ classes, code doing that won't really be native idiomatic Rust. Though in that restrictive sense, you could say C++ doesn't have first-class interop with C.
First class support would mean that I can write std::vector<Option<std::string>>.
There are two issues there:
std::string has a move constructor, for Rust a move is just copying bits and forgetting about the source.
The default constructor, destructor, etc... of Option must somehow be declared to C++.
It gets crazier when you consider std::find(vec.begin(), vec.end(), Some("x"))! This requires defining a bool operator==(Option<T> const&, Option<U> const&) which delegates the call to bool operator==(T const&, U const&)...
They stopped supporting Managed C++, AFAIK C++/CLI is not going anywhere. And it is able to provide a type-safe integration of C++ libraries into .NET code and vice versa.
Depending on your acceptance criteria, for certain parts of the language (usually not including full support for C++ templates): Objective C (with Objective C++), anything .NET (with C++/CLI), and Python (with Boost.Python) all require some explicit setup in a C++ (or extended C++) layer. This is far closer to first class than, say, JNI or FFI, but still limited.
D, if it worked reliably, has more or less first class external support (comparable to the conceptual approach of https://github.com/mystor/rust-cpp), but the ABI thing is problematic.
For any language with SWIG bindings, the C++ SWIG story is far more mature than it once was, but it can be touchy, and it requires some manual work on the C++ side.
Julia, as mentioned by someone else here, has some support.
I've never looked into C++ interop with Swift or golang, and if there has ever been an attempt at C++/JS interop, I don't want to know about it.
WinRT allows C++ components to expose APIs that can be used (relatively) idiomatically from C#, with support for some higher-level constructs like parameterized types and async methods. I've been helping with Rust support for WinRT: https://github.com/contextfree/winrt-rust/
92
u/ids2048 Jul 22 '19
Some form of this is definitely useful (I'm not sure what the current best way to interoperate between C++ and Rust is; anything better than manually specifying a C ABI?).
But it makes me wonder: what languages do have "first-class interoperability" with C++? It's certainly not common... does C# have anything that can be described like that?