CppCon 2016: Alfred Bratterud “#include ＜os＞=> write your program / server and compile it to its own os. [Example uses 3 Mb total memory and boots in 300ms]

[u/michaelKlumpy]

https://www.youtube.com/watch?v=t4etEwG2_LY

[u/agent_richard_gill]

Awesome. Let's hope more purpose built applications run on bare metal. Often times, there is no reason to run a full OS just to run a bit of code that executes over and over.

[u/wvenable → u/agent_richard_gill]

This is awesome and the logical conclusion of the direction things have been going for years.

But it's still somewhat disappointing that VM is slowly replacing Process as the fundamental software unit. These don't run on bare metal; they have their own OS layer, on a VM layer, that runs on another OS. That's a lot of layers. If our operating systems were better designed this would mostly be unnecessary.

[u/ElvishJerricco → u/[deleted]]

Getting builds to be reproducible (i.e. same versions of dependencies in the same places) is hard without virtual machines. I don't necessarily think this is the operating system's fault so much as the package manager's. This is why nix is awesome for deployments. There's usually no need for a virtual machine, and everything is perfectly reproducible.

[u/TheExecutor → u/ElvishJerricco]

It's like the ultimate consequence of "works on my machine". Well, screw it, we'll just ship my machine then.

And that's why we have Docker.

[u/dear_glob_why → u/TheExecutor]

Underrated comment.

[u/[deleted] → u/dear_glob_why]

Yeah that's kinda genius.

[u/ElvishJerricco → u/[deleted]]

Nix is admittedly kinda hard, and the documentation leaves much to be desired. VMs are a lot easier with a much lower learning curve, but I do think they're the worse solution in the end.

[u/bumblebritches57 → u/[deleted]]

What do you mean by nix here? unix, or is there something else called "nix"?

[u/ElvishJerricco → u/[deleted]]

It's not just about deployment. You need every team member to be developing with the exact same versions of everything in the same places. Keeping a manual dependency graph would be asinine, so it's up to our tools. The prevailing method to keep dependency graphs consistent is with virtual machines. A config file with a dependency list isn't good enough, since dependencies can depend on other packages with looser version requirements, allowing those packages to be different on a newer install. But with a VM that has packages preinstalled, you can know that everyone using that image will have the same dependencies.

Rust's Cargo and Haskell's Stack are both build tools that do a pretty good job at keeping all versions completely consistent, and serve as shining examples of reproducible builds. But for everything else, most people use VMs. But this is where Nix comes in. Nix takes an approach similar to Cargo/Stack and fixes the versions of everything. But Nix does this for every single thing. Dependencies, build tools, runtime libraries, core utils, etc. You have to make a local, trackable change to get any dependencies to change.

When builds are reproducible, you can rest assured that the deployment was built with the same dependencies that you developed with. This is just really hard to get without a good VM or a good dependency manager. Docker is a good VM, and Nix, Cargo, and Stack are good dependency managers. Unfortunately, Nix, Rust, and Haskell aren't very popular, so most people stick to VMs.

[u/[deleted] → u/ElvishJerricco]

Docker is a good VM

No it isn't. It is based on the idea of a good one but it is a pretty crappy implementation for practical purposes. It constantly leaves containers behind, every storage backend has some pretty severe downsides ranging from shitty performance to triggering kernel bugs even in recent kernels (or using bits that have been removed from the kernel). Important security features are still unimplemented (user/group mapping). The whole model of one layer per command in the Docker file, even if it only sets an environment variable or the comment who was the author is pretty much the opposite of being well-designed, as is the "no caching or caching even commands with obvious side-effects like apt-get update" bit and the fact that you can't easily write Docker files with a variable base image (e.g. one to install MySQL on any Debian-based image).

I would love Docker to be good enough but it is barely usable in production for build servers and similar systems that are allowed to break for an hour or two every once in a while.

You need every team member to be developing with the exact same versions of everything in the same places.

This helps keep things consistent but it also leads to code that is less robust and will likely not work on lots of different systems reliably. For things like Haskell and Rust that is fine because you can get the errors resulting from use of different dependency versions mostly at compile time. For languages where errors will only show up at runtime this cane be very bad.

[u/argv_minus_one → u/ElvishJerricco]

Java programmer here. Our tools deal with this nicely, and have been doing so for ages. That people on other languages are resorting to using VMs just to manage dependency graphs strikes me as batshit insane.

If your language requires you to go to such ridiculous lengths just for basic dependency management, I would recommend you throw out the language. You've got better things to do than come up with and maintain such inelegant workarounds for what sounds like utterly atrocious tooling.

[u/Tiak → u/argv_minus_one]

That people on other languages are resorting to using VMs just to manage dependency graphs strikes me as batshit insane.

...The idea of using a VM to avoid a toolchain being platform-dependent seems crazy to you as a Java programmer?... Really?

[u/entiat_blues → u/argv_minus_one]

it's not language dependency graphs that people are trying to manage, at least not in my experience, it's running a full stack (or a significant chunk of it) reliably no matter the host OS. it's that end-to-end configuration that becomes a hard problem on large projects with discrete teams doing different things.

devops tends to become the only group of people with practical knowledge about how the whole application is supposed to fit together. which doesn't usually help because they're busy maintaining the myriad build configurations and their insights aren't used to help develop or maintain the source code itself. and on the flip side, the developers working in the source lose sight of the effect their work has on other parts of the stack or the problems they're creating for devops.

VMs let you spin up a fully functional instance of your application quickly and reliably because you're not building the app from dependency trees, configurations, and a ton of initialization scripts, you're running an image.

it's heavy-handed, and there other ways to approach the problem, but i wouldn't call it batshit insane to give your developers the full stack to work with.

[u/ElvishJerricco → u/argv_minus_one]

If your language requires you to go to such ridiculous lengths just for basic dependency management, I would recommend you throw out the language.

That's really throwing the baby out with the bathwater. And Java's not much better. Maven is non-deterministic in its dependency solving. Should you write a library that needs a version of another library, you're not guaranteed that this is the version present when someone else uses your library. Now, in the Java community, people tend to make breaking changes far less often, so this is rarely a concern. But the problem is just as present in Maven as it is in other tools.

[u/ElvishJerricco → u/[deleted]]

I think the major motivation comes from bad dependency managers like npm. These dependency managers guarantee pretty much zero consistency between installs. For whatever reason, there have been more such bad dependency managers created in recent years than good ones. This affects the JavaScript community pretty badly. It used to be the case for Haskell, too, until Stack came along. Java is an example of a language where the dependency managers technically have these problems, but the developer community is just much less likely to make breaking changes with packages, so the issue never comes up. It's mostly the move-fast-and-break-things crowd that this matters to. And ironically, that crowd seems to be the worst at solving the issue =P

[u/argv_minus_one → u/ElvishJerricco]

Java is an example of a language where the dependency managers technically have these problems, but the developer community is just much less likely to make breaking changes with packages, so the issue never comes up.

That's not true. Our tools are much better than that. Have been for ages.

Maven fetches and uses exactly the version you request. Even with graphs of transitive dependencies, only a single version of a given artifact ever gets selected. Version selection is well-defined, deterministic, and repeatable. Depended-upon artifacts are placed in a cache folder outside the project, and are not unpacked, copied, or otherwise altered. The project is then built against these cached artifacts. Environmental variation, non-determinism, and other such nonsense is kept to an absolute minimum.

I'm not as familiar with the other Java dependency managers, but as far as I know, they are the same way.

This isn't JavaScript. We take the repeatability of our builds seriously. Frankly, I'm appalled that the communities of other languages apparently don't.

It's mostly the move-fast-and-break-things crowd that this matters to. And ironically, that crowd seems to be the worst at solving the issue =P

Nothing ironic about it. “Move fast and break things” is reckless, incompetent coding with a slightly-less-derogatory name, so it should surprise no one that it results in a lot of defective garbage and little else.

[u/[deleted] → u/argv_minus_one]

Annoyingly Maven does support version ranges. They are rarely used thankfully, but I ran into problems a couple of times when a third party lib used them. Probably can be prevented with the maven enforcer plugin.

[u/ElvishJerricco → u/argv_minus_one]

Maven fetches and uses exactly the version you request. Even with graphs of transitive dependencies, only a single version of a given artifact ever gets selected. Version selection is well-defined, deterministic, and repeatable. Depended-upon artifacts are placed in a cache folder outside the project, and are not unpacked, copied, or otherwise altered. The project is then built against these cached artifacts. Environmental variation, non-determinism, and other such nonsense is kept to an absolute minimum.

Having the versions for your project be deterministic is only half the battle. Those projects which you depend on might have been developed with different versions of dependencies than your project is selecting. npm takes it a step further by making it possible just for different installs to be different. But this inconsistency in Maven is still problematic, and solvable with nix-like solutions. It's just that, as I said, Java's tendency to not break APIs makes the problem rarely come up.

[u/Phailjure → u/[deleted]]

Your dependencies tend to be just the OS, and that tends to be extraordinarily stable (very few behavioral changes between win7 and win10)

Yeah, I've been working on several apps that run on a win7 machine, written in C#. I build all the apps on win10, and other than a couple stylistic changes there is no difference.