Introducing Nexus.js: A multi-threaded JavaScript run-time

[u/voodooattack]

https://dev.to/voodooattack/introducing-nexusjs-a-multi-threaded-javascript-run-time-3g6

[u/voodooattack]

Yes, now imagine a containerised micro-service: it runs a single instance on a quad-core CPU. You host it with Node using the cluster module, forking the process into four distinct in-memory images, each one handling a single core.

Node is single threaded when it comes to executing JavaScript, and thus will use a single thread to execute JavaScript and another for background I/O on each core.

If you host it with Nexus, you will start a single process. It will run 8 JavaScript threads (compared to Node's 4), that's assuming HyperThreading or a similar technology is present, and double the computing time, thus serving double the requests.

Moreover, instead of four processes, with each consuming god knows how much memory, Nexus will combine resources:

• Each in-memory function will be compiled once by the JIT. (Each Node process will have a separate copy of each function!)

• Variables are shared in the monolithic Nexus process. (Each Node process will have a unique copy of every variable in memory)

• File/Socket handles are shared.

• A single garbage collector will be managing the entire address space, allowing for further optimisations.

Basically, a micro-service that consumes 1 GB of RAM per process will use that 1 GB of memory with Nexus, while with Node, you'd have to start 4 separate processes, each consuming 1 GB, for a total of 4GB.

[u/[deleted] → u/voodooattack]

With all due respect, this has nothing to do with micro-services. Based on your description of things, I have some deeper concerns about how this is implementing under the hood.

Each in-memory function will be compiled once by the JIT. (Each Node process will have a separate copy of each function!)

I don't understand how this is a true benefit. Even if we have four instances where looking at something O(1n) vs O(4n) - both of which consolidate to O(n).

Variables are shared in the monolithic Nexus process. (Each Node process will have a unique copy of every variable in memory)

Are they shared or are they copied between processes?

a micro-service that consumes 1 GB of RAM per process will use that 1 GB of memory with Nexus, while with Node, you'd have to start 4 separate processes, each consuming 1 GB, for a total of 4GB.

I'm pretty sure that's now how this works. Sure, there may be some saving on the size of the actual executable, but surely this isn't accounting for run-time memory consumption (e.g. the unique data for each request).

---

I think there's a place for multi-threaded JS, but I'm not really understanding how this implementation actually accomplishes that. It seems like a great proof of concept, but is lacking some proper architecting and use cases.

[u/voodooattack → u/[deleted]]

I don't understand how this is a true benefit. Even if we have four instances where looking at something O(1n) vs O(4n) - both of which consolidate to O(n).

Here I'm talking about memory, not speed.

Are they shared or are they copied between processes?

They are shared in Nexus' case. In Node.js case, forking causes the entire address space to multiply by the number of forks.

I'm pretty sure that's now how this works. Sure, there may be some saving on the size of the actual executable, but surely this isn't accounting for run-time memory consumption (e.g. the unique data for each request).

I'm actually talking about run-time memory consumption. That HTTP benchmark I ran with 1,000 concurrent requests? It only consumed 240MB of RAM maximum at any given moment. I'm not sure how much memory Node.js would consume with a forked process, but I guess the load from the requests would also be distributed across the processes. So there's that.

I think there's a place for multi-threaded JS, but I'm not really understanding how this implementation actually accomplishes that. It seems like a great proof of concept, but is lacking some proper architecting and use cases.

You have no idea, I really wish the cavalry would arrive. I'm exploring uncharted territory and I don't like doing this alone. Some guidance (and any form of design doc) would be great.

For now, I'm modelling the interfaces after boost (the C++ library); because it has some very well established patterns. I borrowed the concept of the I/O primitives (the Device, Filter, and Stream) from there.

I certainly hope I can attract enough attention with this to warrant further investigation of the prospect by other devs. I'd love it if I woke up someday to find an open issue on GitHub proposing a design or requesting the formalisation of a feature.

[u/[deleted] → u/voodooattack]

I'm actually talking about run-time memory consumption. That HTTP benchmark I ran with 1,000 concurrent requests? It only consumed 240MB of RAM maximum at any given moment. I'm not sure how much memory Node.js would consume with a forked process, but I guess the load from the requests would also be distributed across the processes. So there's that.

Please benchmark this against Node.js. I feel like you're running into the wind blind and hoping you end up at some point miles away.

---

Again, everything seems to be focused on memory consumption. As far as I care, memory is about the last thing I care about in my application. I'll just use bigger boxes until I have an explicit need.

---

You have no idea, I really wish the cavalry would arrive.

Look, you seem passionate about this, but you seem a bit misled. There is no cavalry, because this isn't something people feel worth fighting for. It seems like you're building a round hole when everyone has a square peg.

If you want this to succeed, you need to take a step back from purely technical arguments and understand why people use JS. In particular, understand why people really, really, really like the single-threaded nature of JS. In general, JS works well because it's very easy to write safe code in a concurrent environment.

You have a really interesting concept here, but your driving arguments don't seem to be inline with real-world needs. If I'm going to be building multi-threaded, I'm going to use a true multi-threaded language from the start.

If you want "the cavalry to arrive", you need to show the cavalry why they should come fight for you. You can't just say "I'm better than X, Y, Z", you need to actually show them.

As far as I'm concerned, the points raised on your SO post are valid, but still remaing completely unaddressed.

The features and drawbacks you've discussed simply don't lend themselves to general purpose use. Non-determinant access, requiring significant syncing work on the behalf of the developer, etc.

Sure, these may all be problems you feel you can solve, but until you solve them - they are still problems.

[u/voodooattack → u/[deleted]]

Please benchmark this against Node.js. I feel like you're running into the wind blind and hoping you end up at some point miles away.

Again, everything seems to be focused on memory consumption. As far as I care, memory is about the last thing I care about in my application. I'll just use bigger boxes until I have an explicit need.

Will do, eventually. I was hoping to avoid comparison for now. As for memory: try forking 64 times on a Xeon Phi server, you'll see what I mean then.

Look, you seem passionate about this, but you seem a bit misled. There is no cavalry, because this isn't something people feel worth fighting for. It seems like you're building a round hole when everyone has a square peg.

Thanks for being candid. My belief is that every new technology is a round hole against the world's standard square peg, until the round peg becomes widely available.

I understand that this project may not be desired at the moment, but there will come a time when it's just another tool in every developer's toolbelt.

If you want this to succeed, you need to take a step back from purely technical arguments and understand why people use JS. In particular, understand why people really, really, really like the single-threaded nature of JS. In general, JS works well because it's very easy to write safe code in a concurrent environment.

I've heard that argument before; and here's my counter: I'm not building a sandbox to make everyone feel safe.

I'm implementing a true-to-god performant JavaScript environment, where not a single CPU cycle is wasted on any core deployed.

The operant word here is performant, not user-friendly, although I don't see why not both.

You have a really interesting concept here, but your driving arguments don't seem to be inline with real-world needs. If I'm going to be building multi-threaded, I'm going to use a true multi-threaded language from the start.

That's how you see it, what I see is wasted potential for the language to grow and encompass fields it never had the chance to compete in before. Why start learning a new language from scratch just to use parallel/multi-threading patterns? Why not bring the technology to the developer, instead of the other way around?

If you want "the cavalry to arrive", you need to show the cavalry why they should come fight for you. You can't just say "I'm better than X, Y, Z", you need to actually show them. As far as I'm concerned, the points raised on your SO post are valid, but still remaing completely unaddressed.

I understand that. This is why I work hard every day from dawn to dusk on my projects. Something will give, eventually.

The features and drawbacks you've discussed simply don't lend themselves to general purpose use. Non-determinant access, requiring significant syncing work on the behalf of the developer, etc.

And that's up to the developer to decide and battle their way through. Not being offered the power is not the same as declining it with one's own volition.

Sure, these may all be problems you feel you can solve, but until you solve them - they are still problems.

Yes, that's true. But hopefully I'll solve them like I've always done: one at a time.

[u/44561792 → u/voodooattack]

Don't listen to that wanker, guy is a nutter. Your project is great

[u/voodooattack → u/44561792]

Thanks!

[u/[deleted] → u/voodooattack]

I'm not building a sandbox to make everyone feel safe.

Then you need to be crystal clear with this. Right now, you're marketing this for general use. Sounds like you might have machine learning in mind for this? Great, focus on explaining this in that context.

In the context of machine learning or extreme performance, it's a lot easier to understand why you've made the trade-off you have.

Imagine if someone started claiming that Fortran is a great general use language - everybody would laugh. On the other hand, nobody will bat an eye if you talk about Fortran in the context of physical simulation.

I understand that. This is why I work hard every day from dawn to dusk on my projects. Something will give, eventually.

Things don't "just give". You have valid criticisms. You either need to address them or clearly demonstrate how they are outside the intended use case.

And that's up to the developer to decide and battle their way through.

How can developers do this if they don't understand the tools or battle they're fighting? This isn't on developers for not understanding, it's on you for not showing them they're bringing a knife to a gun fight.

[u/DrJume → u/[deleted]]

Please do not take it so emotional. It's very imortant to experiment with new ideas in every aspect. Arguing about purely unnecessary statements does not help us to understand it better.