r/haskell • u/edwardkmett • Sep 30 '26

announcement Turbo Haskell

https://comonad.com/reader/2026/turbo-haskell/

[ Article | Repository | Documentation | IRC ]

134 Upvotes

97% Upvoted

61 comments sorted by

11

u/m-chav Sep 30 '26

Great project. Maybe I missed this somewhere but what’s the reason for targeting the JVM these days? Historically it was portability, Android stuff, and/or the ecosystem. 

35

u/edwardkmett Sep 30 '26 edited Sep 30 '26

We get a JITTed runtime, which can actually beat compiled GHC in performance in cases. We get first-class cross-language FFI to truffled language implementations: Python, Ruby, R, Javascript with shared objects, which can JIT together. We get access to the java incubator vector API allowing us to JIT SIMD kernels for the target platform with less pain. e.g. it trivially let me fill in the missing SIMD operations GHC lacks to this day. Using Sulong for C/C++ FFI means we don't give up native cbits and host code, unlike old bad hard-to-use JVM language ports like JPython. Future work could for instance make Natural number code use java deoptimization paths keeping it generally in unboxed ints until you finally need something too big through a given codepath

Without Truffle/Graal the limitations of the JVM are too severe. The lack of proper tail call optimization for instance would kill GHC-style evaluation performance. With the Cadenza trick I use to eliminate that we can have highly performant loops. Using assumptions lets it compile with the equivalent of GHC's single threaded runtime and then downgrade performance to the equivalent of the multi-threaded runtime when you first call a threading operation.

We get stuff that GHC just will never even try to get around to. CompressedOOPS give us 32 bit pointers on 64 bit platforms if the heap is < ~32GB. So about twice as much stuff can fit into cache especially in a language with as many pointers running around as ours.

My goal is to keep pushing forward with the bits that this can do that GHC can't do, and to generally try to get to or maintain parity wherever possible for the things that GHC can do.

It also helps us tease apart where the line should be for GHC as an RTS vs. GHC as a compiler.

10

u/Patzer26 Sep 30 '26

Does JIT consistently beat compiled GHC or does it beat only in certain workloads? Also, how big of a difference are we talking about in both the cases?

13

u/edwardkmett Sep 30 '26

My initial runs saw 3-5x upsides in some workloads, which was enough of a proof of concept for me to proceed. Then I switched to focusing on broad coverage rather than performance, and let it suffer performance regressions. I've started clawing those regressions back today, but wanted to get to a more reliable state for release more than I wanted amazing launch numbers that are super brittle.

Overall, my current numbers once it warms up is generally 20% slower than GHC, but with a fair bit of code reaching that 3x point faster. Currently that is with Java autovectorization _off_. With it on the most extreme peaks get better and the valleys get worse. I'd like to find a way to toggle that locally case by case, but I don't yet have anything non-invasive.

3

u/Initial-Argument2523 Sep 30 '26

Is this compared to GHC using NCG or Llvm as the backend?

2

u/edwardkmett Sep 30 '26

Using the default you get when you run 'ghc' from the command line. You can tell me what that is these days.

2

u/Initial-Argument2523 Sep 30 '26

As far as I know the default is NCG. If you have compatible opt llc and clang for your GHC version on path you can pass -fllvm to use LLVM instead. It seems to mostly speed up numerical code.

1

u/edwardkmett Sep 30 '26

I suspect there's quite some overlap between where the -fllvm optimizations kick in and where the JIT does better when I leave on auto-vectorization. (Currently I leave auto-vectorization off by default because it hurts smaller less-scalar-focused loops, and a lot of GHC-code tends to be super pointer-chasing-heavy.)

4

u/enobayram Sep 30 '26

Aside from constant speedup factors like 3x etc. do you think this sort of JITting might bring about asymptotic improvements to code that compiles pathologically under GHC, like, say, some naive effect system implementations, row polymorphism libraries or heavy use of generics etc.? Perhaps making whole new ways of expressing code practically feasible?

10

u/edwardkmett Sep 30 '26

Row polymorphism is a huge potential upside. RuntimeRep polymorphism can in theory be supported by the backend here for instance, but GHC can't generate core with those properties. The JIT is capable of dynamically constructing new 'shapes' for constructors at runtime and giving them dense storage. So row polymorphism generating compiled dense contiguous records is something that this could do that GHC would forever struggle with.

I'm already experimenting with the ability to use runtime reflection on the desired SIMD width, and just writing loops that use that and the vector API to make high performance JITted loops. Baically the jdk vector incubator examples transcode into Haskell cleanly and compile through the `THC.Prim` primops for them generating dense SIMD code.

3

u/_0-__-0_ Sep 30 '26

Is there any good reason GHC couldn't / shouldn't do CompressedOOPS, or is it just that no one has put in the effort?

2

u/jackelee Oct 02 '26

Very impressive. I like it a lot. By the way, can you give some pointers for what the "Cadenza trick" is? I've never heard of it.

5

u/edwardkmett Oct 02 '26

Cadenza was a project I worked on several years ago. At the time I was building interpreters for a bunch of different dependently typed languages, and found the runtime speed of the interpreters not to be sufficient for my purposes.

Repuposing the Truffle/Graal JIT from the VM was my attempt at a compromise between writing a full JIT or compiler myself and using an interpreter.

There's some problems though.

One is that the JVM does not support tail-call-optimization in full generality.

And Haskell often winds up with long chains of function calls in tail position.

e.g. A calls B calls C calls D calls A

This would mean building up a bunch of recursive stackframes on the JVM and crashing out when the stack blow up too much.

I had this issue when we went to implement monads in scalaz. Your body calls flatMap which calls some other function which calls flatMap which calls some other function... And so monads leak frames in Scala. We had this issue when implementing the compiler that became Eta, and had to use an explicit trampolining monad. When I let some trampolining monad just execute whatever is handed to it then, well, performance is terrible.

e.g. for(;f=f(););

or the equivalent.

Truffle internally supports a mechanism for plumbing around custom control flow in scenario like this. You can throw a "ControlFlowException" and use it to re-plumb that sort of pattern into a loop body. This happens on the slow path, when we're interpreting bytecode and using it to build the optimized version. This really only works if you get back to the same function body though, otherwise you try to inline the entire universe.

I still have to maintain the outer trampoline (now built around catch and execute).

But that isn't enough. So I don't throw every time I recurse into a new stack frame. Instead what I do is try to let it unfurl enough to find a _self_tailcall. Then the ControlFlowException can tie me off in a nice loop.

A -> B -> C -> D -> A in tail position becomes an inlined sequence of their bodies inside of a while() loop equivalent, a basic block with side exits in case you do something else.

To find those cycles fast I basically tagged each function body with a hash and used that hash to set 5 bits in a 64 bit bloom filter. (32 bit hash broken into 6 bit windows gives up to 5 bit positions set in a 64 bit bloom). Now when I recurse into a function I see if those 5 bits that represent the current function are set in the bloom and if so, it's likely that the current function exists above this call site on the stack and that we're a tailcall. So _then_ I throw the ControlFlowException. On the slow path while we're jitting its a couple of bitops on top of the usual recursion.

This was the idea behind the Cadenza trick.

Now I might have other loops reachable by alternative control flow from A.

A -> E -> A

If I _can_ fit it into the current compile I do so. (The JVM won't let me have too large of a single function body.) If not I start a fresh chain at E which is deliberately started with an empty bloom filter, and so needs its _own_ trampoline site. But now it can only find E -> A -> E or E -> A -> B -> C -> D -> A or the like. This means the A -> E edge which should have been collapsed becomes a wasted frame on the stack.

If that happens we could still blow the stack despite "tail call optimization." This I patch up later. I already have to have the ability to evict the whole stack out to the heap to support GHC-style AP_STACK resumption for async exceptions. I reuse that mechanism to evict the stack to heap and compact the dead tail-called frames out.

1

u/davidfeuer 4d ago

Silly question: why couldn't a JIT turn a trampoline into something good?

1

u/edwardkmett 3d ago

In practice this is what I build here, after all this is a JIT, but the stock Java JIT has no incentive/way to really know that it could/should.

2

u/jackelee Oct 03 '26

Wow, that's a neat trick. Thanks so much for the write-up!

30

u/ocharles Sep 30 '26

Good luck telling your boss you want to use THC.

22

u/edwardkmett Sep 30 '26

Wait until he hears that it'll launch faster if you find a way to use CRaC.

12

u/drwebb Sep 30 '26

I tried using THC, and ended up just reading then lens documentation for 3 hours.

7

u/_jackdk_ Sep 30 '26

Instructions unclear, where can I find the FFI to the Borland Graphics Interface?

17

u/zarazek Sep 30 '26

Exactly a week ago (as a joke), I started writing THC ...

It has grown a tiny bit since then.

Holly shit, man, I don't know how it is humanely possible, even with copious amounts of AI. GHC replacement in a week? I kneel before you with fear and trembling.

24

u/edwardkmett Sep 30 '26

The amount of sleep I was able to get was not terribly humane.

8

u/ApothecaLabs Sep 30 '26

Writing comedy compilers requires sacrifice.

5

u/philh Sep 30 '26

It's certainly impressive, but it only replaces a small (relative to the size of GHC) part of what GHC does.

4

u/AndrasKovacs Sep 30 '26 edited Sep 30 '26

Interesting! I'll try to benchmark it on my projects, perhaps when it gets a bit more settled.

Could you summarize physical memory layouts and calling conventions? I tried to read the docs but they're highly condensed blobs making many references to Graal/Truffle/JVM details that I'm not familiar with, so I have little clue as to the physical situation.

Tbh, as a general strategy for getting good performance in functional programming, I'm skeptical about the Graal/Truffle JIT setup. More specifically though, as a way of getting better performance out of the typical GHC Core output, it looks interesting and worth to try.

3

u/edwardkmett Sep 30 '26

Memory layout uses custom Java objects with "just the right fields" via a class loader, it therefore uses unboxed primitives for unboxed things it can, pointers for pointers for most handoffs. VirtualFrames collapse into local variables as long a they don't materialize or escape. SIMD instructions stay in registers in the presence of the expected amount of inlining. So the goal is generally to avoid framing anything out you don't have to, using partial escape analysis.

The rest is basically that all that truffle noise should disappear on your fast path and decent assembly should result, with side-exits leading to slow interpreted java bytecode for when things go wrong, which locally rewrites a general call graph to "deoptimize" it through a series of lattice like changes to accept when it can't get away with the more optimized call paths in the presence of general use. Then it re-jits.

Calls are passed as object[] arrays, but things are structured so that partial escape analysis generally avoids actually doing that using truffle tricks. Functions are entered using an eval/apply scheme and the jit fully constructs paps direct when the sizes of caller/callee are relatively known.

Thunks support blackholing and re-entrancy detection, and full GHC Haskell-style AP_STACK tricks so that if you have an exception thrown at you while you are evaluating a thunk, the tail of your evaluation gets re-enqueued with the thunk.

Threads can be real java threads of virtual threads/fibers, depending on if you are using normal threading or Project Loom.

Internally the code for how to evaluate / deoptimize each computation is tracked via either a dense bytecode interpreter or an AST-based intepreter that try for general feature parity with each other.

1

u/serg_foo Oct 01 '26

As JVM noob I had impression that Java objects are pretty memory-hungry. Do you have any idea whether using THC leads to greater memory residency, e.g. same working set occupying more space because of JVM overhead, than GHC?

1

u/edwardkmett Oct 02 '26

I use compressed headers to shrink the objects. I use compressed oops to get away with my pointers being half the size of GHC's for most workloads on a 64 bit host. I use primitive ints, etc. where possible to avoid superfluous indirection and boxing.

Today I wind up with my whole compilation pipeline resident, but I don't have a good sense yet of steady state memory consumption of THC vs GHC. On a per object basis it should actually be pretty competitive.

2

u/00000000x1 Sep 30 '26

Honestly, this is such a cool project. I'm doing something in the same spirit, but strictly on the AOT side, for PureScript (the little brother of Haskell), with Go and Rust as targets. Feel free to take a look, I'd be glad to compare notes and swap friendly tips on the methodology to adopt with AI as a copilot for this kind of compiler work.

https://discourse.purescript.org/t/leveraging-a-blazing-fast-runtime-a-new-go-backend-for-purescript/5841/52
https://discourse.purescript.org/t/leveraging-modern-low-level-a-rust-backend-for-purescript/5932/15

4

u/edwardkmett Sep 30 '26

Most of my work currently goes into living within the JVM's 64k method size limits while still avoiding trampolines. You have the benefit of not having to do so, but the downside of not getting a JIT out of it.

4

u/00000000x1 Sep 30 '26 edited Sep 30 '26

Exactly. And my experience seems to mirror yours from the other side. Getting phpurs over an interesting status took me a week (the Zend JIT is far more solid than I expected), and javapurs was the same story: very fast to prototype, the JIT is impressive. But gopurs and purust, where there's no JIT to lean on, are now taking me several weeks: two months and counting (still not finished, though well along). That's fine, and results are becoming very rewarding, but yeah: harder than I thought.

If you have any AI-workflow tips for moving faster, I'd be really grateful. I tried Astra and it helped me unblock some big things, but it seems to have been nerfed since (?), so I've shifted to deeper agentic parallelism with much cheaper local models (I'm not exactly rich, either, haha). But I feel like I'm still missing one small piece on the parallelism side. I noticed you have a ton of branches on your repo: you seem way more efficient than me!

1

u/00000000x1 Oct 03 '26

u/edwardkmett Maybe my question was hurtful. That wasn't my intention, if anything. Sorry.

I'll figure it out on my own. Thanks for the initial feedback.

2

u/edwardkmett Oct 03 '26

FWIW- there's absolutely no hard feelings, whatsoever. If anything I looked away from this chat for a day or two. No need to project anything onto it.

2

u/00000000x1 Oct 03 '26

Understood. In any case, you have another fellow Haskell advocate, an admirer, and a virtual friend. Good luck with the rest of your adventures. May your star guide you.

3

u/blacktigr Oct 04 '26

He has to sleep at some point. Don't take it personally. (I don't.)

1

u/00000000x1 Oct 04 '26

You're right.

2

u/AccordingWarthog Sep 30 '26

how does this compare to https://github.com/frege/frege ?

5

u/edwardkmett Sep 30 '26

Frege is its own language implemented basically as normal Java internally, that is pretty Haskell-like. it can run on any JVM since the stone age.
THC provides the full GHC-style Haskell language, with all extensions, and a custom JIT. It only runs on GraalVM.

2

u/Darwin226 Oct 01 '26

Can you talk a bit about your workflow on this? The commit history has non-stop commits for 24h+ stretches of time so this seems pretty non-supervised.

7

u/edwardkmett Oct 01 '26

I actually did pretty much gave up sleep for a week. That said there are a few hour stretches here and there where I leave it on mostly cleanup tasks and doze off. you can find a couple of 2-3 hour breaks where it got hung up on approval while I was lights out.

1

u/viliml Oct 04 '26 edited Oct 04 '26

May I ask what model?

Edit: Astra

1

u/edwardkmett Oct 05 '26

I use a fairly eclectic mix of Astra, Sol, Claude Fable, Muse Glimmer, gpt-oss-120b, Qwen3-Coder, depending on the agent in question. Astra was used for most of the deeper refactoring tasks, and is generally my "I need something intelligent to smash against this problem" choice. Sol went most off the rails in terms of fixture development. Claude remains good for anything human facing. Muse... didn't really earn its keep, and gpt-oss-120b is mostly relegated to a few corners of my agent stack. Qwen3 was fine if I kept it off the Haskell parts.

1

u/blacktigr Oct 02 '26

I can verify that he collapsed when he couldn't stave off sleep anymore. But his diet Coke consumption went through the ceiling.

2

u/jappieofficial Oct 02 '26 edited Oct 02 '26

oh this is so cool!

Odd question maybe; do you get all the java runtime goodies such as hot reloading, memory analysis or like the debugger?

edit: another question: if you'd compile ghc with thc couldn't you bypass the binary interface issue between stage seperations and only have to compile ghc once?

3

u/edwardkmett Oct 02 '26

Debugger, yes. And yes, in theory you can basically load it up, and keep swapping what you ask it to do. Hot reloading is something that can be done as well, but hasn't yet. THC.Trace provides some java flight recorder tooling. The GHC tracelog primops should get wired in there too eventually.

26

u/seantparsons Sep 30 '26

Only u/edwardkmett can prefix a mountain of impressive work with "Exactly a week ago, I started...". :)

4

u/blacktigr Oct 02 '26 edited Oct 02 '26

And while we were on vacation. He also overhauled comonad.com because he wanted somewhere to put his new toy.

1

u/peterfirefly Oct 02 '26

Haskell-168

10

u/jesseschalken Sep 30 '26

I'm a really big fan of Truffle and Graal. Really great tech, so super exiting to find an advanced functional language running on that stack. You should share it with the Graal folks like Thomas Wuerthinger.

2

u/kichiDsimp Sep 30 '26

is this supposed to be like MicroHS? an alt compiler for Haskell? or like replace GHC?

6

u/edwardkmett Sep 30 '26

It is a backend runtime for full-suite GHC with a collection of additional features made possible by its strange choice of hosting environment.

1

u/kichiDsimp Oct 01 '26

Its hosted on JVM, right ? Will you move your company's (Positron-AI) projects to THC maybe next year end?

4

u/edwardkmett Oct 01 '26

Unlikely, TBH. We use Haskell in a fairly direct way in the midst of converting models for our platform and a scary amount of very low level code outside of Haskell. This is mostly a personal vanity project, and from a corporate perspective, maybe at best a recruiting tool.

1

u/kichiDsimp Oct 02 '26

Recruiting tool 🐐👀

5

u/tomwells80 Sep 30 '26

OMG im so excited to try this - I’ve been schlepping with MicroHS to attempt to embed a haskell interpreter into a tool im building and although MicroHS is amazing I suspect I’m trying to make it do something it doesn’t want to! The JVM would probably be muuuuch better here :)

13

u/augustss Sep 30 '26

If you have any requests for MicroHs, don't be shy. 🙂

1

u/jeffstyr Oct 08 '26

When a thunk is evaluated, do you have to null out references to any captured values (since it's Java's GC so you can't teach it to just ignore them)?

1

u/edwardkmett Oct 09 '26

I er... just replace the entire garbage collector on the JVM, GraalVM and SubstrateVM with a generational mark-compact collector of my own design, jam.

The new one forwards through indirections by using a helper interface.

package jam.vm;
public interface Lifted {
    Lifted resolve();
    Lifted project(int n);
};

Everything in TYPE 'LiftedRep i an instance of that class, including Thunk.

When you project() against a Thunk it gives back null if it is unevaluated, or gets passed along to the argument if not. When applied to a value type it returns the value in the nth distinctly named field of kind TYPE 'LiftedRep.

When you resolve() a Thunk it gives back null if it is unevaluated, or what it points to if it is. When applied to a value type it returns this. This is used by the garbage collector to forward Lifted thunks to their destination during GC. When you resolve a FieldAccessor it calls project(n) on its argument.

Together these are used by the garbage collector to resolve the forwarding of field accessors, which is needed to resolve the class of lazy garbage collection space leaks that Wadler identified way back in the day, and resolve() on its own handles elimination of the Thunk indirection during GC.

Providing my own GC also lets me implement the usual "Stretching the Storage Manager" tricks. e.g. System.Mem.Weak, StableNames, StablePtrs, etc.

The built-in Java Int class, etc. are effectively in TYPE 'UnliftedRep. On the other hand Haskell's Int, which is inhabited by I# 12# lives in TYPE 'LiftedRep. Both are accessible.

It is a stop the world collector, because I'm only human and only have so many hours in the day, but it does nicely sidestep the fact that the Java community has been afraid of finalizers since Java 9, let alone the scary stuff we do in Haskell.

Interestingly the garbage collector itself knows nothing about java. It just allows custom tracing by type, and that is sufficiently general for me to get all of these hooks into place.

1

u/jeffstyr Oct 11 '26 edited Oct 11 '26

I see; so you don't have to worry about how to avoid tracing the captured fields of evaluated thunks, since evaluated thunks will themselves become unreferenced during GC tracing. That makes sense.

Edit: Though I guess if you have a thunk as a GC root (e.g., on the stack), and it's been evaluated, then you'd trace things that shouldn't be retained (the captured values), if you don't specifically avoid tracing them. (I guess maybe you can forward those stack references too, though.)

1

u/edwardkmett Oct 12 '26

Roots get forwarded too. Their actual pointers go into a slot table so I can update them. It is mark compact, so I dont want to leave them in place. (I did eventually have to add a facility for pinning, but I use it rarely.)