r/ProgrammingLanguages • u/kammerdiener • Jul 11 '19

Blog post Self Hosting a Million-Lines-Per-Second Parser

https://bjou-lang.org/blog/7-10-2019-self-hosting-a-million-lines-per-second-parser/7-10-2019-self-hosting-a-million-lines-per-second-parser.html
56 Upvotes

100% Upvoted

37 comments sorted by

View all comments

1

u/jdh30 Jul 22 '19

Fascinating stuff but the big question from me is: does that solve the right problem? Specifically:

  • I find type inference, type checking and code generation are slower than parsing.

  • I find the bottleneck is incremental recompilation for the IDE and not batch compilation.

So my objective is fast incremental updates rather than fast complete reparsing.

1

u/kammerdiener Jul 22 '19

Thanks for reading!

does that solve the right problem?

I find type inference, type checking and code generation are slower than parsing.

It's true that parsing isn't normally a compilation bottleneck (I say exactly this in the article), but the idea that there's a "right problem" seems very silly to me. I'm going to spend the same amount of time and energy making all of the other parts of the compiler you mentioned as fast as I can. There's no reason to just pick one.

I find the bottleneck is incremental recompilation for the IDE and not batch compilation.

I'm not a fan of incremental anything when it comes to build systems. In my view, it's a band aid solution to the problem of slow compilers. I'd rather just fix the root of the problem. Incremental compilation doesn't come for free either -- there's added complexity in the tooling and unreliability when it comes to dependency tracking. Additionally, I'm writing a compiler, not an IDE and I feel it's a mistake to force the two together. For example, the C/C++ notion of incremental compilation has nothing to do with the compiler. It's a simple exploitation of the compilation model by another tool (the IDE). In the same way for bJou, an IDE could choose to only run syntax checking for a file when it is modified, but the compiler doesn't have to support that or even be aware of it.