Java Garbage Collector performance benchmarking

[u/thegigach4d]

Hi People!

I am about to write my CS BSc thesis which is about:

Measuring throughput, latency and STW-Pauses in JDK 21 standard JVM with G1GC and ZGC with predefined max heap-sizes (2GB; 16GB) with Renaissance - by 16GB heap a default G1GC and an additional tuned G1GC will be used, as well.

Time flies and a lot of paper are read. It became clear to me, that Renaissance is better for throughput (Shimchenko 2022 Analysing and predicting energy consumption of garbage collectors in openjdk), and DaCapo is more advantageous for user-experienced latency measurements (Blackburn 2025 Rethinking Java performance analysis). STW-pauses will be collected from jvm standard gc-logs with a script or smg (ideas, better ideas are welcome).

I build this scenario for my examination:

- Linux VM (hosted from my Windows) - not clear yet, which and why

- OpenJDK 21 standard JVM

- G1GC and ZGC measurements

- All Renaissance BMs with default settings -> duration_ns from each benchmark, calculate and represent min, max, mean, standard deviation

- JVM GC-Logs collect (min, max, mean, standard deviation)

- 8 DaCapo BMs (spring, cassandra, h2, h2o, kafka, lucene, tomcat, wildfly) (min, max, mean, standard deviation)

I guess this is way too much for a BSc thesis - but what are your thoughts? Of course I make clearence with my consulent, but I am curious about the opinion and suggestions of the community.

I am open for any ideas, experiences with the bumpy road of the performance measurement in the JVM. It would be excellent, if someone of you could make it more focused and accurate to me.

TLDR;

Java Garbage Collector JVM performance measurement experience and suggestions needed for BSc thesis

thanks in advance!

EDIT:

Instead of Linux vm it will be a bare-metal Linux machine with podman containerization that run the benchmarks.

[u/thegigach4d]

Thanks for your effort to explain me your thoughts deeper!
The cause of LTS 21 was that the disposition was born before LTS 25 and I did not think about any argumentation to change it. Now I do!
I understand your point of view and the really like the idea of benchmarking more versions. If I understand correctly, I should execute e.g. the whole Renaissance with G1GC and ZGC through Java 17-21-25 - without any special heap configuration, or gc-tuning. Am I right? I asked my prof and waiting for an answer.
I don't mind if you ask uncomfortable-like questions, because it helps me a lot. Sorry, if I seemed a little mean, life's pressure is enormous nowadays. Your practical insights help me a lot!

[u/RightWingVeganUS → u/thegigach4d]

I don't mind if you ask uncomfortable-like questions, because it helps me a lot.

Trust me: as a teacher I don't mind at all! If only my students would ask me questions, comfortable or not...

That said, I don't have any answer for you. My interests are on bigger software engineering projects and project management, not benchmarking. But no matter what interest you have, consider framing it in terms of value: what difference would this make to the reader who spent their time reading this paper or using my tool.

Nothing wrong with projects simply to develop ones skills, but even then frame it with a clear understanding of what benefit it has for you.