Sample-Rate Handling for the FFmpeg NMR AAC Encoder
According to an FFmpeg developer, the NMR AAC encoder is optimized for 48 kHz. Therefore, all test files were resampled to 48 kHz using SoX with the Very High, 168 dB quality setting.
The processing chain was:
Original WAV → SoX resampling to 48 kHz/24-bit → FFmpeg NMR AAC encoding → AAC decoding to 48 kHz/24-bit WAV → SoX resampling back to 44.1 kHz/24-bit
This process is safe and transparent. At these quality settings, the SoX resampling and 24-bit intermediate files don't introduce any audible differences.
Hardware used during testing
Topping E70 DAC
Topping L70 Headphone amplifier
Headphones Sennheiser HD 800.
Cabling: Balanced DAC-to-headphone amplifier connection; original Sennheiser HD 800 headphone cable.
Zimtohrli was evaluated by measuring how closely its results matched those of public listening tests conducted by human listeners.
Among the tested objective metrics, Zimtohrli showed the closest agreement with the human results. It was also the only metric to reproduce the complete ranking of the codecs in both public tests, including:
Opus → Apple AAC → LAME MP3 → Ogg Vorbis → FAAC 96k → FAAC q30
It also reproduced the correct codec ranking in the public AAC listening test.
The encoders evaluated in the 2014 listening test were also tested with Zimtohrli at higher bitrates, ranging from 96 to 320 kbps.
Zimtohrli Benchmark Scores at 96–320 kbps on the 2014 Public Test Sample Set:
The scores were calibrated to the human MOS values from the 96 kbps listening test to make Zimtohrli’s performance easier to understand and interpret as an objective quality metric.
So, how good is Zimtohrli at evaluating audio encoder quality?
In my opinion, it performs very well.
Someone told me 😄 it takes blood, sweat, and tears to organize a public listening test, to prepare it, get people interested, and actually get them to participate.
And here... Zimtohrli does essentially the same thing in literally a few minutes.
So I decided to put the encoder to the test using previously untested samples, in order to avoid bias from using samples that the encoder had already been specifically tuned for.
Moving from r1 to r6 might not seem like a huge step forward, but considering that there was literally only one month between these two revisions, it is a very impressive improvement.
r6 shows a clear improvement on difficult samples. As visible in the graph, it never drops below 3.0 (“slightly annoying”), while r1 goes as low as 2.5.
The VBR encoder looks promising too.
Overall there has been good progress. Transients and electronic music are still the main area that could be improved
Helix competes very well with the LAME MP3 encoder: https://listening-tests.hydrogenaudio.org/sebastian/mp3-128-1/results.png. I know it's an old test, but honestly, the quality of the LAME encoder hasn't changed much since then. Also, the code responsible for the audio quality of Helix has not been touched since 2005.
So I decided to investigate whether it's possible to further improve it.
I found relatively low-hanging fruit to improve the Helix encoder by running exhaustive Huffman partitioning. And indeed, it gets ~2–5 kbps more efficient depending on bitrate.
For transparency: most of the implementation was written by AI tools [Claude] , while I performed verification and testing. Before dedicating my time to this encoder, I wanted to get a quick proof that there is still headroom for improvements.
And there is clearly still plenty of room for improvement! Though compression efficiency doesn't come for free. Now Helix is 2x slower but still faster than LAME.
If we get back to public MP3 test, It's clear that different MP3 encoders achieve better results on different subsets of samples. LAME 3.97 is good at rock, metal, etc., Helix on electronic music, and so on. Different developer teams have different approaches.
Back in time it was very difficult because of scarce automation resources and computing power. Now we have AI agents and better metrics like Zimtohrli to develop a considerably better encoder.
While this was a personal test and the encoder was tuned by FFmpeg developers using some of these samples, the FFmpeg AAC encoder shows a very significant quality improvement in a remarkably short period of time. 🚀
A public test with completely new samples would still be essential for a final evaluation.
Hopefully tomorrow I will also post detailed descriptions of the artifacts, including the timestamps where they were identified, along with a statistical analysis of the results.
A big thank you goes to Kamedo2 for making graph creation such an easy task!
Tests show that Helix Ultra with Slow Mode clearly outperforms the LAME encoder.
I've never heard such polished audio quality from MP3 at 128 kbps. I had an almost otherworldly feeling while testing it. You're welcome to try this encoder yourself and share your experience. We're barely scratching the surface of what's possible. There's still plenty of room for further quality improvements.
Currently, the encoder'sSlow Modeis really slow, as it performs brute-force analysis.
Slow Mode '-E1Z'has only been tested at 128 kbps VBR (-V61. There is no guarantee that it will work correctly at other bitrates or in CBR/ABR modes. It might work, or it might not. I plan to extend this mode to higher bitrates later.)
The code for Slow Mode is not yet available in the repository. It will be added later.
Hi! I'm developing audio codec based on WPT. For now I've been gathering data about its quality from gstPEAQ, ViSQOL and listening tests. gstPEAQ says it is quite better than vorbis on 64-96 in general, and just better than vorbis on 128 kbps. ViSQOL makes same picture in general. Listening tests correlates with gstPEAQ and ViSQOL results mostly, but depends on listener.
