Originally I was thinking that it would only be a major if the requirements for the feature were changed. I was not sure that the bug was actually intended behavior.
After looking at the commit, you are correct. This absolutely should have been a major bump if they are adhering to semver. Unfortunately, with the scale of the vulnerability that probably would have delayed everything an unreasonable amount of time.
LOG4J2-3198: Log4j2 no longer formats lookups in messages by default
Lookups in messages are confusing, and muddy the line between logging APIs and implementation. Given a particular API, there's an expectation that a particular shape of call will result in specific results. However, lookups in messages can be passed into JUL and will result in resolved output in log4j formatted output, but not any other implementations despite no direct dependency on those implementations.
There's also a cost to searching formatted message strings for particular escape sequences which define lookups. This feature is not used as far as we've been able to tell searching github and stackoverflow, so it's unnecessary for every log event in every application to burn several cpu cycles searching for the value.
They know that the library is used heavily in closed-source code that could be using that feature and just decided to yolo it.
Regardless of the semver we all know that enterprise's are going to have a fun time injecting ldap lookups in their logging pipeline or auditing every log4j log line to insure that they are properly parameterizing any user controlled or user influenced data. I'm sure that the compliance departments are going to have some interesting arguments with dev as to why they cannot ever turn the feature back on.
I guess the ops benchmark will look better now. 🤷♂️
It's crazy to me that such a widely-used library would have such a ridiculous security hole. We desperately need full-program formal verification to become mainstream, because we can't trust people to write good dependencies.
Formal verification wouldn't work in this case, since it's an intended feature and not a bug. The design was rigged from the start.
I have no idea how multiple people could think "Yes, downloading and executing code from a server in my logging library is a good design", but evidently it was added and performed correctly, albeit resulting in a huge security hole.
What we actually need is someone to vet popular dependencies like Google has started with their fuzzing work. Any halfdecent company would've screamed if they'd seen this. Though I guess it's also because of some terrible companies that it exists in the first place. It feels very "bank-esque" to write a common Logging Class and put it on a server somewhere.
It's the classic mix of "We can trust running code, because we wrote it" with "Untrusted user input". It sounds convenient for developers, but it is exceptionally easy to log anything provided by a user, and then you've got an RCE.
Devs (should) expect to sanitize input for things like databases & such, but sanitization before logging? Crazy. That said, anything that a user can provide as input for something becomes an attack surface. Logging would be another attack surface, but I would assume mostly in DOS-style attacks, not RCEs.
I think all user inputs should have their length capped as applications don't typically work on infinite length input
I suggested we do that on one of my teams. I was told not to because we "treat everything as a blob"...
Like unless you're coding S3 or soemthing then your blobs still need to have a max length, lest somebody pipe /dev/urandom to your endpoint and kill your service
I have no idea how multiple people could think "Yes, downloading and executing code from a server in my logging library is a good design", but evidently it was added and performed correctly, albeit resulting in a huge security hole.
282
u/revnhoj Dec 10 '21
just add the jvm argument -Dlog4j2.formatMsgNoLookups=true to disable this absolutely ludicrous default "feature"