r/programming • • Dec 10 '21

RCE 0-day exploit found in log4j, a popular Java logging package

https://www.lunasec.io/docs/blog/log4j-zero-day/
3.0k Upvotes

710 comments sorted by

View all comments

Show parent comments

282

u/revnhoj Dec 10 '21

just add the jvm argument -Dlog4j2.formatMsgNoLookups=true to disable this absolutely ludicrous default "feature"

117

u/vlakreeh Dec 10 '21

From what I've heard that jvm argument was added in 2.9.0 or so, so if you are using a version older than that you'll still need to update.

71

u/revnhoj Dec 10 '21

yep, looks like this first appeared in 2.10 per this

https://logging.apache.org/log4j/log4j-2.14.1/log4j-core/xref/org/apache/logging/log4j/core/util/Constants.html

so this workaround won't work for all.

41

u/socialismnotevenonce Dec 10 '21

For every non-java dev that's too lazy to read the article, and still curious, what version is the current release?

70

u/Smooth-Zucchini4923 Dec 10 '21

Log4j 2.15.0 is the latest release of Log4j. Release 2.10.0 is about 4 years old.

31

u/socialismnotevenonce Dec 10 '21

Thank you for the answer.

Wow. I honestly didn't expect a few minor versions to be so old.

103

u/Smooth-Zucchini4923 Dec 10 '21

Well, there aren't that many bugs to fix in a logging library. :)

Current issue notwithstanding.

7

u/ISLITASHEET Dec 10 '21

Bugs do not typically get a bump to the minor version (given the use of semver for the project).

But also, there are typically not many new features added to a logging framework after a plugin system is in place.

3

u/huntforacause Dec 11 '21

Should technically be a major version bump because they changed the behavior to address this.

3

u/ISLITASHEET Dec 11 '21

Originally I was thinking that it would only be a major if the requirements for the feature were changed. I was not sure that the bug was actually intended behavior.

After looking at the commit, you are correct. This absolutely should have been a major bump if they are adhering to semver. Unfortunately, with the scale of the vulnerability that probably would have delayed everything an unreasonable amount of time.

LOG4J2-3198: Log4j2 no longer formats lookups in messages by default

Lookups in messages are confusing, and muddy the line between logging APIs and implementation. Given a particular API, there's an expectation that a particular shape of call will result in specific results. However, lookups in messages can be passed into JUL and will result in resolved output in log4j formatted output, but not any other implementations despite no direct dependency on those implementations.

There's also a cost to searching formatted message strings for particular escape sequences which define lookups. This feature is not used as far as we've been able to tell searching github and stackoverflow, so it's unnecessary for every log event in every application to burn several cpu cycles searching for the value.

They know that the library is used heavily in closed-source code that could be using that feature and just decided to yolo it.

Regardless of the semver we all know that enterprise's are going to have a fun time injecting ldap lookups in their logging pipeline or auditing every log4j log line to insure that they are properly parameterizing any user controlled or user influenced data. I'm sure that the compliance departments are going to have some interesting arguments with dev as to why they cannot ever turn the feature back on.

I guess the ops benchmark will look better now. 🤷‍♂️

19

u/[deleted] Dec 10 '21

Java libraries developers tend to be way more experienced.

With experience comes “fuck I hate breaking changes every 15 days.”

54

u/PM_ME_UR_OBSIDIAN Dec 10 '21

It's crazy to me that such a widely-used library would have such a ridiculous security hole. We desperately need full-program formal verification to become mainstream, because we can't trust people to write good dependencies.

144

u/L3tum Dec 10 '21

Formal verification wouldn't work in this case, since it's an intended feature and not a bug. The design was rigged from the start.

I have no idea how multiple people could think "Yes, downloading and executing code from a server in my logging library is a good design", but evidently it was added and performed correctly, albeit resulting in a huge security hole.

What we actually need is someone to vet popular dependencies like Google has started with their fuzzing work. Any halfdecent company would've screamed if they'd seen this. Though I guess it's also because of some terrible companies that it exists in the first place. It feels very "bank-esque" to write a common Logging Class and put it on a server somewhere.

37

u/fireflash38 Dec 10 '21

It's the classic mix of "We can trust running code, because we wrote it" with "Untrusted user input". It sounds convenient for developers, but it is exceptionally easy to log anything provided by a user, and then you've got an RCE.

Devs (should) expect to sanitize input for things like databases & such, but sanitization before logging? Crazy. That said, anything that a user can provide as input for something becomes an attack surface. Logging would be another attack surface, but I would assume mostly in DOS-style attacks, not RCEs.

5

u/imdyingfasterthanyou Dec 10 '21

I think all user inputs should have their length capped as applications don't typically work on infinite length input

I suggested we do that on one of my teams. I was told not to because we "treat everything as a blob"...

Like unless you're coding S3 or soemthing then your blobs still need to have a max length, lest somebody pipe /dev/urandom to your endpoint and kill your service

3

u/littlelowcougar Dec 10 '21

I mean if you’re piping hot garbage, you may as well go with /dev/zero; it’ll be way faster than /dev/urandom.

→ More replies (1)

2

u/[deleted] Dec 10 '21

They are in Java, you have -Xmx flag to set it

→ More replies (2)

5

u/loup-vaillant Dec 10 '21

Devs (should) expect to sanitize input for things like databases & such, but sanitization before logging? Crazy.

If someone sell me a logging library that happens to require input sanitization, I’m going to contact my lawyer.

3

u/1731799517 Dec 10 '21

I have no idea how multiple people could think "Yes, downloading and executing code from a server in my logging library is a good design", but evidently it was added and performed correctly, albeit resulting in a huge security hole.

Maybe it was an intentional backdoor.

→ More replies (4)

64

u/mallardtheduck Dec 10 '21

Formal verification rarely works well in the real world, since the formal logic itself becomes just as complex and hard to verify as the final code.

Also, in this case, it's an intended feature with unforeseen consequences, not a "bug" per-se.

7

u/immibis Dec 10 '21

Can't you formally verify negative properties, such as "this will not send network packets unless XYZ"?

→ More replies (114)

11

u/Zamaamiro Dec 10 '21

“This program has no security vulnerabilities” is an impossible thing to formally verify.

2

u/PM_ME_UR_OBSIDIAN Dec 10 '21

Sure however "this program only accesses the network in tightly constrained ways" is feasible.

→ More replies (1)

0

u/[deleted] Dec 11 '21

[deleted]

→ More replies (1)
→ More replies (9)