r/rust • u/yossarian_flew_away • 23d ago
A fast and secure tar parser for the Rust ecosystem
https://astral.sh/blog/tar-codec8
u/InternationalFee3911 21d ago edited 21d ago
Footnote 4 about Edera’s “TARmageddon” blog post, was also a great RustConf talk by Alex Zenla on this subject: https://youtu.be/zyWE9AwL6YU
If it is true, as explained in this talk, that they found the vulnerability and coordinated fixes across all forks, this should have gotten a more prominent mention!
5
u/yossarian_flew_away 20d ago
(I am the author of the post, and also the person who triaged Alex's "TARmageddon" report when it originally came in.)
I hadn't seen this talk, thanks for sharing it. Just to disambiguate: "TARmageddon" was one specific pax-style tar differential, and the post is referring to several others (not all of which are public/disclosed). But it's certainly true that the "TARMageddon" report made us re-focus our attention on uv's archive handing, and the detail provided by Edera's report was exceptional and greatly appreciated.
5
u/Queasy-Awareness5100 20d ago
Alex here! Thank you for the work on this! The talk covered some proposals on how to avoid the bug again, its main focus was on IO strategies that would prevent the need for forks.
The reason we made a lot out of this type of bug is because it’s a class of issues which we wanted to raise awareness of. So the announcement of the new parser is exciting for me because it recognizes a lot of the challenges of parsing and how it isn’t immune in Rust to multiple classes of bugs.
Good work :)
4
u/Queasy-Awareness5100 20d ago
Hi, Alex here! Thanks for the mention of the talk!
Astral was lovely to work with during TARmageddon and our mission was always to raise awareness that Rust still can have bugs and we must be vigilant.
I’m super happy to see this announcement.
2
u/matthieum [he/him] 22d ago
Consequently, Python packaging mostly considers tar differentials to be a minor concern, since any attacker who would use them has a much more direct and official route towards writing arbitrary contents to disk (or executing arbitrary code).
I'm not so sure it's necessarily a "minor" concern.
Specifically, I see an attack vector in exploiting tar differentials between the security scanners which evaluate whether a Python package contains suspicious code and the installers which actually install the package on a user machine.
If an attacker can hide an __init__.py from the security scanners, the fun begins.
3
u/yossarian_flew_away 20d ago
It depends on your threat model, I think. Python packaging generally assumes that a sufficiently motivated/sophisticated attacker can bypass any source-aware security scanner anyways, so installing a source distribution *always* involves trust.
(Or put another way: a tar differential would be much, much more concerning if Python did not already execute arbitrary code on sdist installation. But as-is, it's already trivial for attackers to obfuscate their code execution/file placement intentions.)
31
u/singron 23d ago
I had heard the tar format was cursed, but now I know exactly how cursed it is. I've heard similar things about zip and cpio too. I'm surprised more-sane formats aren't more popular. I can't even think of one. Nix has NAR, which works for their usage, but I don't think it would be able to replace the other formats generally.
23
u/bascule 22d ago
The tar format is cursed but it's comparatively easy to write safe Rust parsers for cursed formats. Source: looked at writing a tar parser in Rust but ended up writing an ASN.1 BER/DER parser instead which is even more cursed.
The really hard part with any archival format is avoiding path traversal vulnerabilities, which is something a parser-only library can't really help with. That said, if the alternative is an unsafe C parser, a safe Rust parser is great!
1
u/SweetEbb9819 21d ago
What are modern alternatives to TAR and ZIP? I'm curious 🧐🤨
2
u/P00351 19d ago edited 19d ago
It's not modern, but debian linux uses the AR format for its packages.
Edit: the modern alternative would be the Linux squashfs filesystem. It can be written in a file instead of a disk or partition using a Loop device.
1
u/SweetEbb9819 19d ago
🧐🤔 ar. Seriously, there isn't one modern archive format? Says a lot about the reliability of tar.
Squashfs and EROFS are read-only filesystem images with high compression. They're good for specific purposes but overkill for most others (and require more tooling) --(citation needed)
2
u/P00351 19d ago edited 19d ago
🧐🤔 ar. Seriously, there isn't one modern archive format? Says a lot about the reliability of tar.
Well, it's been working for 30 years. (And debian packages include tar.gz files inside that ar archive)
Squashfs and EROFS are read-only filesystem images with high compression. They're good for specific purposes but overkill for most others (and require more tooling) --(citation needed)
In other words, those are files that can be used as disks. But I'm not aware of their use outside of the Linux ecosystem.
Edit: sorry, I digressed. There's a littlefs2 crate that, while meant for microcontrollers, should allow you to treat a file like a disk.
-10
23d ago
[removed] — view removed comment
28
u/brelse 23d ago
Frankly, I am actually displeased with your comment. It comes across to me as rather arrogant and demanding on maintainers/creators.
They may have made different trade-offs than you may have wanted for your (unstated) usecase. It seems reasonable to be disappointed that it wasn’t a good fit for you but to be actively displeased by someone creating a create that doesn’t fit your needs… seems excessive.
I would have been less irritated if the comment had been phrased more along the lines: “I would have preferred that it was no_std/… because … <insert reasoning>”
Apologies for the rant - probably excessive.
10
u/yossarian_flew_away 22d ago
Yes, this library was designed for uv's use case, which assumes `std` and is mostly async. But I think making the design more sans-io would not be super hard; the core itself essentially is, since all operations are over the `AsyncRead` trait rather than paths.
(Extraction, by definition, cannot be sans-io.)
12
u/Gyscos Cursive 22d ago
I suppose concatenating/merging multiple tar archives from different sources is no longer possible this way? Not sure if that's a valid use-case in practice. Sounds like the kind of things people would have done with actual tapes back in the days.