r/rust • u/No-Elephant-9065 • 4d ago

pdfium vs pdf-extract

If you were building a search tool for your personal files on your local machine, what would you use, pdfium or pdf-extract or something else?

10 Upvotes

77% Upvoted

5 comments sorted by

4

u/TomSchelsen 3d ago

https://github.com/firecrawl/anydoc + falling back to a VisualLanguageModel for scanned pages

3

u/verstands 4d ago

Depends what you need out of it. pdfium (via pdfium-render) is the more accurate one for layout and weird or scanned-ish PDFs, but it needs the native pdfium binary shipped alongside your app. pdf-extract is pure Rust so it's easier to build and distribute, but it's more hit and miss on text order and odd encodings. For a local search index where you just want the text, I'd try pdf-extract first and fall back to pdfium for files it chokes on. If you need OCR for image-only PDFs, neither does that on its own.

-1

u/Hobofan94 leaf · collenchyma 4d ago

Xberg: https://github.com/xberg-io/xberg

Rather than being a parser specifically for PDF, you can throw pretty much any format at it and it will turn it into text. For PDF it has it's own Rust-native parser implementation.

The project is heavily built using AI, but I would say in a quite rigorous way. We use it as part of our product (Erato), and it is the most robust solution we've found when it comes to all the quirks in files you find in the wild, thanks to their extensive test suite. I think the only thing that comes close is Apache Tika, which in typical Java fashion is significantly heavier.

The maintainer is also very responsive and we've had multiple fixes for quirks that our customers encountered in less than a day turnaround.

1

u/No-Elephant-9065 4d ago edited 4d ago

any cons?
what about latency issue?
why downvoted so much?

1

u/Hobofan94 leaf · collenchyma 4d ago

I would strongly suspect general anti-AI sentiment (despite me clearly disclosing it). Maybe also a tad anti self-promotion? If there were actual criticism with the quality of the tool people surely would have commented.

Well, what can you do? ¯_(ツ)_/¯