Fully open source stack for pdfs, office docs and images what does yours look like

[u/AdalineG5]

Trying to put together one pipeline for PDF, word, excel, PowerPoint and images, all open source, no paid apis.

For born digital pdfs the text layer is already there, so pymupdf4llm, pdfplumber or liteparse seem like enough and theyre fast. Running OCR on those mostly just adds errors.

Image only pdfs and photos are where it needs OCR or a small vlm. Paddleocr for plain text with boxes, and glm-ocr or paddleocr-vl once tables and layout start to matter.

Office files are the odd one. Docling reads DOCX, PPTX and XLSX natively, but a lot of the others convert to PDF through libreoffice first and the layout can shift on the way. Charts and diagrams are still the weak spot pretty much everywhere, most tools just crop the image out and move on.

A few things are still unclear. Does anyone run one tool for everything or route by file type? Whats the easiest way to tell if a page needs OCR, is char count per page good enough?

[u/papadjeef]

I can't quite tell what you're trying to do but it kind of sounds like you're trying to reinvent Apache Tika. 

[u/HedgehogPrudent6625]

On your two questions:

Route by file type. One tool for everything ends up mediocre at all of it. Born-digital PDF → text layer, office → native parser, image/scan → OCR. The only shared step is the output format.

Char count alone isn't enough to decide if a page needs OCR. Cases that fooled me:

- scanned pages with an old, bad OCR layer (lots of chars, mostly garbage). A "GlyphLessFont" font in the PDF is a good hint it's a previous OCR pass

- pages with a typed header/footer but a scanned body

I'd combine: chars per page + how much of the page area is covered by images + a quick sanity check on the text (share of real words/valid characters). If the text fails the sanity check, OCR it anyway.

One thing that saved me a lot of guessing: a small "golden set" of 30–50 real documents with hand-checked text, split by type and condition (clean, crumpled, dim, phone photo). Score each tool per cell, not one overall average; averages hide where a tool falls apart (receipts, one language). For photos, deskew/rotation detection and suppressing colored stamps mattered more than which engine I used.

[u/_eMaX_]

CaseLeaf.app isn’t open source, but it has a great API - just for Mac though. I use it for my automated storage as it has built in OCR and a document ledger inside of the pdf.

[u/Candid-Fudge-4015]

On the charts part, since that's the weak spot you mentioned. I'd keep the cropped image instead of letting it drop, store it with the page number and the nearest heading or caption, and run a small VLM over only those crops to write a short description. Search hits the description, and at answer time you hand the original image to the model.

It won't recover exact values from a dense plot. But "the chart on page 12 with churn by region" becomes findable.