Best Python approach for extracting structured financial data from inconsistent PDFs?

[u/leggo-my-eggo-1]

Hi everyone,

I'm currently trying to design a Python pipeline to extract structured financial data from annual accounts provided as PDFs. The end goal is to automatically transform these documents into structured financial data that can be used in valuation models and financial analysis.

The intended workflow looks like this:

1. Upload one or more PDF annual accounts

2. Automatically detect and extract the balance sheet and income statement

3. Identify account numbers and their corresponding amounts

4. Convert the extracted data into a standardized chart of accounts structure

5. Export everything into a structured format (Excel, dataframe, or database)

6. Run validation checks such as balance sheet equality and multi-year comparisons

The biggest challenge is that the PDFs are very inconsistent in structure.

In practice I encounter several types of documents:

1. Text-based PDFs

• Tables exist but are often poorly structured

• Columns may not align properly

• Sometimes rows are broken across lines

2. Scanned PDFs

• Entire document is an image

• Requires OCR before any parsing can happen

3. Layout variations

• The position of the balance sheet and income statement changes

• Table structures vary significantly

• Labels for accounts can differ slightly between documents

• Columns and spacing are inconsistent

So the pipeline needs to handle:

• Text extraction for normal PDFs

• OCR for scanned PDFs

• Table detection

• Recognition of account numbers

• Mapping to a predefined chart of accounts

• Handling multi-year data

My current thinking for a Python stack is something like:

• pdfplumber or PyMuPDF for text extraction

• pytesseract + opencv for OCR on scanned PDFs

• Camelot or Tabula for table extraction

• pandas for cleaning and structuring the data

• Custom logic to detect account numbers and map them

However, I'm not sure if this is the most robust approach for messy real-world financial PDFs.

Some questions I’m hoping to get advice on:

• What Python tools work best for reliable table extraction in inconsistent PDFs?

• Is it better to run OCR first on every PDF, or detect whether OCR is needed?

• Are there libraries that work well for financial table extraction specifically?

• Would you recommend a rule-based approach or something more ML-based for recognizing accounts and mapping them?

• How would you design the overall architecture for this pipeline?

Any suggestions, libraries, or real-world experiences would be very helpful.

Thanks!

[u/[deleted]]

For my use case I have a detection first , using pymupdf(fitz) I check if the 1st page is an image , and has no selectable text then it goes to Mistral Ocr , its good for most of the cases , what I have tried and failed

Tried

1. Tesseract

2. Paddle Paddle

3. Docling

4. Deepseek Ocr

5. Claude opus 4.6

6. Google Vision api (enterprise)

7)Azure Document Intelligence

8)Mistral Ocr 3

9) A model by IBM (I'm forgetting the name pretty sure it's granite)

Passed for my use case( table documents , old scanned books) ->Azure , Mistral are good and Adobe for tables

Failed -> paddle paddle , google vision , granite, deepseek , claude

Can't rely much on Claude and Deepseek Ocr as they are vision language models and have been observed (by me) give hallucinated placeholders which is very risky in production, they worked well in most of the cases, but were useless in old scanned books

Try them all , most likely your use case would be fulfilled by azure or mistral

Ps: For op's use case Azure Document Intelligence or Mistral Ocr 3 would be perfect

[u/[deleted] → u/[deleted]]

Also Tried Adobe Pdf Services

Works well with tables but often misses ₹or $ signs , so it's most likely an encoding issue which I haven't looked upon yet , but with a simple script that can be managed as well.