PDF to Markdown benchmark: PDFluent on opendataloader-bench

In its default auto tier, PDFluent 1.4.0 scores 0.9282 overall on opendataloader-bench, a public PDF to Markdown benchmark of 200 documents.* The score was measured with the benchmark's own harness and evaluator and submitted to the benchmark repository; PDFluent is not on its public leaderboard yet.

pip install pdfluent==1.4.0
Last updated · PDFluent measured 11 Nov 2026 (auto and fast, SDK 1.4.0) · leaderboard read 3 Nov 2026
SDK 1.4.0 · opendataloader-bench · 200 documents · measured 11 Nov 2026

SDK 1.4.0 against SDK 1.3.0 on opendataloader-bench*

On opendataloader-bench, SDK 1.4.0 in its default auto tier scores 0.9282 overall*, 0.9463 on reading order, 0.9140 on table structure and 0.8767 on heading hierarchy, on all 200 documents, with the benchmark's own harness and evaluator. The same harness and settings gave SDK 1.3.0 0.9154, 0.9359, 0.8695 and 0.8678.

RowOverallReading orderTable structureHeading hierarchy
PDFluent 1.4.0 auto0.92820.94630.91400.8767
PDFluent 1.3.0 auto0.91540.93590.86950.8678

*Measured 11 Nov 2026 on the final macOS arm64 release wheel (Apple M1 Pro, macOS 26.5), quality="auto" with keep_furniture=True and the layout, table and formula models installed. Table structure counts 42 documents, heading hierarchy 107, reading order and overall 200.

  • The scores are measured on the benchmark's own 200 documents.
  • One run, no confidence interval.
  • Both rows keep page headers and footers, because the benchmark's ground truth counts them as content.
  • The table structure score depends on the platform, so the comparison is Mac against Mac.

Known issue: on the olmOCR-bench headers and footers category, SDK 1.4.0 quality="auto" scores 0.0013 lower than SDK 1.3.0 (one run, 6 November 2026). The other seven categories are equal or higher. keep_furniture=True keeps page headers and footers on every tier.

The sections below describe SDK 1.3.0 with their own dates and footnotes; to reproduce those rows, install the version that was measured.

opendataloader-bench · 200 documents · two PDFluent tiers

PDFluent 1.4.0 scores

Two rows, one per extraction tier, each scored on all 200 documents of the benchmark.

PDFluent 1.4.0 autoauto · default in 1.3.0
0.9282*

Overall score, default tier: layout and table models, OCR for pages without a text layer.

Reading order
0.9463
Table structure
0.9140
Heading hierarchy
0.8767
PDFluent 1.4.0 fastfast · no models
0.8869*

Overall score, fast tier: the engine alone, no models installed.

Reading order
0.9145
Table structure
0.7213
Heading hierarchy
0.8601
Harness
opendataloader-bench at commit 7af1d8f4d0c09f51ea1a5c6ba5f66e993286d109, its own evaluator
Documents
all 200
Machine
Google Cloud n2-standard-8, CPU only, Linux; table scores differ slightly by platform, see the laptop measurement below
Measured
PDFluent 11 Nov 2026 (auto and fast, SDK 1.4.0); leaderboard README read 3 Nov 2026
Who measured
PDFluent rows: measured by us. The 12 other engines: the published figures from the benchmark's README, not re-run by us.
Status

Submitted to the benchmark repository, PR #37 (scores) and PR #38 (per-document predictions), not on its public leaderboard yet. The steps to reproduce both rows are on the opendataloader-bench page.

Reproduced from the PyPI wheel on a laptop†
RowOverallReading orderTable structureHeading hierarchySeconds per document
PDFluent 1.3.0 auto0.91540.93590.86950.86782.6619
PDFluent 1.3.0 fast0.85730.89980.63160.81470.0091
  • † Second measurement, reproduced from the PyPI wheel (pdfluent 1.4.0) on a laptop: Apple M1 Pro, macOS 26.5, Python 3.13, harness at commit 7af1d8f, all 200 documents, keep_furniture=True, measured 2 Nov 2026.
  • Median of three timed runs after a warm-up. These are the figures in the submission (PR #37).
  • Table structure differs from the Linux run above by platform; every number here belongs to this machine.
  • Seconds per document are for this laptop only. They are not compared with the speeds in the benchmark's README, which come from other machines.

*Measured with the benchmark's own harness and evaluator, opendataloader-bench at commit 7af1d8f.

Submitted to the benchmark repository, PR #37 (scores) and PR #38 (per-document predictions), not on its public leaderboard yet. The steps to reproduce both rows are on the opendataloader-bench page.

  • The scores are measured on the benchmark's own 200 documents.
  • One run, no confidence interval.
  • Both PDFluent rows keep page headers and footers (keep_furniture=True), because the benchmark's ground truth counts them as content.
  • The table structure score depends on the platform; PDFluent 1.4.0 was measured on macOS arm64, both rows.
  • Competitor scores are the published figures from the benchmark's README, not re-run by us.
Table 1 · 12 engines from the README at 7af1d8f plus PDFluent

Published leaderboard

Every engine in the benchmark's README, with its published scores and speed, and the two PDFluent rows, which are submitted and not on the leaderboard yet. Sorted by overall.

Sorted by one measure: reading order · table structure · heading hierarchy. Licence as the README states it (A67); PDFluent: the SDK licence.

EngineOverallReading orderTable structureHeading hierarchySpeed (s)Licence
PDFluent 1.4.0 autosubmitted, PR #370.92820.94630.91400.8767s/doc, Apple M1 ProAGPL-3.0-only or the PDFluent Commercial Licence, at your option
opendataloader [hybrid]0.9070.9340.9280.8210.463s/page as published, Apple M4Apache-2.0
PDFluent 1.4.0 fastsubmitted, PR #370.88690.91450.72130.8601s/doc, Apple M1 ProAGPL-3.0-only or the PDFluent Commercial Licence, at your option
nutrient0.8850.9250.7080.8190.008s/page as published, Apple M4Commercial
docling0.8820.8980.8870.8240.762s/page as published, Apple M4MIT
marker0.8610.8900.8080.79653.932s/page as published, Apple M4GPL-3.0
unstructured [hi_res]0.8410.9040.5880.7493.008s/page as published, Apple M4Apache-2.0
edgeparse0.8370.8940.7170.7060.036s/page as published, Apple M4Apache-2.0
opendataloader0.8310.9020.4890.7390.015s/page as published, Apple M4Apache-2.0
mineru0.8310.8570.8730.7435.962s/page as published, Apple M4AGPL-3.0
pymupdf4llm0.7320.8850.4010.4120.091s/page as published, Apple M4AGPL-3.0
unstructured0.6860.8820.0000.3880.077s/page as published, Apple M4Apache-2.0
markitdown0.5890.8440.2730.0000.114s/page as published, Apple M4MIT
liteparse0.5760.8660.0000.0001.061s/page as published, Apple M4Apache-2.0

A77, A78, A65, A66, A71 · opendataloader-bench README at commit 7af1d8f, read 3 Nov 2026 · leaderboard speeds as published (column "Speed (s/page)"; each engine's summary.json names an Apple M4 and the values equal its seconds per document) · PDFluent scores macOS arm64, Apple M1 Pro (A77, A78); speeds 1.3.0, Apple M1 Pro (A66). The leaderboard speeds were measured on an Apple M4, the PDFluent speeds on an Apple M1 Pro, and the submissions on the hardware each pull request states. The speeds are not comparable, and no speed comparison is made in the text. Source: opendataloader-bench at 7af1d8f; PDFluent: PR #37 (scores) and PR #38 (per-document predictions).

Table 2 · open pull requests to opendataloader-bench

Submitted, not yet on the leaderboard

4 other open pull requests add 8 configurations of 4 engines to the benchmark. None of them is on the leaderboard yet. The figures and the hardware are as each pull request states them, read 3 Nov 2026.

Engine and configurationOverallReading orderTable structureHeading hierarchys/docHardwarePull request
kordoc 4.15.7kordoc-plain-ocr0.97110.97510.98280.95790.458Apple M5#35, 27 Sep 2026
kordoc 4.15.7kordoc-ocr0.95980.95980.97900.94890.447Apple M5#35, 27 Sep 2026
kordoc 4.15.7kordoc-plain0.94910.95400.93950.94250.032Apple M5#35, 27 Sep 2026
kordoc 4.15.7kordoc (defaults)0.93710.93800.93570.93260.035Apple M5#35, 27 Sep 2026
LM-Kit One 2026.9.8lm-kit-ocr (text layer + OCR)0.96300.97180.95770.94870.0243Apple M4 Max#34, 25 Sep 2026
LM-Kit One 2026.9.8lm-kit (text layer, no OCR)0.94780.95450.93420.93890.0041Apple M4 Max#34, 25 Sep 2026
folio (version not stated)folio0.9240.9470.8940.8790.0394-core Xeon 2.1 GHz#33, 22 Sep 2026
liteparse 2.10.1liteparse0.87320.91270.69290.81140.0039Apple M2 Max#28, 18 Jun 2026Figures not stated in the PR text; from the PR's committed prediction/liteparse/evaluation.json, rounded to four decimals.

A71 · figures and hardware as each pull request gives them, read 3 Nov 2026 · open pull requests to opendataloader-bench, not on its leaderboard yet. Each row is the submitter's own run on the hardware it names, not re-run by us. PDFluent's own submission, PR #37 (scores) and PR #38 (per-document predictions), is in table 1. Source: open pull requests of opendataloader-bench.

Per measure · 0 to 1 · higher is better

Where PDFluent stands on each measure

In its default auto tier, PDFluent 1.4.0 scores 0.9282 overall on all 200 documents: 0.9463 on reading order, 0.9140 on table structure and 0.8767 on heading hierarchy.

Overall0 to 1 · higher is better
PDFluent 1.4.0 auto0.9282
opendataloader [hybrid]0.907
PDFluent 1.4.0 fast0.8869
nutrient0.885
docling0.882
marker0.861
unstructured [hi_res]0.841
edgeparse0.837
opendataloader0.831
mineru0.831
pymupdf4llm0.732
unstructured0.686
markitdown0.589
liteparse0.576

In the auto tier, PDFluent scores 0.9282 on overall.

In the fast tier, with no models, it scores 0.8869.

Reading order (NID)0 to 1 · higher is better
PDFluent 1.4.0 auto0.9463
opendataloader [hybrid]0.934
nutrient0.925
PDFluent 1.4.0 fast0.9145
unstructured [hi_res]0.904
opendataloader0.902
docling0.898
edgeparse0.894
marker0.890
pymupdf4llm0.885
unstructured0.882
liteparse0.866
mineru0.857
markitdown0.844

In the auto tier, PDFluent scores 0.9463 on reading order.

In the fast tier, with no models, it scores 0.9145.

Table structure (TEDS)0 to 1 · higher is better
opendataloader [hybrid]0.928
PDFluent 1.4.0 auto0.9140
docling0.887
mineru0.873
marker0.808
PDFluent 1.4.0 fast0.7213
edgeparse0.717
nutrient0.708
unstructured [hi_res]0.588
opendataloader0.489
pymupdf4llm0.401
markitdown0.273
unstructured0.000
liteparse0.000

In the auto tier, PDFluent scores 0.9140 on table structure.

In the fast tier, with no models, it scores 0.7213.

Heading hierarchy (MHS)0 to 1 · higher is better
PDFluent 1.4.0 auto0.8767
PDFluent 1.4.0 fast0.8601
docling0.824
opendataloader [hybrid]0.821
nutrient0.819
marker0.796
unstructured [hi_res]0.749
mineru0.743
opendataloader0.739
edgeparse0.706
pymupdf4llm0.412
unstructured0.388
markitdown0.000
liteparse0.000

In the auto tier, PDFluent scores 0.8767 on heading hierarchy.

In the fast tier, with no models, it scores 0.8601.

A77, A78, A65, A64 · opendataloader-bench @ 7af1d8f · 200 documents · PDFluent measured 11 Nov 2026 (auto and fast, SDK 1.4.0) · leaderboard README read 3 Nov 2026 · PDFluent four decimals as submitted, others three as published. Caveats.
opendataloader-bench

What do NID, TEDS and MHS measure?

NID scores reading order, TEDS scores table structure and MHS scores heading hierarchy, each from 0 to 1 against the benchmark's ground truth.

Overall

The benchmark's own overall score: the mean over the documents of each document's combined score, as its evaluator computes it.

One number for a first comparison. The three measures below tell you where it comes from.

Reading order (NID)

Normalized indel distance between the extracted text and the ground truth: how much of the text comes out, in the right order.

Reading order matters for RAG chunking: a chunk whose sentences come out of order retrieves and reads badly.

NID = 1 − distance / (len(gt) + len(pred))

Table structure (TEDS)

Tree edit distance similarity between each extracted table and the true one, counting rows, columns, spans and cell text.

Tables matter for invoices, financial statements and reports, where a row has to stay a row and a cell has to stay in its column.

TEDS = 1 − EditDist(T_gt, T_pred) / max(|T_gt|, |T_pred|, 1)

Heading hierarchy (MHS)

How closely the Markdown headings and their levels match the document's own outline.

Heading levels decide where a section-aware splitter can cut a document and which section title a chunk carries.

MHS = 1 − EditDist(H_gt, H_pred) / max(|H_gt|, |H_pred|, 1)

Formulas as the benchmark's README gives them (section 3, Metrics). Documents without a table or headings in the ground truth are left out of that measure's mean.

Choosing a PDF parser

Which PDF parser fits which job?

Pick the measure that matches your documents, then read that column of the leaderboard, not only the overall score.

JobWhat mattersPDFluent tierPDFluent on this benchmark
RAG pipelineReading order (NID) and heading hierarchy (MHS): chunks in order, cut at sectionsautoReading order: 0.9463. Headings: 0.8767.
Invoices, statements, reportsTable structure (TEDS)auto, with the table modelTable structure: 0.9140.
Speed-sensitive batchTime per page on your hardwarefast, no modelsNot compared with other engines. On a laptop (Apple M1 Pro, macOS 26.5, Python 3.13) the fast tier takes 0.0091 seconds per document; see the laptop measurement.
No model downloadsThe text layer only: the package, with no models and no TesseractfastOverall: 0.8869.
Commercial licence neededThe licence column of the leaderboardeither tierAGPL-3.0-only or the PDFluent Commercial Licence, at your option.

Scores from the leaderboard table above. Licences of other engines as their README states them; check each project's own terms before you ship.

opendataloader-bench

How to reproduce the PDFluent score

Run the benchmark's own harness at the same commit, convert each PDF with the call below and score the output with its evaluator.

  1. Check out the harness at commit 7af1d8f4d0c09f51ea1a5c6ba5f66e993286d109 and install it as its README describes:
    shell
    git clone https://github.com/opendataloader-project/opendataloader-bench
    cd opendataloader-bench
    git checkout 7af1d8f4d0c09f51ea1a5c6ba5f66e993286d109
    git lfs install && git lfs pull
    uv sync
  2. Install PDFluent and its models once:
    shell
    pip install pdfluent==1.4.0
    pdfluent models install layout tables
    OCR of scanned pages in the auto row needs Tesseract installed on the machine and on PATH; the fast row needs neither the models nor Tesseract.
  3. Convert every PDF. Auto row, recipe auto+kf: extract_markdown(path, quality="auto", keep_furniture=True). Fast row, recipe fast+kf: extract_markdown(path, quality="fast", keep_furniture=True).
  4. Write the Markdown where the harness keeps predictions (prediction/<engine>/markdown) and run its evaluator, for example uv run src/evaluator.py --engine <engine>.

Expected scores

RowOverallReading orderTable structureHeading hierarchy
auto+kf0.92820.94630.91400.8767
fast+kf0.88690.91450.72130.8601

Measured 11 Nov 2026 (auto and fast, SDK 1.4.0) on Apple M1 Pro (macOS 26.5), final macOS arm64 release wheel, one run. Scores on other platforms can differ slightly, most on table structure.

The harness at this commit has no PDFluent engine of its own. The recipe files auto+kf and fast+kf in pdfluent-bench hold the exact configuration of both rows. More on the calls in the extraction guide.

The submission adds both rows to the harness as pdfluent-auto and pdfluent-fast. On the branch of PR #37 the run is:

shell
uv sync --extra pdfluent
python -m pdfluent models install layout tables
uv run src/run.py --engine pdfluent-auto --force
uv run src/run.py --engine pdfluent-fast --force
opendataloader-bench

What this result does not say

It is one benchmark of 200 documents and one run, with these limits.

  • The scores are measured on the benchmark's own 200 documents.
  • One run, no confidence interval.
  • Both PDFluent rows keep page headers and footers (keep_furniture=True), because the benchmark's ground truth counts them as content.
  • The table structure score depends on the platform; PDFluent 1.4.0 was measured on macOS arm64, both rows.
  • Competitor scores are the published figures from the benchmark's README, not re-run by us.
  • There is no confidence interval: each score is one run.
  • Submitted to the benchmark repository, PR #37 and #38, not on its public leaderboard yet.
More results

Other PDFluent benchmarks

19 measures · held-out half

Highest average score across 19 extraction measures*, against Docling 2.126.0 and pymupdf4llm 1.28.2.

Balanced tier + table model · opt-in · Preview

*Balanced tier with the table model (Preview), on our own held-out half of four public datasets, 17 Oct 2026. Average 0.721 for PDFluent, 0.693 for pymupdf4llm and 0.672 for Docling, over FinTabNet, PubTables-1M, DocLayNet and ParseBench. The table model was trained on the source datasets of PubTables-1M and FinTabNet, and most of those held-out pages are in their training split. On the 13 measures without that overlap the ranking is the same (0.650, 0.642, 0.610). Details: blog post SDK 1.2.0 and the benchmarks page.

All 19 measures
Release post

What changed in SDK 1.4.0

The auto tier, OCR for pages without a text layer, heading levels, and the upgrade notes.

Read the 1.3.0 post
opendataloader-bench

Questions about this benchmark

What is opendataloader-bench?

opendataloader-bench is a public benchmark on GitHub that scores PDF to Markdown engines on 200 PDFs. It converts each PDF with each engine and scores the output with its own evaluator on reading order (NID), table structure (TEDS) and heading hierarchy (MHS), plus an overall score. Its README publishes a leaderboard of 12 engines.

Which PDF parser scores highest on opendataloader-bench?

On the benchmark's published leaderboard, its README read 3 Nov 2026, the top listed engine is opendataloader [hybrid] at 0.907 overall. PDFluent 1.4.0 in its auto tier scored 0.9282 with the benchmark's own harness and evaluator; it is submitted (PR #37) and not on the leaderboard yet. Other engines also have submissions open that are not on the leaderboard yet; the table of submitted results on this page gives each one's own figures and hardware. The leaderboard is updated by the benchmark's maintainers, so its README has the current table.

What is the best PDF to Markdown converter for RAG?

The best one for RAG is the one that keeps reading order and heading levels right on your own documents, so test a sample of yours. Reading order decides whether a chunk keeps its sentences in order, and heading levels decide where a section-aware splitter can cut; on documents with many tables, table structure matters too. On opendataloader-bench, PDFluent 1.4.0 in the auto tier scores 0.9463 on reading order, 0.9140 on table structure and 0.8767 on heading hierarchy; the two tables on this page have every published and submitted engine's numbers. Licence and whether you can install models and Tesseract are the other two factors.

How accurate is PDF table extraction on opendataloader-bench?

Table structure (TEDS) on the published leaderboard (README read 3 Nov 2026) runs from 0.000 to 0.928, and PDFluent 1.4.0 scores 0.9140 in the auto tier. The two top published values are opendataloader [hybrid] at 0.928 and docling at 0.887 PDFluent's fast tier, which runs no table model, scores 0.7213. A TEDS of 1 means every row, column, span and cell text matches the true table; the submissions not yet on the leaderboard give their own TEDS in the second table.

Which PDF parsers are free for commercial use?

By the licences the benchmark's README states, opendataloader [hybrid], docling, unstructured [hi_res], edgeparse, opendataloader, unstructured, markitdown and liteparse are under Apache-2.0 or MIT. The same README lists marker, mineru and pymupdf4llm under GPL-3.0 or AGPL-3.0, and nutrient as Commercial. The PDFluent SDK is licensed AGPL-3.0-only or the PDFluent Commercial Licence, at your option. This is not legal advice; check each project's own terms before you ship.

How do I reproduce the benchmark?

Check out opendataloader-bench at commit 7af1d8f4d0c09f51ea1a5c6ba5f66e993286d109, install PDFluent and its models, convert each PDF with extract_markdown and score the output with the benchmark's evaluator. Install PDFluent with pip install pdfluent==1.4.0, the models with pdfluent models install layout tables, and Tesseract on the machine for OCR of scanned pages. Use extract_markdown(path, quality="auto", keep_furniture=True) for the auto row and write the Markdown where the harness expects predictions. The recipe files auto+kf and fast+kf in pdfluent-bench hold the exact configuration of both rows, and the branch of PR #37 adds both rows to the harness.

What do NID, TEDS and MHS measure?

NID measures reading order, TEDS measures table structure and MHS measures heading hierarchy, each from 0 to 1 with higher better. NID (normalized indel distance) counts how much of the text comes out, in the right order. TEDS (tree edit distance similarity) compares rows, columns, spans and cell text against the true table. MHS compares the Markdown headings and their levels with the document's outline.

Is PDFluent on the public leaderboard?

Not yet. PDFluent is submitted to the benchmark repository: PR #37 with the scores and PR #38 with the per-document predictions, both opened 2 Nov 2026. The PDFluent scores on this page were measured with the benchmark's own harness and evaluator on its 200 documents, at commit 7af1d8f. The other engines are the figures the benchmark publishes in its README; we did not re-run them.

Does PDFluent need models?

Only the auto tier does: you install its models once with pdfluent models install layout tables, and without them auto falls back to the fast tier and says so. The auto tier, the default from SDK 1.4.0, uses a layout model, a table model and a heading-level model, and runs OCR on pages without a text layer; that OCR needs Tesseract installed on the machine. The fast tier needs neither: it reads the PDF's text layer with the engine alone.

How does auto differ from fast?

Auto reads each document with the layout and table models, gives headings their level and sends pages without a text layer to OCR; on opendataloader-bench it scores 0.9282 overall. Fast runs no model and was the default up to SDK 1.2; on the same benchmark it scores 0.8869 overall. To keep the 1.2 behaviour after upgrading, pass quality="fast".

Is PDFluent free for commercial use?

The PDFluent desktop editor is free for everyone, including commercial use. The SDK, which is what this benchmark measures, is licensed AGPL-3.0-only or the PDFluent Commercial Licence, at your option.

Why do both PDFluent rows use keep_furniture=True?

Because the benchmark's ground truth counts page headers and footers as content. PDFluent drops them by default, which this benchmark would score as missing text, so both rows keep them. Outside this benchmark the default is unchanged.

SDK 1.4.0

Get started with PDFluent

Install the Python package, install the models once, and call extract_markdown.

pip install pdfluent==1.4.0

PyMuPDF and MuPDF are trademarks of Artifex Software, Inc. PDFluent is an independent product of Innovation Trigger B.V. and is not affiliated with or endorsed by Artifex.