PDF processing crashed or timed out on large fileLarge PDF files cause crashes when the processor loads the entire document into memory at once. On constrained servers, the process is killed by the OS OOM killer. On cloud functions, the timeout is hit before processing completes. PDFluent's streaming parser processes any file size in constant memory.
Most PDF libraries read the full document into memory on open(). A 500 MB PDF with embedded images and fonts may consume 1–3 GB of working set after parsing. On a server with 2 GB RAM, this triggers an OOM kill. The process dies with no error message — just a sudden termination.
Non-streaming PDF processors accumulate all output in memory before writing the result file. For a 1000-page document, this means holding the entire input and output simultaneously. Peak memory usage is the sum of both, which can be many times the input file size.
Cloud functions (Lambda, Cloud Run) and API gateways impose hard timeouts, often 30 seconds or 60 seconds. Processing a 200-page PDF with text extraction, font loading, and image decompression can exceed these limits with non-optimised libraries.
PDFluent's streaming parser reads each page from disk on demand and drops it from memory immediately after processing. Memory usage stays constant regardless of document size — a 10 MB and a 10 GB PDF use the same peak memory.
use pdfluent::PdfDocument;
// Open with streaming mode — pages are read from disk on demand
let doc = Document::open_streaming("large.pdf")?;
println!("Pages: {}", doc.page_count()); // reads only the index, not all pages
for page in doc.pages() {
let page = page?;
let text = page.text()?;
println!("{}", text);
// page is dropped here — memory freed immediately
// next page is read from disk on the next iteration
}When you need to modify or render pages, process them one at a time and write incremental output. This avoids accumulating the full output document in memory before saving.
use pdfluent::{PdfDocument, DocumentWriter};
let source = Document::open_streaming("large.pdf")?;
let mut writer = DocumentWriter::new("output.pdf")?;
for page in source.pages() {
let mut page = page?;
// Process page
page.set_text_color((0, 0, 0))?;
// Write page directly to output — not buffered in memory
writer.append_page(page)?;
}
writer.finish()?;
println!("Done — processed {} pages", source.page_count());For workloads that must fit within a timeout, split large documents into batches and process each batch independently. PDFluent supports page-range extraction to split a document without loading all pages.
use pdfluent::PdfDocument;
const BATCH_SIZE: usize = 50;
let doc = Document::open_streaming("large.pdf")?;
let total = doc.page_count();
for start in (0..total).step_by(BATCH_SIZE) {
let end = (start + BATCH_SIZE).min(total);
let batch = doc.extract_page_range(start..end)?;
batch.save(&format!("batch_{start}_{end}.pdf"))?;
println!("Processed pages {start}–{end}");
}