LLM Document Parsing: From Raw Files to a RAG-Ready Corpus

LLM document parsing is the step that decides whether your RAG system works. Retrieval quality is bounded by corpus quality: no reranker, no chunking trick and no prompt template will recover a table that was flattened during parsing. This page walks through the pipeline we use and where it usually breaks.

Updated 2026-10-13

Start parsing API documentation

What LLM document parsing is (and is not)

Parsing for LLMs is not text extraction. Extraction asks "what characters are on this page". Parsing asks "what is this document saying, and how is it organised". The output is not a string — it is structured content with headings, tables, formulas and reading order, plus a signal for how sure the parser is.

Extraction output: characters, in page order, with layout artefacts.
Parsing output: semantic blocks, in reading order, with structure and confidence.
What your vector store wants: the second one.

The five-stage pipeline

Stage	Input	Output	What goes wrong
1. Ingest	PDF, scans, office files, images	Normalised file + page count	Encrypted or password-protected files; 200 MB scans
2. Parse	Normalised file	Text + structure (OCR when needed)	Image-only pages parsed as blank; columns interleaved
3. Structure	Parsed blocks	Markdown with headings, tables, LaTeX	Headings detected as bold text; tables flattened
4. Chunk	Markdown	Chunks + heading path metadata	Fixed-size chunks cut mid-table and mid-section
5. Embed & index	Chunks	Vectors + metadata	Boilerplate embedded; low-quality chunks pollute recall

Most teams spend their time on stage 4 and 5. In our experience the failures that hurt retrieval happen in stage 2 and 3, and they are invisible until someone complains about a wrong answer.

Why Markdown changes how you chunk

With raw extracted text you have one option: split every N characters, and hope. With Markdown you can split on structure.

def chunk_markdown(md: str, max_chars: int = 1500):
    """Split Markdown on heading boundaries, keeping the heading path."""
    chunks, path, buf = [], [], []
    for line in md.splitlines():
        if line.startswith('#'):
            level = len(line) - len(line.lstrip('#'))
            if buf:
                chunks.append({"path": " > ".join(path), "text": "\n".join(buf)})
                buf = []
            path = path[: level - 1] + [line.lstrip('# ').strip()]
        else:
            buf.append(line)
        if sum(len(x) for x in buf) > max_chars:
            chunks.append({"path": " > ".join(path), "text": "\n".join(buf)})
            buf = []
    if buf:
        chunks.append({"path": " > ".join(path), "text": "\n".join(buf)})
    return chunks

The heading path becomes metadata. At query time you can show it in the citation ("Q3 Results > Revenue by Region"), which is what makes users trust an answer.

The cases that break parsers

Document type	Failure mode	What to check
Two-column academic papers	Columns interleaved mid-sentence	Reading order, not page order
Financial reports	Tables flattened into whitespace	Cells and headers survive as a Markdown table
Scanned contracts	Blank output (no OCR)	OCR path is used and text is non-empty
Slides (PPTX)	One bullet per "page", context lost	Slide titles become headings
Forms and invoices	Key-value pairs scrambled	Label stays adjacent to its value

Build a golden set: Pick 20 representative documents, parse them, and manually score the output. Re-run that set after every parser change. It takes an afternoon and it is the only reliable way to know whether a change helped.

Controlling quality at scale

Per-block confidence — MarkdFlow returns a confidence score per block. Route low-confidence blocks to review instead of indexing them blind.
Structural assertions — assert that a document produced at least one heading and that expected tables were detected. Fail loudly.
Non-empty checks — a parsed page with 20 characters means the OCR path did not run. Catch it before embedding.
Sampling review — human-check 2% of every batch. Confidence scores tell you which 2%.

What parsing costs

MarkdFlow charges one credit per processed page, which makes corpus cost predictable: a 5,000-page corpus costs 5,000 credits. The Annual plan (12,000 credits for $99/year) covers roughly 12,000 pages, and credit packs never expire if your volume is spiky.

Compare that to the engineering cost of owning the parsing layer: OCR vendor integration, table post-processing, handling of two-column layouts, and the on-call burden when a new document type breaks it.

Frequently asked questions

Is LLM document parsing the same as OCR?

OCR is one stage of it. OCR turns pixels into characters; parsing turns characters (from OCR or from the PDF text layer) into structured content with headings, tables, formulas and reading order.

Should I chunk before or after conversion?

After. Chunk raw extracted text and you slice mid-table and mid-section. Chunk structured Markdown and your chunks correspond to sections a reader would recognise.

How do I know if my parsing is good enough?

Build a 20-document golden set, score it by hand, and track the score over time. Per-block confidence scores let you automate most of the monitoring.

Can I parse documents programmatically?

Yes. The conversion endpoint is documented on the API page, and it is the recommended path for corpora larger than a handful of files.

What file types are supported?

PDF, Office documents (Word, PowerPoint, Excel), images (including scans), EPUB, notebooks and plain text or source files.

Related guides

Turn your document pile into a corpus

Parse PDFs, scans and Office files into structured Markdown with confidence scores. 30 credits free.

Start parsing API documentation