Skip to main content
Back

IBM Research's Heron Boosts Open-Source PDF Parsing Accuracy by 23.5%

IBM Research ships Heron, a compact RT-DETRv2 layout detector that jumps 23.5% in mAP and now powers Docling's PDF pipeline by default.

  • Heron is Docling's new default layout model, built on RT-DETRv2 by IBM Research (HF)
  • Delivers 23.5% mAP gain over the previous Docling baseline at comparable runtime
  • Top variant heron-101 hits 78% mAP at 28 ms/image on A100 (paper)
  • 42.9M parameters, Apache 2.0, over 1.8M downloads, drops into HF Transformers as RTDetrV2
  • Detects 17 classes including tables, formulas, code, forms, and checkboxes
  • Trained on 150,000 mixed public and proprietary documents with post-processing tuned for conversion

Docling, the open-source document conversion toolkit from IBM Research, just swapped its default layout analysis brain. The new model, Heron, is already the workhorse behind millions of PDF-to-Markdown conversions feeding RAG pipelines. It posts a 23.5% gain in mAP (mean Average Precision) over the previous Docling baseline, the largest single accuracy jump the project has shipped since launch, and it gets there without giving up runtime.

What Heron actually is

Heron is an object detector trained to find and classify the visual regions on a document page: titles, paragraphs, figures, tables, formulas, footnotes, form fields, and so on. Built by IBM Research on top of RT-DETRv2 (Real-Time DEtection TRansformer), a transformer-based detector designed for real-time use, it avoids the hand-tuned anchor boxes that older detectors like YOLO rely on.

The checkpoint on Hugging Face weighs in at 42.9M parameters, ships under Apache 2.0, and has already crossed 1.8M downloads. It predicts 17 layout classes, which is worth listing because it tells you exactly what the model can carve out of a page:

  • Text blocks: Title, Section-header, Text, List-item, Caption, Footnote