Guide · Degradation

Building Degraded Document Datasets for OCR Robustness

Models trained on clean documents fail on real scans — faxes, photocopies, phone photos, thermal-paper receipts. But collecting genuinely degraded documents with accurate ground truth is nearly impossible: the worse the scan, the harder it is to label. Synthetic degradation inverts this: perfect labels on arbitrarily bad pages.

Realistic degradation is character-level, not blur

Naive augmentation (Gaussian noise, blur, JPEG artifacts) produces images that look degraded but don't reproduce how OCR actually fails. Real OCR errors are structured: rn→m, l→1, O→0, broken glyphs, merged characters. Degradation modeled on these substitution patterns produces training signal that transfers to real failure modes — a continuous 0–100% severity spectrum beats a handful of fixed presets, because it lets you sample the exact difficulty curve your curriculum needs.

The killer feature: clean/degraded twins

Seeded generation means running the same seed at quality 100 and quality 40 yields the same documents — identical content and layout, different corruption. That gives you:

• Exact before/after pairs for training denoising and restoration models.
• Controlled robustness curves: accuracy vs. degradation with content held constant, isolating the effect of noise from the effect of content.
• Perfect labels at any severity — ground truth comes from the pre-render document model, so a 90%-corrupted page is labeled as accurately as a clean one, with source-to-rendered token mappings recording what each mangled word was supposed to be.

The second axis: corrupt text layers

Visual degradation isn't the only way documents lie. A PDF can render a perfectly clean page while its embedded text layer is corrupted — the digital equivalent of a document that looks fine but was OCR'd badly upstream. Tools that trust embedded text extract garbage; tools that OCR the pixels are fine. Including corrupt-text-layer documents in your dataset tests a failure mode that visual augmentation can never reach. The two axes combine: clean-looking pages with bad text, degraded pages with intact text, and both at once.

Dataset design suggestions

• Sample degradation uniformly across the severity range rather than clustering at presets.
• Keep a clean twin for every degraded document (same seed, quality 100).
• Cover text-dense and layout-dense types — degradation interacts differently with tables than with prose.
• Record exact corruption settings per document (DocSet Generator writes them to manifest.jsonl) so failures are reproducible.

Frequently asked questions

How is this different from image augmentation libraries like Augraphy?

Image augmentation operates on pictures of documents after the fact; it can't give you the intended text of what it corrupted beyond what you already had. Generation-time degradation knows the ground truth before corrupting, records source-to-rendered mappings, and can also corrupt the embedded PDF text layer — which no image-space tool can do.

Can I reproduce an exact degraded dataset later?

Yes — seeded generation is deterministic (byte-identical PDFs on the same platform and dependency versions), and a replayable dataset_recipe.json captures the full run configuration.