Guide · Invoices & Receipts

Generating a Synthetic Invoice Dataset for OCR and Extraction Testing

Invoice extraction is the most common document-AI workload, and the hardest to test honestly: public invoice datasets are small, clean, and overfit-to. Here's how to generate unlimited realistic invoices — with line items that actually add up — for training and stress-testing extraction pipelines.

The problem with public invoice datasets

Most teams test invoice extraction on the same handful of public sets (SROIE, a few Kaggle corpora, ~200-form FUNSD). Vendors have seen them; models have memorized them; and none of them look like your invoice mix. Worse, real invoice datasets can't be shared with third-party OCR APIs without PII exposure — you can't legally upload a customer's invoice archive to a cloud OCR service just to benchmark it.

What makes a synthetic invoice realistic

• Internally consistent math. Line items × quantities must sum to subtotals; tax and totals must reconcile. Extraction validators check this — fake invoices with random numbers fail immediately.
• Structural variety. Varied line-item counts, multi-page invoices, different table layouts, headers, remit-to blocks, and payment terms. A single template teaches your model one invoice.
• Companion document types. Real AP workflows also see receipts, checks, financial statements, and purchase-order-style correspondence. Test the pipeline, not just one document class.
• Fabricated but plausible entities. Company names, addresses, EINs, and account numbers that look real but are structurally guaranteed fake (reserved phone ranges, invalid SSN patterns, .example email domains).

Degradation: where extraction actually fails

Extraction accuracy on clean PDFs is a vanity metric. Real invoices arrive as faxed scans, phone photos, and third-generation photocopies. Two adversarial modes matter:

• Visual degradation (0–100%): character substitution and corruption modeled on real OCR failure modes — tests the OCR layer itself.
• Corrupt text layer: the page looks clean, but the embedded PDF text is garbage. Tools that shortcut by reading embedded text (many do) silently extract nonsense. This single test exposes more extraction bugs than any other.

Ground truth for scoring

Every generated invoice should ship with its intended values — vendor, dates, line items, totals — in a machine-readable manifest, so you can score extraction output automatically instead of eyeballing PDFs. DocSet Generator writes a manifest.jsonl plus per-document ground-truth sidecars and key/value pair annotations for exactly this.

Frequently asked questions

Can I upload synthetic invoices to AWS Textract or Azure Document Intelligence?

Yes — that's a primary use case. Because every entity is fabricated, synthetic invoices can be sent to any third-party OCR service with zero PII exposure or compliance review.

Do the invoice totals actually reconcile?

In DocSet Generator, yes: line items, subtotals, tax, and totals are internally consistent, so extraction validators and business-rule checks behave as they would on real documents.

Is there a free way to try this?

The live browser demo generates watermarked synthetic invoices (with an OCR degradation slider) client-side with no signup, and the free sample pack includes clean and degraded invoice examples.