Guide · Annotation Formats

COCO Annotations for Document Text Detection

COCO is the lingua franca of object detection — and for document AI, 'objects' are words. If you're training a text detector (the first stage of any OCR pipeline), you need COCO-format word boxes over page images. Here's the format, the document-specific gotchas, and how to generate it.

COCO applied to documents

A COCO dataset is one JSON file with three arrays: images, annotations, and categories. For text detection, each word on each page becomes one annotation:

{
  "images": [{"id": 1, "file_name": "letter_0001_p1.png",
              "width": 1700, "height": 2200}],
  "annotations": [{"id": 501, "image_id": 1, "category_id": 1,
                   "bbox": [84, 109, 68, 21], "area": 1428,
                   "iscrowd": 0}],
  "categories": [{"id": 1, "name": "word"}]
}

Two conventions bite people constantly: COCO's bbox is [x, y, width, height] (not [x0,y0,x1,y1]), and coordinates are pixels in the rendered image — not PDF points. If your generator computes boxes in PDF space and your training images render at a different DPI, every box is scaled wrong and detection training silently degrades.

Document-specific requirements

• Density: a text-heavy page has 300–800 word boxes — orders of magnitude more objects per image than natural-scene COCO. Verify your training pipeline's per-image annotation limits.
• Degradation-aligned boxes: if you train detection on degraded scans, boxes must align with the rendered (possibly corrupted) glyphs, and you still want the intended text recorded for the recognition stage.
• Small-object handling: footnotes, superscripts, and dense tables produce tiny boxes; anchor/stride settings tuned for natural images often miss them.

Generating it synthetically

DocSet Generator's COCO export writes word-detection boxes in rendered-image pixel coordinates (box scaling matched to the emitted page images), across 33 document types, with deterministic train/val/test splits. Export reference loaders re-read and validate every export, so a malformed JSON never reaches your trainer. Pair the COCO export with the LayoutLM or FUNSD exports from the same run to train detection and understanding stages on identical documents.

Frequently asked questions

Word-level or line-level boxes for text detection?

Word-level is the common choice for OCR pipelines (recognition models consume word crops). DocSet Generator records both: COCO word boxes plus explicit line and layout-block geometry in its annotation schema if you train at line level.

Can I train with Detectron2 / MMDetection / YOLO on this?

Yes — all three consume standard COCO JSON directly (YOLO after format conversion). The export is standard COCO with a single 'word' category.