Worked Example: Document Classification

Consider a document classification system: incoming documents of mixed types must be sorted into categories so they can be routed to the right process.

The division of labour

AWS Glue handles data preparation and transformation: extracting text and metadata from the raw documents, normalising formats, deduplicating, cataloguing what exists and writing the result to S3 in a consistent shape. Where the source documents are scans or PDFs rather than text, Amazon Textract does the extraction and Glue does the shaping around it.

Labelling produces the ground-truth categories that supervised training needs. This is the step to plan carefully, because the AWS service that used to own it is closed — see below.

Amazon SageMaker AI trains on the combined processed and labelled dataset, and hosts the resulting model.

The distinction to hold on to is that Glue prepares data and knows nothing about the labels, while the labelling step produces the target variable and knows nothing about the transformation. They meet only in the training manifest.

The pipeline

1. Raw documents land in S3.
2. Textract extracts text, forms and tables from scanned documents and PDFs.
3. Glue catalogues the extracted output, cleans it and writes a normalised dataset back to S3.
4. Documents are labelled with their category, and the labels are written to S3 as a manifest alongside the data.
5. SageMaker AI trains a classifier on the labelled dataset.
6. The model is registered, deployed to an endpoint, and called by whatever routes the documents.

Labelling, now that Ground Truth is closed

Amazon SageMaker Ground Truth was the AWS service for this step, coordinating a private, vendor or Mechanical Turk workforce and reducing effort through automated labelling. It is no longer open to new customers — AWS moved it to maintenance on 30 July 2026 — and Ground Truth Plus reached end of support on 30 June 2026. Existing customers can continue to use it, but a new build cannot start there. Amazon Mechanical Turk itself closes on 30 September 2026, which removes that workforce option entirely.

Practical options for a new pipeline:

• A third-party labelling platform, writing its output to S3 in the manifest format the training job expects. This is the closest replacement for Ground Truth Plus.
• An internal labelling tool for a private workforce, where the documents are too sensitive to send outside the organisation. The output is again a manifest in S3.
• Existing business labels. Documents that already passed through a manual process frequently carry a category assigned by whoever handled them. This is often the cheapest labelled dataset available and it is usually already in a database rather than needing a labelling exercise at all.
• Weak supervision — labelling functions or heuristics that produce noisy labels at volume — to bootstrap a first model, refined by human review of the cases it is least confident about.

An alternative without training a model

For text document classification specifically, Amazon Comprehend custom classification trains a classifier from labelled examples using AutoML, with no model architecture work and no endpoint to size unless real-time inference is needed. It still requires labelled data, but it removes most of the training and hosting effort. Reach for SageMaker AI when the document types or the required accuracy exceed what a managed classifier delivers, or when the model needs to be yours.