Transcribe, Polly and Lex

Three services cover the voice loop: Transcribe turns speech into text, Lex works out what the text means and what to do about it, and Polly turns the response back into speech.

Amazon Transcribe

Automatic speech recognition. Converts audio and video into text, in batch over stored files or in real time over a stream.

Common uses: call recording transcription, subtitling, meeting notes, and producing text for downstream analysis by Comprehend.

Amazon Polly

Text to speech, producing spoken audio from written input.

• Many languages and voices, including neural voices
• Speech Synthesis Markup Language (SSML) for control over pronunciation, emphasis, pauses and speaking style
• Output as an audio stream or a file in S3

Common uses: voice responses in contact centres, narration for e-learning and articles, accessibility features, IVR prompts.

Amazon Lex

Conversational AI for building chatbots and voice interfaces. Lex is the component that understands and drives the conversation: it recognises the caller’s intent, collects the slots (parameters) that intent needs, and decides what to say or do next. It is built on the same speech recognition and natural language understanding technology as Amazon Alexa.

Inputs and outputs

Lex accepts text or speech. It returns JSON containing:

1. Text response — the bot’s reply
2. Dialog state — ElicitIntent, ElicitSlot, ConfirmIntent, Fulfilled or Failed
3. Slot data — captured slot values and their validation results
4. Intent information — the identified intent, confidence scores and fulfilment state
5. Response cards — buttons, images and other formatted elements
6. Session attributes — context carried between turns

Worked examples

Input type	Example input	Intent	Slot values	Output type	Example output	Use case
Text	“Book a table for 4 tomorrow at 7 PM”	BookRestaurant	Date: tomorrow; Time: 7 PM; PartySize: 4	Text	“I’ve booked a table for 4 tomorrow at 7 PM. Your confirmation number is #12345.”	Restaurant reservation
Speech	“I want to order a large pepperoni pizza”	OrderPizza	Size: large; Topping: pepperoni	Text and card	Text: “Your pizza order total is £18.99”; card showing image, price and confirm button	Food ordering
Text	“What’s the balance in my checking account?”	CheckBalance	AccountType: checking	Text and session attributes	Text: “Your current account balance is £1,234.56”; session stores account type for follow-ups	Banking
Speech	“Book a flight to Seattle next Friday”	BookFlight	Destination: Seattle; Date: next Friday	ElicitSlot	“What time would you like to depart?”	Travel booking
Text	“I need to reschedule my appointment”	RescheduleAppointment	Action: reschedule	ElicitSlot and card	Text: “I can help you reschedule. Please select a new date”; card with calendar picker	Healthcare
Text	“Track my order #123456”	TrackOrder	OrderId: 123456	Text and response card	Text: “Your order is in transit”; card showing tracking map	E-commerce
Speech	“Turn on the living room lights”	SmartHomeControl	Device: lights; Location: living room; Action: on	Text and fulfilment	“Turning on the living room lights”	Smart home
Text	“What’s the weather like?”	GetWeather	None (eliciting)	ElicitSlot	“For which city would you like to know the weather?”	Weather
Text	“I want to file a complaint”	FileComplaint	None (initial)	Text and intent confirmation	“I understand you want to file a complaint. Is that correct?”	Customer service
Speech	“Transfer £100 to savings”	TransferMoney	Amount: 100; ToAccount: savings	ConfirmIntent	“Do you want to transfer £100 to your savings account?”	Banking transfer

Inputs and outputs of the AWS language and speech services.

A voice assistant pipeline

Speech arrives, is transcribed, is interpreted as an intent, is acted on, and the answer is spoken back. Comprehend is an optional branch for analysing the transcript — sentiment, entities, PII redaction — rather than a step in the main path.

graph TD A[Voice input] --> B[Amazon Transcribe] B --> C[Text transcript] C --> D[Amazon Lex] D --> E{Intent processing} E -->|Query| F[Content generation] E -->|Command| G[Action execution] F --> H[Amazon Polly] G --> H H --> I[Spoken response] C -.->|Optional text analytics| J[Amazon Comprehend] J -.->|Enriched text| D style B fill:#f9d5e5,stroke:#333,stroke-width:2px style D fill:#e06377,stroke:#333,stroke-width:2px style E fill:#b5e7a0,stroke:#333,stroke-width:2px style H fill:#f9d5e5,stroke:#333,stroke-width:2px style J fill:#eeac99,stroke:#333,stroke-width:2px

Where the transcript needs cleaning before Lex sees it — removing disfluencies, redacting PII, normalising domain terms — a Comprehend or Lambda step sits between the raw transcript and Lex, and the response can be published as text as well as spoken.