AWS Elemental Inference now supports Smart Subtitles for automated live captioning
Broadcasters can now auto-caption live streams in 6 languages with no third-party tools—and save more by bundling it with other AI video features.
View original announcement →Visual Summary
graph TD
A{{Smart Subtitles}}:::announced
B(AWS Elemental MediaLive):::compute
C(AWS Elemental Inference):::compute
D([Custom Dictionaries]):::feature
E([ASR Transcription]):::feature
F([TTML/WebVTT Output]):::feature
G(MediaPackage):::compute
H((Broadcaster)):::external
I([Smart Cropping]):::feature
H ==>|"configures"| B
B ==>|"enables"| A
A -->|"powered by"| C
C -->|"applies"| E
A -->|"generates"| F
D -.->|"improves accuracy"| E
F -->|"delivers to"| G
A -.->|"bundles with"| I
classDef announced fill:#ff9900,stroke:#ec7211,color:#fff,font-weight:bold
classDef compute fill:#e3f2fd,stroke:#1565c0,color:#1565c0
classDef storage fill:#e8f5e9,stroke:#2e7d32,color:#2e7d32
classDef feature fill:#fff3e0,stroke:#e65100,color:#e65100
classDef external fill:#f5f5f5,stroke:#616161,color:#616161
What's New
AWS Elemental Inference has added Smart Subtitles, an AI-powered feature that automatically generates real-time subtitles for live video streams using automatic speech recognition (ASR). The feature delivers subtitles in TTML or WebVTT format with low latency and integrates natively with AWS Elemental MediaLive, eliminating the need for manual captioning workflows or third-party services. It supports eight language variants across six languages and includes custom dictionary support for domain-specific terminology.
How It Works
- ASR-based transcription: Smart Subtitles applies automatic speech recognition to the audio track of live video streams, converting spoken content into timed subtitle text in real time.
- Dual output formats: Subtitles are delivered in TTML format for MediaPackage V2, CMAF Ingest, and Microsoft Smooth output groups, and in WebVTT format for HLS and MediaPackage output groups.
- Native MediaLive integration: The feature is enabled directly within an AWS Elemental MediaLive channel configuration via the console or API, requiring no external pipeline or third-party service.
- Custom dictionaries: Users can create custom dictionaries through the Elemental Inference API or console to improve transcription accuracy for specialized vocabulary such as athlete names, technical jargon, or brand terms.
- Multi-feature processing: Smart Subtitles shares underlying video/audio analysis with other Elemental Inference features (e.g., smart cropping, clip generation), enabling cost-efficient multi-feature deployments on the same stream.
- Non-linear pricing: When multiple features are enabled simultaneously, a bundled per-minute rate applies (e.g., $0.23/min for two features vs. $0.30/min if priced separately), reducing effective per-feature cost.
Why It's Important
- Accessibility compliance: Broadcasters can meet accessibility requirements (e.g., FCC captioning mandates, WCAG standards) for live content without building or procuring separate captioning infrastructure.
- Operational simplification: Eliminating manual captioning workflows and third-party integrations reduces operational complexity, vendor management overhead, and potential points of failure in live production pipelines.
- Broad language coverage: Support for English (US, GB, AU), French, German, Italian, Portuguese, and Spanish enables international broadcasters to serve multilingual audiences from a single integrated service.
- Low-latency delivery: Real-time subtitle generation with low latency ensures subtitles remain synchronized with live content, which is critical for sports, news, and event broadcasting.
- Cost efficiency at scale: The non-linear pricing model means organizations running multiple AI features simultaneously—such as vertical video cropping plus subtitles for an 8-hour sports day—pay significantly less than they would with separate point solutions.
How It's Different
- Integrated vs. bolt-on: Unlike third-party captioning services that require separate API integrations, webhook pipelines, or manual operator intervention, Smart Subtitles is natively embedded in the MediaLive channel workflow.
- Bundled AI pricing: Competing solutions typically charge independently per feature or per service; Elemental Inference's non-linear pricing reduces per-feature costs when combining Smart Subtitles with smart cropping or clip generation on the same content.
- Custom dictionary support: Many automated captioning services offer limited or no domain-specific vocabulary customization; Elemental Inference allows custom dictionaries via API or console for specialized content verticals.
- Multi-format output: The service outputs both TTML and WebVTT natively, covering the full range of common streaming delivery formats (HLS, CMAF, MediaPackage, Smooth Streaming) without additional transcoding steps.
- Unified ML platform: Smart Subtitles is part of a broader real-time ML video service (Elemental Inference) that also handles smart cropping and clip generation, allowing teams to consolidate AI video capabilities under a single service rather than managing multiple vendors.
When to Prefer It
- Live sports broadcasting: When commentary includes rapidly changing athlete names, team names, or play-by-play terminology that benefits from custom dictionaries to maintain transcription accuracy.
- News and current affairs: When real-time captioning is required for regulatory compliance and latency must be minimized to keep subtitles synchronized with breaking news coverage.
- Multilingual international streams: When content needs to be subtitled in French, German, Italian, Portuguese, or Spanish for regional audiences without deploying separate per-language captioning services.
- Multi-feature AI workflows: When a broadcaster is already using smart cropping for vertical video or clip generation, adding Smart Subtitles at the bundled rate delivers meaningful cost savings versus a standalone captioning solution.
- Organizations replacing third-party captioning vendors: When teams want to reduce vendor sprawl, simplify billing, and consolidate live AI video capabilities within the AWS ecosystem.
- OTT platforms with HLS or CMAF delivery: When the delivery stack uses HLS (WebVTT) or CMAF/MediaPackage V2 (TTML) output groups and subtitle format compatibility is a key requirement.
Availability
- General Availability: Smart Subtitles launched as a generally available feature on May 27, 2026.
- Supported AWS Regions: Available in US East (N. Virginia), US West (Oregon), Asia Pacific (Mumbai), and Europe (Ireland).
- Supported languages: English (United States, Great Britain, Australia), French, German, Italian, Portuguese, and Spanish.
- Pricing model: Consumption-based, pay-as-you-go; single-feature rate is $0.15/min per pipeline; two-feature bundled rate is $0.23/min per pipeline in US East (N. Virginia).
- Dual-pipeline note: When used with a Standard (dual-pipeline) MediaLive channel, costs are applied per pipeline, effectively doubling the per-minute charge for redundancy configurations.
- Integration requirement: Requires AWS Elemental MediaLive; Smart Subtitles is enabled through the MediaLive channel configuration and managed via the Elemental Inference API or console.
