What's New

Amazon Bedrock has expanded its AWS Service Quotas integration to include the bedrock-mantle endpoint, which serves the OpenAI Responses API, OpenAI Chat Completions API, and Anthropic Messages API. Customers can now view per-model input-tokens-per-minute and output-tokens-per-minute quotas for this endpoint directly in the AWS Service Quotas console, providing the same familiar visibility they already have for the bedrock-runtime endpoint. This enables teams to proactively monitor and plan capacity for production workloads running OpenAI- or Anthropic-compatible applications on Amazon Bedrock.

How It Works

• The bedrock-mantle endpoint (bedrock-mantle.<region>.api.aws) serves three compatible APIs — OpenAI Responses API, OpenAI Chat Completions API, and Anthropic Messages API — allowing existing application code to run on Bedrock with minimal changes.
• Two quota types are now visible per model, per Region in Service Quotas: input tokens per minute and output tokens per minute, both shared across all APIs served by the endpoint for a given model.
• When a request arrives, Bedrock first checks the input-tokens-per-minute quota using the request's input token count plus max_tokens (or the model maximum if unset); if the reservation would exceed the quota, an HTTP 429 is returned immediately.
• Output tokens are counted against the output-tokens-per-minute quota as the model generates them; if the quota is reached mid-generation, output stops and the response is returned with a finish reason indicating the cutoff.
• Any unused portion of the initial max_tokens reservation (i.e., max_tokens minus actual output tokens) is replenished to the quota after the response completes, preventing over-reservation from penalizing subsequent requests.
• Prompt-cached input tokens read through prompt caching do not count against the input-tokens-per-minute quota.
• The endpoint does not enforce requests-per-minute (RPM) quotas; throttling is governed solely by token-based quotas.
• Quota increases follow the standard Amazon Bedrock limit increase process via the Service Quotas console.

Why It's Important

• Teams migrating OpenAI or Anthropic workloads to Bedrock can now audit their throughput limits before going to production, reducing the risk of unexpected throttling at scale.
• A single, consistent interface (AWS Service Quotas console) covers both bedrock-runtime and bedrock-mantle endpoints, simplifying capacity management across an organization's entire Bedrock footprint.
• The token-reservation model (input + max_tokens checked upfront) gives operators a predictable, deterministic view of how quota is consumed, making capacity planning more accurate than RPM-only systems.
• Prompt-caching exemption for cached input tokens means high-cache-hit workloads can sustain higher effective throughput without consuming quota, rewarding well-architected applications.
• Visibility into per-model, per-Region quotas enables platform and FinOps teams to right-size quota increase requests before they become production incidents.

How It's Different

• Unlike the bedrock-runtime endpoint, bedrock-mantle quotas are token-only (no RPM limits), meaning bursty, large-context requests are throttled on token consumption rather than request count — a more granular and workload-appropriate model.
• The bedrock-mantle endpoint maintains a separate quota namespace from bedrock-runtime, so traffic on one endpoint does not consume or interfere with limits on the other, enabling independent scaling of native Bedrock and compatibility-layer workloads.
• The input-token reservation mechanism (reserving max_tokens upfront and replenishing unused tokens post-response) is unique to bedrock-mantle and prevents quota starvation from over-estimated max_tokens values over time.
• Cached tokens are quota-exempt on bedrock-mantle, a distinction not universally present in competing managed inference services, providing a cost and throughput advantage for prompt-caching-heavy workloads.
• The endpoint supports three distinct API surfaces (OpenAI Responses, OpenAI Chat Completions, Anthropic Messages) under a single quota umbrella, simplifying governance compared to managing separate endpoints per API style.

When to Prefer It

• Use bedrock-mantle quota visibility when migrating existing OpenAI or Anthropic SDK-based applications to Bedrock and needing to validate that default quotas are sufficient before cutover.
• Prefer this endpoint and its quota model when building multi-tenant SaaS platforms where predictable per-model throughput limits are required for tenant isolation and SLA guarantees.
• Ideal for FinOps and platform engineering teams that need a unified Service Quotas dashboard to audit and govern all Bedrock inference limits across accounts and Regions.
• Choose bedrock-mantle when your workload uses prompt caching heavily, since cached token reads are quota-exempt, allowing higher sustained throughput within the same quota tier.
• Use this endpoint when you need to run large-context inference jobs (e.g., long document processing) where token-based throttling is more predictable than RPM-based throttling for capacity planning.
• Appropriate for teams that want to proactively request quota increases ahead of planned traffic spikes (e.g., product launches) using the standard AWS Service Quotas increase workflow.

Availability

• Status: Generally Available (GA) in all Regions where the bedrock-mantle endpoint is offered.
• Supported Regions: US East (N. Virginia, Ohio), US West (Oregon), Asia Pacific (Mumbai, Tokyo, Sydney, Jakarta), Europe (Frankfurt, Ireland, London, Milan, Stockholm), and South America (São Paulo).
• Pricing: No additional charge for viewing or managing quotas via AWS Service Quotas; standard Amazon Bedrock inference pricing applies to model usage on the endpoint.
• Quota increase process: Follows the standard Amazon Bedrock limit increase workflow via the AWS Service Quotas console.
• Limitations: The endpoint may apply additional internal rate limiting not exposed in Service Quotas; AWS recommends retry logic with exponential backoff for transient throttling. bedrock-mantle quotas are independent of and do not share limits with bedrock-runtime quotas.
• How to access: Open the AWS Service Quotas console → select Amazon Bedrock → search for "Bedrock Mantle" to view current per-model quotas.

Related Resources

• https://docs.aws.amazon.com/bedrock/latest/userguide/quotas-mantle.html