What's New

AWS has expanded availability of Amazon EC2 P5.48xlarge instances on SageMaker Notebook Instances to the Asia Pacific (Tokyo) region. These instances are powered by NVIDIA H100 Tensor Core GPUs and are designed for demanding deep learning and high-performance computing workloads. This expansion brings cutting-edge GPU compute directly into the SageMaker notebook environment for customers in the Tokyo region.

How It Works

• P5.48xlarge instances are backed by 8 NVIDIA H100 Tensor Core GPUs, delivering up to 20 exaflops of aggregate compute when deployed in EC2 UltraClusters interconnected via a petabit-scale nonblocking network.
• The instances complement H100 GPUs with 2x higher CPU performance, 2x higher system memory, and 4x higher local storage compared to previous-generation GPU-based EC2 instances.
• Networking is supported at up to 3,200 Gbps using second-generation Elastic Fabric Adapter (EFA), enabling high-throughput distributed training workloads.
• Within SageMaker Notebook Instances, the P5.48xl runs a Jupyter server on EC2, pre-configured with the SageMaker Python SDK, Boto3, AWS CLI, deep learning framework libraries (PyTorch, TensorFlow, etc.), and Conda environments.
• Users can interact with these instances through JupyterLab or Code Editor (VS Code-based) applications within SageMaker Studio, or via classic SageMaker Notebook Instances.
• The instances are deployed in Amazon EC2 UltraClusters, enabling scaling to up to 20,000 H100 GPUs for large-scale distributed training scenarios.

Why It's Important

• Tokyo-region customers can now access H100-class GPU compute directly within SageMaker Notebook Instances without needing to route workloads to distant regions, reducing latency and simplifying data residency compliance.
• The P5.48xl delivers up to 4x faster time-to-solution compared to previous-generation GPU instances, dramatically shortening iteration cycles for LLM fine-tuning, diffusion model training, and HPC workloads.
• Training cost reductions of up to 40% versus prior-generation GPU instances make large-scale generative AI experimentation more economically viable for organizations in the Asia Pacific region.
• Availability within SageMaker Notebook Instances lowers the barrier to entry—researchers and data scientists can prototype and run large-scale GPU workloads interactively without managing raw EC2 infrastructure.
• The expansion supports a broad range of high-value generative AI use cases including LLM training, question answering, code generation, image/video generation, and speech recognition directly from a managed notebook environment.

How It's Different

• Unlike previous-generation P4 (A100-based) instances, P5.48xl uses NVIDIA H100 GPUs with NVLink and NVSwitch, offering significantly higher GPU-to-GPU bandwidth for multi-GPU training jobs.
• P5 instances provide 2x the CPU performance and system memory, and 4x the local NVMe storage compared to P4d instances, reducing I/O bottlenecks during large dataset training.
• The second-generation EFA at 3,200 Gbps on P5 instances provides substantially higher network bandwidth than prior-generation instances, enabling faster all-reduce operations in distributed training.
• Compared to using raw EC2 P5 instances, SageMaker Notebook Instances add managed Jupyter infrastructure, pre-installed ML libraries, built-in SageMaker SDK integration, and automatic software maintenance—reducing operational overhead.
• The P5 family (H100) is distinct from the newer P5e/P5en families (H200), which offer even higher memory bandwidth; P5.48xl remains the established, broadly available option for H100-based workloads.

When to Prefer It

• Choose P5.48xlarge on SageMaker Notebook Instances when interactively developing, fine-tuning, or experimenting with large language models (e.g., 7B–70B+ parameter models) that require H100-class GPU memory and compute.
• Ideal for teams in the Asia Pacific (Tokyo) region that need to keep training data and model artifacts within Japan for data sovereignty or regulatory compliance reasons.
• Use when running diffusion model training or inference (e.g., image/video generation) that benefits from H100's Transformer Engine and FP8 precision support.
• Well-suited for HPC workloads such as pharmaceutical discovery, seismic analysis, weather forecasting, or financial modeling that require high-throughput GPU compute within a managed notebook environment.
• Prefer this option when your team wants the productivity of a managed Jupyter/VS Code environment (SageMaker Studio) combined with top-tier GPU hardware, without managing EC2 instance lifecycle manually.
• Appropriate for distributed training prototyping where you want to validate multi-GPU or multi-node training scripts interactively before scaling to full SageMaker Training Jobs.

Availability

• Status: Generally Available (GA) as of May 27, 2026.
• Newly added region: Asia Pacific (Tokyo) — ap-northeast-1.
• Access surface: Available on SageMaker Notebook Instances and compatible with SageMaker Studio applications (JupyterLab and Code Editor).
• Pricing: Charged at standard Amazon EC2 P5.48xlarge on-demand or reserved instance rates plus SageMaker Notebook Instance overhead; refer to the Amazon SageMaker Pricing page for current rates in ap-northeast-1.
• Limitations: P5.48xl instances are large, expensive instances subject to EC2 capacity availability; users may need to request service quota increases to access them in new regions.
• Other regions: P5.48xl was previously available in select regions (e.g., us-east-1, us-west-2) on SageMaker; Tokyo represents a regional expansion, not a global launch.

Related Resources

• https://aws.amazon.com/ec2/instance-types/p5/
• https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated-jl.html
• https://docs.aws.amazon.com/sagemaker/latest/dg/code-editor.html
• https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated.html
• https://docs.aws.amazon.com/sagemaker/latest/dg/nbi.html