SageMaker Notebook Instances now support P5en.48xl instance types
H200 GPUs with 1.7× more memory and 35% lower network latency are now available in SageMaker notebooks for large-scale AI prototyping.
View original announcement →Visual Summary
graph TD
A{{P5en.48xl on SageMaker Notebooks}}:::announced
B((ML Practitioner)):::external
C(SageMaker Studio):::compute
D([8x NVIDIA H200 GPUs]):::feature
E([Gen5 PCIe 4x Bandwidth]):::feature
F([3200 Gbps EFA Networking]):::feature
G(JupyterLab IDE):::compute
H(Code Editor IDE):::compute
I([Distributed Training]):::feature
J(Amazon S3):::storage
B ==>|"launches"| C
C ==>|"provisions"| A
A -->|"provides"| D
A -->|"enables"| E
A -->|"connects via"| F
C -->|"opens"| G
C -->|"opens"| H
D -->|"accelerates"| I
F -->|"35% lower latency"| I
I -.->|"stores artifacts"| J
classDef announced fill:#ff9900,stroke:#ec7211,color:#fff,font-weight:bold
classDef compute fill:#e3f2fd,stroke:#1565c0,color:#1565c0
classDef storage fill:#e8f5e9,stroke:#2e7d32,color:#2e7d32
classDef feature fill:#fff3e0,stroke:#e65100,color:#e65100
classDef external fill:#f5f5f5,stroke:#616161,color:#616161
What's New
Amazon SageMaker Notebook Instances now support the EC2 P5en.48xlarge instance type, bringing NVIDIA H200 GPU-powered compute directly into the SageMaker notebook environment. The P5en.48xl instance features 8 H200 GPUs, third-generation EFA networking at up to 3,200 Gbps via Nitro v5, and Gen5 PCIe connectivity between CPU and GPU. This makes it possible to prototype, experiment, and iterate on large-scale deep learning, generative AI, and HPC workloads from a fully managed Jupyter notebook interface.
How It Works
- SageMaker Notebook Instances provision a managed EC2 instance running a Jupyter server; selecting the P5en.48xl instance type allocates a full P5en host with 8 NVIDIA H200 Tensor Core GPUs directly accessible from notebook kernels.
- The H200 GPUs provide 141 GB of HBM3e memory per GPU (approximately 1.7× the memory of H100), enabling larger model weights and activation tensors to reside entirely on-device.
- Gen5 PCIe interconnect between the 4th-generation Intel Xeon Scalable (Sapphire Rapids) CPU and the GPUs delivers up to 4× the CPU-GPU bandwidth compared to P5 instances, reducing data-staging bottlenecks during training and inference.
- Third-generation Elastic Fabric Adapter (EFA) running over Nitro v5 provides up to 3,200 Gbps of network bandwidth and up to 35% lower collective-communication latency versus the EFA generation used in P5 instances.
- The notebook environment comes pre-configured with the SageMaker Python SDK, Boto3, deep learning framework libraries (PyTorch, TensorFlow, etc.), and Conda, so GPU workloads can be launched immediately without manual environment setup.
- Users can access the instance through JupyterLab or Code Editor (VS Code–based) IDEs within SageMaker Studio, or through the classic SageMaker Notebook Instances interface.
Why It's Important
- Researchers and engineers can now interactively prototype and debug distributed training jobs on the same class of hardware they will use in production, eliminating the "works on small GPU, breaks on cluster" gap.
- The 1.7× increase in GPU memory per device (H200 vs. H100) allows larger language models and diffusion models to be loaded and fine-tuned without aggressive quantization or model sharding during the exploration phase.
- The 35% latency improvement in collective communications (AllReduce, AllGather) directly accelerates multi-GPU training loops run from a notebook, shortening iteration cycles for distributed experiments.
- Having H200-class compute available in a managed notebook removes the operational overhead of standing up and tearing down EC2 clusters for exploratory work, lowering the barrier to experimenting with frontier-scale models.
- The 4× CPU-GPU bandwidth improvement benefits data-preprocessing pipelines that feed GPUs, reducing GPU idle time caused by CPU-side bottlenecks in data-intensive workflows.
How It's Different
- H200 vs. H100 GPUs: P5en uses NVIDIA H200 GPUs with 1.7× more GPU memory and 1.4× higher memory bandwidth than the H100 GPUs in P5 instances, enabling larger models and faster memory-bound operations.
- Gen5 PCIe vs. Gen4 PCIe: P5en's Gen5 PCIe delivers up to 4× the CPU-GPU bandwidth of P5/P5e instances, significantly reducing host-to-device transfer latency for large tensor operations.
- Third-gen EFA + Nitro v5 vs. second-gen EFA: P5en achieves up to 35% lower network latency for collective communications compared to P5, which uses an older EFA and Nitro generation.
- Intel Sapphire Rapids CPU vs. prior CPU generations: P5en pairs H200 GPUs with high-performance Intel Xeon Scalable (Sapphire Rapids) processors, whereas P5/P5e use different CPU configurations, giving P5en a more balanced CPU-GPU compute profile.
- Notebook-native access: Unlike running P5en via SageMaker Training Jobs or EC2 directly, the notebook instance integration provides an interactive, persistent Jupyter environment with pre-installed ML libraries and no cluster orchestration required.
When to Prefer It
- Large model prototyping: When you need to load and experiment with LLMs or diffusion models that exceed the ~80 GB HBM capacity of H100 GPUs, the H200's larger memory pool allows full-precision or lightly quantized models to fit on fewer devices.
- Distributed training experiments: When iterating on multi-GPU training strategies (data parallelism, tensor parallelism, pipeline parallelism) and needing fast collective communication feedback without provisioning a full training cluster.
- Memory-bandwidth-bound workloads: When running inference or fine-tuning tasks where GPU memory bandwidth is the primary bottleneck (e.g., autoregressive decoding of large transformers), the H200's 1.4× bandwidth advantage over H100 yields measurable throughput gains.
- HPC and scientific computing: When performing seismic analysis, molecular dynamics, weather forecasting, or financial simulations that require both high GPU throughput and low-latency inter-node communication.
- Interactive data preprocessing at scale: When CPU-to-GPU data pipelines are a bottleneck, the Gen5 PCIe bandwidth improvement makes P5en preferable for workflows that continuously stream large datasets from host memory to GPU.
- Cost-sensitive large-scale exploration: When the goal is to validate that a model or algorithm works at scale before committing to a long-running SageMaker Training Job, using a notebook instance for short interactive sessions can be more economical than running full training jobs repeatedly.
Availability
- GA Status: Generally available as of May 27, 2026.
- Supported Regions: US East (N. Virginia), US East (Ohio), US West (Oregon), and Asia Pacific (Tokyo).
- Instance Type: ml.p5en.48xlarge (SageMaker-prefixed naming) with 8 NVIDIA H200 GPUs, available through the SageMaker Notebook Instances console or API.
- IDE Support: Accessible via SageMaker Notebook Instances (classic Jupyter), SageMaker Studio JupyterLab spaces, and SageMaker Studio Code Editor spaces.
- Pricing: Billed at standard SageMaker Notebook Instance on-demand rates for the ml.p5en.48xlarge instance type; consult the SageMaker Pricing page for current per-hour costs.
- Limitations: Availability is currently limited to four regions; customers in other regions (e.g., EU, other APAC) must wait for future regional expansion or use cross-region options.
Related Resources
- https://aws.amazon.com/ec2/instance-types/p5/
- https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated-jl.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/code-editor.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/studio-updated.html
- https://docs.aws.amazon.com/sagemaker/latest/dg/nbi.html
