HPC on AWS: Innovation without infrastructure constraints
summary
AWS shows specific HPC stacks: EFA-enabled C5n delivers near-linear scaling and >75% parallel efficiency up to ~1,200 cores versus large degradation without EFA, while P4d ultra-clusters provide 8x A100 GPUs and 400 Gbps fabric for multi‑thousand‑GPU training. Storage using FSx for Lustre scales throughput (example: ~2 GB/s at small size to ~100 GB/s at hundreds of TB). Orchestration uses ParallelCluster and AWS Batch (Mobileye runs 200–300k cores daily). Formula 1 drops 60+ hour CFD runs to under 10 hours by moving to AWS.
AI-written summary · about
chapters (17)
- 0:00Intro
- 1:12HPC impacts your life every day
- 2:55Matching demand to capacity is challenging
- 3:36We think the metric for success for any business should be time to results
- 7:00Virtually any type of HPC workload is possible on AWS
- 10:11Key services that enable HPC on AWS
- 13:16AWS ParallelCluster architecture
- 15:02Amazon EC2
- 16:54Bare metal-like performance Nitro Gen 4
- 17:43Broadest and deepest platform choice
- 19:06EC2 UltraClusters of P4d: Supercomputing-class performance for deep learning workflows
- 24:42Amazon FSx for Lustre
- 25:09Powered by Lustre - Amazon FSx performance scales with storage capacity
- 27:17How AWS Batch with Fargate works
- 28:22HPC workloads across industries
- 29:02Customer case study: Formula 1 on AWS
- 33:08AWS Training and Certification
related talks
33mHPC on AWS: Innovation Without Infrastructure Constraints
38mHigh Performance Computing on AWS (HPC): Innovation Without Limits
39mWhat would you do with 10^6 cores? High Performance Computing in the AWS Cloud
47mPowering global-scale predictive intelligence using HPC on
35mAccelerate your ML innovation with AWS ML infrastructure services
46mHPC to Accelerate Innovation in Industry
10mLightning Talk: Does Cloud Elasticity Pay off for HPC Workloads?
43mWhat’s new in Amazon EC2