NVIDIA · 2022-03-27
H100
SXM
The NVIDIA H100 SXM variant features exceptional performance and scalability for a wide range of workloads. It includes fourth-generation Tensor Cores and a Transformer Engine with FP8 precision, providing up to 4X faster training over the prior generation for large language models.
Provider Marketplace
All Cloud Providers
Estimates only — rates are collected automatically from public provider pages and may be out of date. Prices vary by region, commitment term, and availability, and typically exclude storage, egress, and tax. Confirm current pricing with the provider before purchasing. Last collected .
Compute Performance
Architecture
Memory & VRAM
Connectivity & Scaling
Virtualization
Power & Efficiency
Physical Design
Thermals & Cooling
Server & Deployment
System Compatibility
Benchmarks & Throughput
Structured Sparsity
With sparsity
Scaling Efficiency
NVIDIA NVLink: 900GB/s
Multi-GPU Scalability
Scaling Characteristics
Workload Readiness
LLM Training
3,958 teraFLOPS
LLM Inference
3,958 TOPS
Vision Training
1,979 teraFLOPS
Diffusion Models
1,979 teraFLOPS
Multimodal AI
80GB
Reinforcement Learning
989 teraFLOPS
HPC / Simulation
34 teraFLOPS
Scientific Computing
3.35TB/s
Real-Time Serving
3,958 TOPS
Market Authority
Key Strengths
The H100 SXM excels in AI and machine learning workloads, particularly in training large neural networks and performing inference at scale. It offers significant performance improvements over its predecessors due to its advanced architecture and increased memory bandwidth. The H100 is also well-suited for high-performance computing (HPC) applications, providing exceptional computational power and efficiency.
Limitations
One limitation of the H100 SXM is its high power consumption, which may not be suitable for all datacenter environments. Additionally, its reliance on specific server platforms and cooling solutions can limit deployment flexibility. Availability can be constrained due to high demand and production capacities, potentially leading to longer lead times for procurement.
Also in the Lineup
Expert Insight
The H100 represents a strategic leap in AI compute. When comparing cloud providers, consider not just the hourly rate, but also the interconnect bandwidth (InfiniBand/NVLink) and regional availability which can significantly impact total cost of ownership for large-scale training.