NVIDIA · November 2020
A100
80GB SXM
The NVIDIA A100 80GB SXM is a high-performance GPU designed for data centers, targeting AI, machine learning, and high-performance computing workloads. It is part of the Ampere architecture, offering significant improvements in memory capacity and bandwidth over its predecessors. The 80GB variant provides enhanced memory for large-scale models and datasets, making it ideal for demanding applications.
Provider Marketplace
All Cloud Providers
Estimates only — rates are collected automatically from public provider pages and may be out of date. Prices vary by region, commitment term, and availability, and typically exclude storage, egress, and tax. Confirm current pricing with the provider before purchasing. Last collected .
Compute Performance
Architecture
Memory & VRAM
Connectivity & Scaling
Virtualization
Power & Efficiency
Physical Design
Thermals & Cooling
Software Ecosystem
Server & Deployment
System Compatibility
Benchmarks & Throughput
Structured Sparsity
Supported (up to 2X vs dense)
Training Benchmarks
A100 80GB reaches up to 1.3 TB of unified memory per node and delivers up to a 3X throughput increase over A100 40GB (DLRM).
Inference Benchmarks
On state-of-the-art conversational AI models like BERT, A100 accelerates inference throughput up to 249X over CPUs.
Scaling Efficiency
NVLink/NVSwitch/InfiniBand scaling: possible to scale to thousands of A100 GPUs; NVLink/NVSwitch interconnect up to 600 GB/s.
Multi-GPU Scalability
Scaling Characteristics
Workload Readiness
LLM Training
312 TFLOPS (TF32 peak for A100 80GB SXM)
LLM Inference
1248 TOPS (INT8 Tensor Core peak for A100 80GB SXM with sparsity)
Vision Training
624 TFLOPS (FP16 Tensor Core peak for A100 80GB SXM with sparsity)
HPC / Simulation
9.7 TFLOPS (FP64 peak for NVIDIA A100 for NVLink / SXM)
Scientific Computing
19.5 TFLOPS (FP64 Tensor Core peak)
Real-Time Serving
Up to 7 MIGs @ 10GB (Multi-Instance GPU for A100 80GB SXM)
Market Authority
Key Strengths
This GPU excels at AI training and inference, offering exceptional performance for deep learning frameworks. Its large memory capacity and high bandwidth make it particularly effective for large-scale models and data-intensive tasks. The A100's support for multi-instance GPU (MIG) technology allows for efficient resource partitioning, enhancing its versatility.
Limitations
While the A100 80GB SXM offers exceptional performance, its high power consumption and cooling requirements may limit its use to well-equipped data centers. The SXM form factor restricts compatibility to specific platforms, and its premium pricing can be a barrier for smaller organizations. Availability may also be constrained by high demand and production limitations.
Also in the Lineup
Expert Insight
The A100 represents a strategic leap in AI compute. When comparing cloud providers, consider not just the hourly rate, but also the interconnect bandwidth (InfiniBand/NVLink) and regional availability which can significantly impact total cost of ownership for large-scale training.