NVIDIA H100 SXM
NVIDIA: specifications, comparisons and alternatives.
The NVIDIA H100 SXM is a data centre GPU from the Hopper generation, designed for AI training, inference and HPC, among other things. According to the manufacturer, it has 80 GB of HBM3 memory, 3.35 TB/s of memory bandwidth and a TDP of up to 700 W. The SXM version is connected via NVLink at 900 GB/s. That way you quickly learn about the most important characteristics.
NVIDIA
- 80 GB
- Memory
- HBM3
- Memory type
- 3,35 TB/s
- Memory bandwidth
- 700 W
- Power (board power or TDP)
You can see the current price and availability with the partner as soon as a link has been set up.
How are the points calculated?
Clouwino points are an arithmetic summary of the specifications the manufacturer states, on a scale of 0 to 100. It is not a benchmark and not a judgement on quality or suitability: a product with fewer points may suit your situation better. Products are compared with other products of the same type. For each property we count how far a product sits between the lowest and highest value in the group:
- FP32 compute (for: consumer), carries heavy weight
- Memory (for: consumer), carries heavy weight
- Memory bandwidth (for: consumer), carries heavy weight
- Compute units (for: consumer), carries light weight
- Boost clock (for: consumer), carries light weight
- Memory (for: professional and data centre), carries medium weight
- Memory bandwidth (for: professional and data centre), carries medium weight
- FP16/BF16 (for: professional and data centre), carries heavy weight
- FP8 (for: professional and data centre), carries medium weight
- FP64 (for: professional and data centre), carries medium weight
If too much data is missing, we do not show points. Points are relative: when a new product is added, they can shift. Read more at how we compare.
Specifications of NVIDIA H100 SXM
All specifications the manufacturer lists, per item. Want to see another product next to it? Pick one below.
| Specificatie | NVIDIA H100 SXM |
|---|---|
| Overview | |
| Clouwino points | 40 punten |
| General | |
| Architecture | Hopper |
| Manufacturing process | TSMC 4N |
| Form | SXM |
| Launch | 2022-03 |
| Compute | |
| FP32 compute | 67 TFLOPS |
| FP64 | 34 TFLOPS |
| FP16/BF16 | 1.979 TFLOPS |
| FP8 | 3.958 TFLOPS |
| FP4 | Not stated by the manufacturer |
| Performance notes | FP64 34 TFLOPS (vector; FP64 Tensor Core 67 TFLOPS) and FP32 67 TFLOPS: no sparsity mention. FP16/BF16 1,979 TFLOPS and FP8 3,958 TFLOPS: WITH SPARSITY (NVIDIA footnote '* With sparsity'); dense values are not on the product page and have not been converted. FP4 not supported/stated. Also TF32 989 TFLOPS and INT8 3,958 TOPS with sparsity. |
| Memory | |
| Memory | 80 GB |
| Memory type | HBM3 |
| Memory bandwidth | 3,35 TB/s |
| Power and connections | |
| Power (board power or TDP) | 700 W |
| GPU interconnect | NVLink 900 GB/s; PCIe Gen5 128 GB/s |
| Multiple instances | MIG: up to 7 instances of 10 GB |
Note: Figures per GPU (SXM column). TDP up to 700 W (configurable). Memory type, process node and launch date (announced 22 March 2022; available from Q3 2022) come from the NVIDIA press release, not the specification table. H100 NVL (PCIe) is a separate variant: 94 GB, 3.9 TB/s, 350-400 W.
Alternatives to NVIDIA H100 SXM
Products with similar specifications from the same overview. This is not a ranking and not buying advice.
- AMD Instinct MI250X128 GB · HBM2e · 3,2 TB/sCompare with NVIDIA H100 SXM →
- NVIDIA H200 SXM141 GB · HBM3e · 4,8 TB/sCompare with NVIDIA H100 SXM →
- Intel Gaudi 3128 GB · HBM2e · 3,7 TB/sCompare with NVIDIA H100 SXM →
- AMD Instinct MI300X192 GB · HBM3 · 5,3 TB/sCompare with NVIDIA H100 SXM →
- AMD Instinct MI325X256 GB · HBM3E · 6 TB/sCompare with NVIDIA H100 SXM →
- AMD Instinct MI21064 GB · HBM2e · 1,6 TB/sCompare with NVIDIA H100 SXM →
What the H100 SXM is designed for
The H100 SXM is an accelerator for server environments where large AI models are trained or run, and for scientific computing tasks. NVIDIA announced the Hopper architecture in March 2022, with availability from the third quarter of that year. The chip is made on the TSMC 4N process.
The product comes as an SXM module: you do not place it as a separate card in a standard PCIe slot, but on a server board made for the purpose. The H100 NVL with a PCIe connection is a separate variant with 94 GB of memory and a bandwidth of 3.9 TB/s. The H100 SXM also supports MIG, which lets you divide one GPU into up to 7 instances of 10 GB.
Memory, power and compute precision in plain language
The memory determines how large a model or dataset can be, and the bandwidth how quickly that data reaches the compute cores. You can find more explanation of this type of memory in our article on HBM memory. The TDP of 700 W is a maximum and can be configured.
For compute precision, NVIDIA gives FP64 as 34 TFLOPS and FP32 as 67 TFLOPS, without mention of sparsity. The FP16/BF16 value of 1,979 TFLOPS and the FP8 value of 3,958 TFLOPS, however, apply with sparsity; the dense values are not on the product page. FP4 is not mentioned. If you want to know what those formats mean, read compute precision explained.
That way you keep an idea in plain words of what the card can do, without having to read through all the datasheets.
What to look out for with the H100 SXM
An SXM GPU needs a suitable server platform with enough power and cooling for 700 W per GPU. Also look at the connection between GPUs: NVLink at 900 GB/s and PCIe Gen5 at 128 GB/s play different roles, as you can read in NVLink, PCIe and GPU interconnects.
- Only compare figures with the same units: dense with dense, sparse with sparse.
- Check whether a provider supplies the SXM or the PCIe variant.
- Consider whether renting via a cloud or having your own server suits you better.
You can see the current price and availability at the partner via the button on the page.
Frequently asked questions
Are the H100 SXM's FP16 and FP8 figures dense or sparse?
According to NVIDIA, the FP16/BF16 value of 1,979 TFLOPS and the FP8 value of 3,958 TFLOPS apply with sparsity. The dense values are not on the product page, and we do not convert them.
How much memory does the H100 SXM have?
The H100 SXM has 80 GB of HBM3 memory with a bandwidth of 3.35 TB/s, per GPU. With MIG you can divide the GPU into up to 7 instances of 10 GB.
Is the H100 SXM the same as the H100 NVL?
No, the H100 NVL is a separate PCIe variant. According to the data, it has 94 GB of memory, 3.9 TB/s of bandwidth and a power rating of 350 to 400 W, compared with 80 GB and up to 700 W for the SXM.
Last updated on
