| GPU Model | NVIDIA® H100 80GB SXM5 |
| Architecture | NVIDIA® Hopper™ |
| Motherboard | Asus |
| GPUs per node | 8 |
| CPU Model | Intel® Xeon™ Platinum 8480+ 2 |
| CPU Cores | 112 |
| VRAM Type | HBM3 |
| VRAM Size | 80GB (640GB total) |
| RAM | 2TB |
| RAM type | DDR5 |
| Memory bandwidth | - 4th-generation NVIDIA® NVLink™
- 900 GB/s total bandwidth for multi-GPU I/O ops
|
| Form factor | HGX with a 7U form factor, leveraging SMX5 GPUs |
| Thermal Design Power (TDP) | 9.19 kW |
| Processing Power | TensorCore GPU & Transformer engine |
| Interconnect | Inter-GPU communication utilizes the 4th-generation NVIDIA® NVLink™, boasting a total bandwidth of 900 GB/sec for multi-GPU I/O ops. Communication between GPUs across nodes occurs through InfiniBand Quantum 2 switches, offering a speed of 3200 GB/sec. |
| Cooling | 45KW High-Density Racks utilize RDHx technology to efficiently manage cooling, addressing heat dissipation while ensuring optimal efficiency and reliability. |
| Deep Learning Framework Support | Seamlessly integrated with the NVIDIA NGC registry, which encompasses all the leading deep learning frameworks |
| Compatibility | Linux-based operating systems |
| Local storage | - 2x1.92TB NVMe U.2 Gen5
- 8x7.68TB NVMe U.2 Gen5
|