{"id":5479,"date":"2024-07-19T13:50:16","date_gmt":"2024-07-19T13:50:16","guid":{"rendered":"https:\/\/openzeka.com\/en\/?page_id=5479"},"modified":"2024-07-19T13:53:27","modified_gmt":"2024-07-19T13:53:27","slug":"gb200-nvl72","status":"publish","type":"page","link":"https:\/\/openzeka.com\/en\/gb200-nvl72\/","title":{"rendered":"GB200 NVL72"},"content":{"rendered":"

NVIDIA GB200 NVL72<\/h1><\/div>

Powering the new era of computing.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

NVIDIA GB200 NVL72<\/h1><\/div>

Powering the new era of computing.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Unlocking Real-Time Trillion-Parameter Models<\/h1><\/div>

GB200 NVL72 connects 36 Grace CPUs and 72 Blackwell GPUs in a rack-scale design. The GB200 NVL72 is a liquid-cooled, rack-scale solution that boasts a 72-GPU NVLink domain that acts as a single massive GPU and delivers 30X faster real-time trillion-parameter LLM inference.<\/p>\n

The GB200 Grace Blackwell Superchip is a key component of the NVIDIA GB200 NVL72, connecting two high-performance NVIDIA Blackwell Tensor Core GPUs and an NVIDIA Grace CPU using the NVIDIA NVLink-C2C interconnect to the two Blackwell GPUs.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Supercharging Next-Generation AI and Accelerated Computing<\/h1><\/div><\/div><\/div>

LLM Inference<\/strong><\/h1><\/div>

30X<\/h1><\/div>

vs. NVIDIA H100 Tensor Core GPU<\/p>\n<\/div><\/div><\/div>

LLM Training<\/strong><\/h1><\/div>

4X<\/h1><\/div>

vs. H100<\/p>\n<\/div><\/div><\/div>

Energy Efficiency<\/strong><\/h1><\/div>

25X<\/h1><\/div>

vs. H100<\/p>\n<\/div><\/div><\/div>

Data Processing<\/strong><\/h1><\/div>

18X<\/h1><\/div>

vs. CPU<\/p>\n<\/div><\/div><\/div>

LLM inference and energy efficiency: TTL = 50 milliseconds (ms) real time, FTL = 5s, 32,768 input\/1,024 output, NVIDIA HGX\u2122 H100 scaled over InfiniBand (IB) vs. GB200 NVL72, training 1.8T MOE 4096x HGX H100 scaled over IB vs. 456x GB200 NVL72 scaled over IB. Cluster size: 32,768
\nA database join and aggregation workload with Snappy \/ Deflate compression derived from TPC-H Q4 query. Custom query implementations for x86, H100 single GPU and single GPU from GB200 NLV72 vs. Intel Xeon 8480+
\nProjected performance subject to change.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Technological Breakthroughs<\/h1><\/div><\/div><\/div>
<\/span><\/div>

Blackwell Architecture<\/h1><\/div>

The NVIDIA Blackwell architecture delivers groundbreaking advancements in accelerated computing, powering a new era of computing with unparalleled performance, efficiency, and scale.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

NVIDIA Grace CPU<\/h1><\/div>

The NVIDIA Grace CPU is a breakthrough processor designed for modern data centers running AI, cloud, and HPC applications. It provides outstanding performance and memory bandwidth with 2X the energy efficiency of today\u2019s leading server processors.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

NVIDIA Grace CPU<\/h1><\/div>

The NVIDIA Grace CPU is a breakthrough processor designed for modern data centers running AI, cloud, and HPC applications. It provides outstanding performance and memory bandwidth with 2X the energy efficiency of today\u2019s leading server processors.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

NVIDIA Networking<\/h1><\/div>

The data center\u2019s network plays a crucial role in driving AI advancements and performance, serving as the backbone for distributed AI model training and generative AI performance. NVIDIA Quantum-X800 InfiniBand, NVIDIA Spectrum-X800 Ethernet, and NVIDIA BlueField\u00ae-3 DPUs enable efficient scalability across hundreds and thousands of Blackwell GPUs for optimal application performance.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/span><\/div>

Real-Time LLM Inference<\/h1><\/div>

GB200 NVL72 introduces cutting-edge capabilities and a second-generation Transformer Engine which enables FP4 AI and when coupled with fifth-generation NVIDIA NVLink, delivers 30X faster real-time LLM inference performance for trillion-parameter language models. This advancement is made possible with a new generation of Tensor Cores, which introduce new microscaling formats, giving high accuracy and greater throughput. Additionally, the GB200 NVL72 uses NVLink and liquid cooling to create a single massive 72-GPU rack that can overcome communication bottlenecks.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Massive-Scale Training<\/h1><\/div>

GB200 NVL72 includes a faster second-generation Transformer Engine featuring FP8 precision, enabling a remarkable 4X faster training for large language models at scale. This breakthrough is complemented by the fifth-generation NVLink, which provides 1.8 terabytes per second (TB\/s) of GPU-to-GPU interconnect, InfiniBand networking, and NVIDIA Magnum IO software.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Energy-Efficient Infrastructure<\/h1><\/div>

Liquid-cooled GB200 NVL72 racks reduce a data center\u2019s carbon footprint and energy consumption. Liquid cooling increases compute density, reduces the amount of floor space used, and facilitates high-bandwidth, low-latency GPU communication with large NVLink domain architectures. Compared to NVIDIA H100 air-cooled infrastructure, GB200 delivers 25X more performance at the same power while reducing water consumption.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Data Processing<\/h1><\/div>

Databases play critical roles in handling, processing, and analyzing large volumes of data for enterprises. GB200 takes advantage of the high-bandwidth memory performance, NVLink-C2C, and dedicated decompression engines in the NVIDIA Blackwell architecture to speed up key database queries by 18X compared to CPU and deliver a 5X better TCO.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

GB200 NVL72 Specs<\/h1><\/div>\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
<\/th>\nGB200 NVL72<\/th>\nGB200 Grace Blackwell Superchip<\/th>\n<\/tr>\n<\/thead>\n
Configuration<\/td>\n36 Grace CPU : 72 Blackwell GPUs<\/td>\n1 Grace CPU : 2 Blackwell GPU<\/td>\n<\/tr>\n
FP4 Tensor Core<\/td>\n1440 PFLOPS<\/td>\n40 PFLOPS<\/td>\n<\/tr>\n
FP8\/FP6 Tensor Core<\/td>\n720 PFLOPS<\/td>\n20 PFLOPS<\/td>\n<\/tr>\n
Int8 Tensor Core<\/td>\n720 POPS<\/td>\n20 POPS<\/td>\n<\/tr>\n
FP16\/BF16 Tensor Core<\/td>\n360 PFLOPS<\/td>\n10 PFLOPS<\/td>\n<\/tr>\n
TF32 Tensor Core<\/td>\n180 PFLOPS<\/td>\n5 PFLOPS<\/td>\n<\/tr>\n
FP32<\/td>\n6480 TFLOPS<\/td>\n180 TFLOPS<\/td>\n<\/tr>\n
FP64<\/td>\n3240 TFLOPS<\/td>\n90 TFLOPS<\/td>\n<\/tr>\n
FP64 Tensor Core<\/td>\n3240 TFLOPS<\/td>\n90 TFLOPS<\/td>\n<\/tr>\n
GPU Memory | Bandwidth<\/td>\nUp to 13.5 TB HBM3e | 576 TB\/s<\/td>\nUp to 384 GB HBM3e | 16 TB\/s<\/td>\n<\/tr>\n
NVLink Bandwidth<\/td>\n130TB\/s<\/td>\n3.6TB\/s<\/td>\n<\/tr>\n
CPU Core Count<\/td>\n2,592 Arm Neoverse V2 cores<\/td>\n72 Arm Neoverse V2 cores<\/td>\n<\/tr>\n
CPU Memory | Bandwidth<\/td>\nUp to 17 TB LPDDR5X | Up to 18.4 TB\/s<\/td>\nUp to 480GB LPDDR5X | Up to 512 GB\/s<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div><\/div><\/div><\/div>

NVIDIA DGX SuperPOD<\/h1><\/div>

Leadership-Class AI Infrastructure<\/p>\n<\/div>

LEARN MORE<\/span><\/a><\/div><\/div><\/div><\/div><\/div>

For More Information<\/h1><\/div>