{"id":5504,"date":"2024-07-19T13:48:51","date_gmt":"2024-07-19T13:48:51","guid":{"rendered":"https:\/\/openzeka.com\/en\/?page_id=5504"},"modified":"2024-07-19T13:48:51","modified_gmt":"2024-07-19T13:48:51","slug":"gb200-nvl2","status":"publish","type":"page","link":"https:\/\/openzeka.com\/en\/gb200-nvl2\/","title":{"rendered":"GB200 NVL2"},"content":{"rendered":"

NVIDIA GB200 NVL2<\/h1><\/div>

Bringing the new era of computing to every data center.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

NVIDIA GB200 NVL2<\/h1><\/div>

Bringing the new era of computing to every data center.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Unparalleled Single-Server Performance<\/h1><\/div>

The NVIDIA GB200 NVL2 platform brings the new era of computing to every data center, delivering unparalleled performance for mainstream large language model (LLM) inference, vector database search, and data processing through 2 Blackwell GPUs and 2 Grace GPUs. With its scale-out, single node NVIDIA MGX architecture, its design enables a wide variety of system designs and networking options to seamlessly integrate accelerated computing into existing data center infrastructure.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Turbocharging Accelerated Computing<\/h1><\/div><\/div><\/div>

Llama 3 Inference<\/strong><\/h1><\/div>

5X<\/h1><\/div>

vs. NVIDIA H100 Tensor Core GPU<\/p>\n<\/div><\/div><\/div>

Vector Database Search<\/strong><\/h1><\/div>

9X<\/h1><\/div>

vs. H100<\/p>\n<\/div><\/div><\/div>

Data Processing<\/strong><\/h1><\/div>

18X<\/h1><\/div>

vs. CPU<\/p>\n<\/div><\/div><\/div>

Llama3 LLM inference: Token-to-token latency (TTL) = 50 milliseconds (ms) real time, first token latency (FTL) = 2s, input sequence length = 2.048, output sequence length = 128 output, 8x NVIDIA HGX\u2122 H100 air-cooled vs. GB200 NVL2 air-cooled single node, per-GPU performance comparison
\nVector database search performance within RAG pipeline using memory shared by NVIDIA Grace CPU and Blackwell GPU. 1x x86, 1x H100 GPU, and 1x GPU from GB200 NVL2 node.
\nData processing: A database join and aggregation workload with Snappy\/Deflate compression derived from TPC-H Q4 query. Custom query implementations for x86, H100 single GU, and single GPU from GB200 NVL2 node: GB200 vs. Intel Xeon 8480+
\nProjected performance subject to change.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/span><\/div>

Real-Time Mainstream LLM Inference<\/h1><\/div>

GB200 NVL2 introduces massive coherent memory up to 1.3 terabytes (TB) shared between two Grace CPUs and two Blackwell GPUs. This shared memory is coupled with fifth-generation NVIDIA\u00ae NVLink and high-speed, chip-to-chip (C2C) connections to deliver 5X faster real-time LLM inference performance for mainstream language models such as Llama 3 70B.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Vector Database Search<\/h1><\/div>

GB200 NLV2 accelerates RAG vector search operation by up to 9X. The vector database of the Wikipedia dataset is over 200 gigabytes (GB), and access to the Grace CPU\u2019s 960GB of memory and 900GB\/s high-speed C2C link supercharges low-latency vector search.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Data Processing<\/h1><\/div>

Databases play critical roles in handling, processing, and analyzing large volumes of data for enterprises. GB200 NVL2 takes advantage of high-bandwidth memory performance, NVLink-C2C, and dedicated decompression engines in the NVIDIA Blackwell architecture to speed up key database queries by 18X compared to CPU.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Technological Breakthroughs<\/h1><\/div><\/div><\/div>
<\/span><\/div>

Blackwell Architecture<\/h1><\/div>

The NVIDIA Blackwell architecture delivers groundbreaking advancements in accelerated computing, powering a new era of computing with unparalleled performance, efficiency, and scale.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

NVIDIA Grace CPU<\/h1><\/div>

The NVIDIA Grace CPU is a breakthrough processor designed for modern data centers running AI, cloud, and HPC applications. It provides outstanding performance and memory bandwidth with 2X the energy efficiency of today\u2019s leading server processors.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

NVLINK C2C<\/h1><\/div>

NVIDIA NVLink-C2C coherently interconnects each Grace CPU and Blackwell GPU at 900GB\/s. The GB200 NVL2 uses both NVLink-C2C and the fifth-generation NVLink to deliver a 1.4 TB coherent memory model for accelerated AI.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Key Value (KV) Caching<\/h1><\/div>

Key Value (KV) Caching improves LLM response speeds by storing conversation context and history. GB200 NVL2 optimizes KV Caching through its fully coherent Grace GPU and Blackwell GPU memory connected by NVLink-C2C, 7X faster than PCIe, enabling LLMs to predict words faster than x86-based GPU implementations.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Fifth- Generation NVIDIA NVLink<\/h1><\/div>

Unlocking the full potential of exascale computing and trillion-parameter AI models requires swift, seamless communication between every GPU in a server cluster. Fifth-generation NVLink is a scale-up interconnect that unleashes accelerated performance for trillion- and multi-trillion-parameter AI models.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

NVIDIA Networking<\/h1><\/div>

The data center\u2019s network plays a crucial role in driving AI advancements and performance, serving as the backbone for distributed AI model training and generative AI performance. NVIDIA Quantum-X800 InfiniBand, NVIDIA Spectrum-X800 Ethernet, and NVIDIA BlueField-3 DPUs enable efficient scalability across hundreds and thousands of Blackwell GPUs for optimal application performance.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

GB200 NVL2 Specs<\/h1><\/div>\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
<\/th>\nGB200 NVL2<\/th>\n<\/tr>\n<\/thead>\n
Configuration<\/td>\n2x Grace CPUs, 2x Blackwell GPUs<\/td>\n<\/tr>\n
FP4 Tensor Core<\/td>\n40 PFLOPS<\/td>\n<\/tr>\n
FP8\/FP6 Tensor Core<\/td>\n20 PFLOPS<\/td>\n<\/tr>\n
INT8 Tensor Core<\/td>\n20 POPS<\/td>\n<\/tr>\n
FP16\/BF16 Tensor Core<\/td>\n10 PFLOPS<\/td>\n<\/tr>\n
TF32 Tensor Core<\/td>\n5 PFLOPS<\/td>\n<\/tr>\n
FP32<\/td>\n180 TFLOPS<\/td>\n<\/tr>\n
FP64\/FP64 Tensor Core<\/td>\n90 TFLOPS<\/td>\n<\/tr>\n
GPU Memory | Bandwidth<\/td>\nUp to 384GB | 16TB\/s<\/td>\n<\/tr>\n
CPU Core Count<\/td>\n144 Arm Neoverse V2 cores<\/td>\n<\/tr>\n
LPDDR5X Memory | Bandwidth<\/td>\nUp to 960GB | Up to 1,024GB\/s<\/td>\n<\/tr>\n
Interconnect<\/td>\nNVLink: 1.8TB\/s
\nNVLink-C2C: 2x 900GB\/s
\nPCIe Gen6: 2x 256GB\/s<\/td>\n<\/tr>\n
Server Options<\/td>\nVarious NVIDIA GB200 NVL2 configuration options using NVIDIA MGX<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div><\/div><\/div><\/div>

NVIDIA DGX SuperPOD<\/h1><\/div>

Leadership-Class AI Infrastructure<\/p>\n<\/div>

LEARN MORE<\/span><\/a><\/div><\/div><\/div><\/div><\/div>

For More Information<\/h1><\/div>