{"id":5561,"date":"2024-07-23T10:30:41","date_gmt":"2024-07-23T10:30:41","guid":{"rendered":"https:\/\/openzeka.com\/en\/?page_id=5561"},"modified":"2024-07-23T10:30:41","modified_gmt":"2024-07-23T10:30:41","slug":"h100-tensor-core-gpu","status":"publish","type":"page","link":"https:\/\/openzeka.com\/en\/h100-tensor-core-gpu\/","title":{"rendered":"H100 Tensor Core GPU"},"content":{"rendered":"

NVIDIA H100 Tensor Core GPU<\/h1><\/div>

Extraordinary performance, scalability, and security for every data center.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

NVIDIA H100 Tensor Core GPU<\/h1><\/div><\/div><\/div><\/div><\/div>

An Order-of-Magnitude Leap for Accelerated Computing<\/h1><\/div>

Tap into exceptional performance, scalability, and security for every workload with the NVIDIA H100 Tensor Core GPU. With the NVIDIA NVLink Switch System, up to 256 H100 GPUs can be connected to accelerate exascale workloads. The GPU also includes a dedicated Transformer Engine to solve trillion-parameter language models. The H100\u2019s combined technology innovations can speed up large language models (LLMs) by an incredible 30X over the previous generation to deliver industry-leading conversational AI.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Securely Accelerate Workloads From Enterprise to Exascale<\/h1><\/div><\/div><\/div>
<\/span><\/div>

Projected performance subject to change. GPT-3 175B training A100 cluster: HDR IB network, H100 cluster: NDR IB network | Mixture of Experts (MoE) Training Transformer Switch-XXL variant with 395B parameters on 1T token dataset, A100 cluster: HDR IB network, H100 cluster: NDR IB network with NVLink Switch System where indicated.<\/p>\n<\/div><\/div><\/div>

Transformational AI Training<\/h1><\/div>

H100 features fourth-generation Tensor Cores and a Transformer Engine with FP8 precision that provides up to 4X faster training over the prior generation for GPT-3 (175B) models. The combination of fourth-generation NVLink, which offers 900 gigabytes per second (GB\/s) of GPU-to-GPU interconnect; NDR Quantum-2 InfiniBand networking, which accelerates communication by every GPU across nodes; PCIe Gen5; and NVIDIA Magnum IO\u2122 software delivers efficient scalability from small enterprise systems to massive, unified GPU clusters.<\/p>\n

Deploying H100 GPUs at data center scale delivers outstanding performance and brings the next generation of exascale high-performance computing (HPC) and trillion-parameter AI within the reach of all researchers.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

Real-Time Deep Learning Inference<\/h1><\/div>

AI solves a wide array of business challenges, using an equally wide array of neural networks. A great AI inference accelerator has to not only deliver the highest performance but also the versatility to accelerate these networks.<\/p>\n

H100 extends NVIDIA\u2019s market-leading inference leadership with several advancements that accelerate inference by up to 30X and deliver the lowest latency. Fourth-generation Tensor Cores speed up all precisions, including FP64, TF32, FP32, FP16, INT8, and now FP8, to reduce memory usage and increase performance while still maintaining accuracy for LLMs.<\/p>\n<\/div><\/div><\/div>

<\/span><\/div>

Projected performance subject to change. Inference on Megatron 530B parameter model based chatbot for input sequence length=128, output sequence length =20 | A100 cluster: HDR IB network | H100 cluster: NVLink Switch System, NDR IB<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/span><\/div>

Projected performance subject to change. 3D FFT (4K^3) throughput | A100 cluster: HDR IB network | H100 cluster: NVLink Switch System, NDR IB | Genome Sequencing (Smith-Waterman) | 1 A100 | 1 H100<\/p>\n<\/div><\/div><\/div>

Exascale High-Performance Computing<\/h1><\/div>

The NVIDIA data center platform consistently delivers performance gains beyond Moore\u2019s law. And H100\u2019s new breakthrough AI capabilities further amplify the power of HPC+AI to accelerate time to discovery for scientists and researchers working on solving the world\u2019s most important challenges.<\/p>\n

H100 triples the floating-point operations per second (FLOPS) of double-precision Tensor Cores, delivering 60 teraflops of FP64 computing for HPC. AI-fused HPC applications can also leverage H100\u2019s TF32 precision to achieve one petaflop of throughput for single-precision matrix-multiply operations, with zero code changes.<\/p>\n

H100 also features new DPX instructions that deliver 7X higher performance over A100 and 40X speedups over CPUs on dynamic programming algorithms such as Smith-Waterman for DNA sequence alignment and protein alignment for protein structure prediction.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

<\/span><\/div>

Supercharge Large Language Model Inference with H100 NVL<\/h1><\/div>

For LLMs up to 175 billion parameters, the PCIe-based NVIDIA H100 NVL with NVLink bridge utilizes Transformer Engine, NVLink, and 188GB HBM3 memory to provide optimum performance and easy scaling across any data center, bringing LLMs to the mainstream. Servers equipped with H100 NVL GPUs increase GPT-175B model performance up to 12X over NVIDIA DGX A100 systems while maintaining low latency in power-constrained data center environments.<\/p>\n<\/div><\/div><\/div><\/div><\/div>

NVIDIA H200 Tensor Core GPU<\/h1><\/div>\n
\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Form Factor<\/th>\nH100 SXM<\/th>\nH100 NVL<\/th>\n<\/tr>\n<\/thead>\n
FP64<\/td>\n34 teraFLOPS<\/td>\n30 teraFLOPS<\/td>\n<\/tr>\n
FP64 Tensor Core<\/td>\n67 teraFLOPS<\/td>\n60 teraFLOPS<\/td>\n<\/tr>\n
FP32<\/td>\n67 teraFLOPS<\/td>\n60 teraFLOPS<\/td>\n<\/tr>\n
TF32 Tensor Core<\/td>\n989 teraFLOPS<\/td>\n835 teraFLOPS<\/td>\n<\/tr>\n
BFLOAT16 Tensor Core<\/td>\n1979 teraFLOPS<\/td>\n1671 teraFLOPS<\/td>\n<\/tr>\n
FP16 Tensor Core<\/td>\n1979 teraFLOPS<\/td>\n1671 teraFLOPS<\/td>\n<\/tr>\n
FP8 Tensor Core<\/td>\n3958 teraFLOPS<\/td>\n3341 teraFLOPS<\/td>\n<\/tr>\n
INT 8 Tensor Core<\/td>\n3958 TOPS<\/td>\n3341 TOPS<\/td>\n<\/tr>\n
GPU Memory<\/td>\n80GB<\/td>\n94GB<\/td>\n<\/tr>\n
GPU Memory Bandwidth<\/td>\n3.35TB\/s<\/td>\n3.9TB\/s<\/td>\n<\/tr>\n
Decoders<\/td>\n7 NVDEC
\n7 JPEG<\/td>\n		7 NVDEC
\n7 JPEG<\/td>\n<\/tr>\n
NVIDIA AI Enterprise<\/td>\nAdd-on<\/td>\nIncluded<\/td>\n<\/tr>\n
Max Thermal Design Power (TDP)<\/td>\nUp to 700W(configurable)<\/td>\n350-400W(configurable)<\/td>\n<\/tr>\n
Multi-Instance GPUs<\/td>\nUp to 7 MIGs @10GB each<\/td>\nUp to 7 MIGS @12GB each<\/td>\n<\/tr>\n
Form Factor<\/td>\nSXM<\/td>\nPCIe dual-slot-air-cooled<\/td>\n<\/tr>\n
Interconnect<\/td>\nNVIDIA NVLink 900 GB\/s
\nPCIe Gen5: 128GB\/s<\/td>\n		2- or 4-way NVIDIA NVLink bridge: 900GB\/s PCIe Gen5: 128GB\/s<\/td>\n<\/tr>\n
Server Options<\/td>\nNVIDIA HGX H100 partner and NVIDIA-Certified Systems with 4 or 8 GPUs
\nNVIDIA DGX H100 with 8 GPUs<\/td>\n		Partner and NVIDIA-Certified Systems with 1-8 GPUs<\/td>\n<\/tr>\n
NVIDIA AI Enterprise<\/td>\nAdd-on<\/td>\nIncluded<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<\/div><\/div><\/div><\/div>

NVIDIA DGX SuperPOD<\/h1><\/div>

Leadership-Class AI Infrastructure<\/p>\n<\/div>

LEARN MORE<\/span><\/a><\/div><\/div><\/div><\/div><\/div>

For More Information<\/h1><\/div>