NVIDIA Merlin HugeCTR (Huge Click-Through-Rate) is a deep neural network (DNN) training and inference framework designed for recommender systems. It provides distributed training with model-parallel embedding tables, an embeddings cache, and data-parallel neural networks across multiple GPUs and nodes for maximum performance. HugeCTR covers common and recent architectures such as Deep Learning Recommendation Model (DLRM), Wide and Deep, Deep Cross Network (DCN), and DeepFM.

Features

• Training Embeddings at Scale: Model parallelism and embedding cache designed for recommender workflows, enabling training of large embedding tables and full leverage of compute memory.
• Asynchronous, Multi-Threaded Pipeline: Inherently asynchronous and multi-threaded data reader that handles high-dimensional, sparse, or categorical data, feeding records directly to fully connected layers.
• Inference on Multiple GPUs: Concurrent model inference execution across multiple GPUs using a parameter server and embedding cache shared between multiple model instances.
• Interoperability with Open Source: Compatible with TensorFlow Distribute Strategy and Horovod, optimizing embeddings training within recommender workflows.
• Embeddings Optimization: Optimized embedding implementation up to 8X more performant than other frameworks, available as a TensorFlow plug-in.

Benefits

• High Performance: Distributed training and inference across multiple GPUs and nodes for maximum performance.
• Efficiency: Asynchronous, multi-threaded data loading and optimized embeddings for efficient training and inference.
• Scalability: Suitable for large-scale recommender systems with model-parallel embeddings and data-parallel neural networks.
• Flexibility: Interoperable with open-source components and compatible with TensorFlow and Horovod.
• Optimization: Enhanced embeddings optimization for better prediction at scale.