The Merlin HugeCTR container enables users to perform data preprocessing, feature engineering, train models with HugeCTR, and then serve the trained model with Triton Inference Server. This container is part of the NVIDIA Merlin framework, which accelerates the entire recommender systems pipeline on the GPU, from data ingestion and training to deployment. Merlin empowers data scientists, machine learning engineers, and researchers to build high-performing recommenders at scale. Each stage of the Merlin pipeline offers an easy-to-use API and is optimized to support hundreds of terabytes of data.

Features

• NVTabular: Performs data preprocessing and feature engineering for tabular data, scaling to manipulate terabyte-scale datasets.
• HugeCTR: Trains deep learning recommender models, written in CUDA C++ for optimal performance with NVIDIA GPUs. It includes optimized data loaders for distributed training across multiple GPUs and nodes, and strategies for scaling large embedding tables beyond available memory.
• Triton Inference Server: Provides GPU-accelerated inference, simplifying the deployment of AI models at scale. It supports models from various frameworks including TensorFlow, TensorRT, PyTorch, ONNX Runtime, or custom frameworks.
• Multi-Arch Support: Compatible with Linux/amd64 and Linux/arm64 architectures.
• Security: Signed images and comprehensive security scanning.

Benefits

• High Performance: Accelerates the entire recommender systems pipeline on the GPU.
• Scalability: Supports large datasets, making it suitable for extensive data processing and model training.
• Ease of Deployment: Simplifies the deployment of trained models with Triton Inference Server.
• Versatility: Supports various AI frameworks and deployment environments.
• Efficiency: Optimized for supercomputing scale with multi-node and multi-GPU support.