The NVIDIA NIM Operator enables Kubernetes cluster administrators to operate the software components and services necessary to run NVIDIA NIMs in various domains such as reasoning, retrieval, speech, and biology. Additionally, it allows the use of NeMo Microservices to fine-tune, evaluate, or apply guardrails to your models.

Features

• NVIDIA NIM Models: Supports reasoning LLMs, retrieval (embedding, reranking, etc.), speech, and biology models.
• NeMo Core Microservices: Includes NeMo Customizer, NeMo Evaluator, and NeMo Guardrails.
• NeMo Platform Component Microservices: Provides NeMo Data Store and NeMo Entity Store.
• Helm Chart: Facilitates easy deployment of the NIM operator in a cluster to provision NVIDIA NIMs on GPU-enabled nodes.
• Multi-Arch Support: Compatible with Linux/amd64 and Linux/arm64 architectures.
• Security: Signed images and comprehensive security scanning.

Benefits

• Efficient Management: Simplifies the deployment and maintenance of NVIDIA NIMs and NeMo microservices.
• Versatility: Supports a wide range of models and microservices for various applications.
• Ease of Use: Helm chart for straightforward deployment in Kubernetes environments.
• Security: Ensures secure operations with signed images and thorough security scans.