NVIDIA® Transformer Engine is a library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs. This provides better performance with lower memory utilization in both training and inference.

Features

• FP8 Precision: Utilizes 8-bit floating point (FP8) precision for improved performance and reduced memory usage.
• GPU Optimization: Optimized for NVIDIA Hopper and Ada GPUs.
• Training and Inference: Enhances both training and inference processes.
• Comprehensive Documentation: Includes user guides, release notes, and software license agreements.

Benefits

• Enhanced Performance: Achieves better performance with lower memory utilization.
• Efficient Resource Usage: Reduces memory usage, making it more efficient.
• Versatile Application: Suitable for both training and inference of Transformer models.
• Detailed Documentation: Provides comprehensive documentation for ease of use.