Kong AI Gateway allows you to expose, secure, and govern LLM resources. It helps make AI initiatives secure, reliable, and cost-efficient by turning your Kong Gateway into an AI Gateway. You can use the same Gateway to secure, govern, and control LLM consumption from popular AI providers, including OpenAI, Azure AI, AWS Bedrock, and GCP Vertex. It provides visibility into AI consumption by tracking LLM usage via pre-built dashboards and AI-specific analytics, enabling informed decisions around LLM exposure and AI project rollouts. The AI Gateway also enhances cost efficiency by caching responses to redundant prompts and automatically routing requests to the best model for the prompt. It allows you to build new AI applications faster with multiple LLMs, AI security, and AI metrics. You can also power all existing API traffic with AI without writing code using declarative configuration, and transform, enrich, and augment API responses with no-code AI integrations. The gateway also helps implement advanced prompt security and build prompt templates and AI contexts.

Features:

• Multi-LLM support: Ensure every LLM use case is covered. Use Kong’s unified API interface to work with multiple different AI providers at the flip of a switch. Seamlessly switch between AI providers to unlock new use cases and ensure high availability in the event of downtime.
• AI Traffic Control: Enforce advanced AI policies. Make AI traffic more efficient with semantic caching, routing, and load balancing. Protect resources and ensure compliance with semantic prompt guards, PII sanitization, and more.
• Simplify RAG pipelines: Let Kong implement RAG pipelines for you. Automatically build RAG pipelines at the gateway layer without needing developer or AI agent intervention. Consistently implement RAG pipelines at scale to ensure higher quality LLM responses and reduce hallucinations. Enhance governance with the ability to easily configure and update RAG pipelines in a centralized manner.
• AI metrics and observability: L7 observability on AI traffic for cost monitoring and tuning. Track AI consumption as API requests and token usage. Optimize AI usage and cost with predictive consumption models. Debug AI exposure via logging, tracing, and more.
• No-code Integrations: Accelerate AI development with no-code plugins. Introduce AI inside of your organization without needing to write a single line of code. Easily augment, enrich, or transform API traffic using any LLM provider that Kong supports.