# How to Monitor and Control LLM Cost with Kong AI Gateway

This talk explains how Paychex built LLM cost and observability dashboards around Kong AI Gateway to make multi-model AI usage measurable, accountable, and easier to operate. Likhita, an AI platform engineer at Paychex, describes how the company routes requests from nearly 100 internal consumers across more than 50 models and multiple providers, including Azure and GCP, and why manual reporting from separate provider consoles became too slow and error-prone. By scraping Kong-exposed metrics into Prometheus and visualizing them in Grafana, the team created a unified view of request volume, success and error rates, prompt and completion token usage, latency, streaming versus non-streaming behavior, and estimated cost by consumer, provider, and model. The dashboards helped identify blind spots such as rate-limit issues, traffic spikes, and high-cost models, while improving chargeback visibility for engineering, leadership, and finance. The speaker also notes current limitations, including manual pricing inputs and gaps between estimated operational cost and actual contracted cloud spend.