# Self-Hosting LLMs on Kubernetes: Securing llm-d and vLLM with Kong AI Gateway (Part 1)

llm-d is an open-source, Kubernetes-native framework for serving large language models at scale. It doesn't run models itself. It sits in front of inference engines such as vLLM, another open-source, high-throughput serving engine, and decides how requests are spread across them. Teams taking a…