The entire system runs inside of Kubernetes and takes advantage of Kubernetes features. It cannot run outside of Kubernetes, but this functionality might be built in the future.
The proxy is primarily responsible for accepting requests from the internet and forwarding them to the right backend. A backend is a set of containers that can scale up and down, including to 0 containers.
An incoming request to the system will reach the proxy first. When it does, the proxy first publishes an event to Redis. After this point, it forwards the request to a Kubernetes Service
for the backend intended to serve the request. This backend might not currently have any running containers.
KEDA (detailed below) is responsible for scaling (up and down) the pods that the Service
load balances over.
The proxy also has an "admin" API that is not intended to be exposed to the internet without authentication. Generally speaking, a command line tool or cloud portal would do operations on this API.
The API supports two major operations: (a) create new backend and (b) delete backend.
When the proxy gets a request to create a new backend, it does the following:
- Create a new Kubernetes
Deployment
s andService
s when a new application is created - Create a Kubernetes
Service
that forwards to and load-balances over the pods in theDeployment
- Create a new KEDA
ScaledObject
to indicate that the pods in the deployment should be scaled based on NATS traffic (more on all of this next)
Deleting a backend reverses the steps in the "create" step. That means it will delete all of the resources that it created there.
Behind the scenes, the prometheus the proxy publishes are consumed by KEDA. KEDA is responsible for responding to these events and scale up / down the number of pods in the Deployment
mentioned above. The Service
that the proxy forwards to provides a stable endpoint that load balances over all of the pods, regardless of how many there are.