Strongly discourage applications from talking to the Kubernetes API Server

[scothis]

Spring Cloud Kubernetes encourages applications to talk directly to the Kubernetes API Server to get ConfigMap/Secret values and resolve Services. This is an anti-pattern that should be discouraged.

1. An application that needs to talk to the API service is unnecessarily coupled to Kubernetes violating Inversion of Control.
2. The API Server is a control plane and is not designed to scale out along side an application. If every application instance needs a connection to the server, the scalability of the cluster will be limited.
3. It violates least privilege. An exploited application should not be able to fetch configuration and secrets that the application shouldn’t have access to. While RBAC is able to limit access to specific resources for a service account, this is inefficient and requires significant additional configuration that must be maintained.

In general, applications should not need to know that they are running in Kubernetes.

Alternatives:

ConfigMaps and Secrets can instead be bound to Pods via volume mounts where each key is exposed as a file in a directory. The PropertySource can read from the filesystem to get the same effect as talking to the API Server. The files will be updated with the latest values, so reloadability is still available by watching the filesystem for changes.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#add-configmap-data-to-a-volume

Services are exposed via DNS within the cluster. Lookup the service name as an A record to resolve the IP, additional information is available as SRV records.

https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#services

Of course there is an exception to every rule. It is legitimate to talk to the API Server if the application needs to update resources or if it needs to watch for new resources that didn’t exist when the application was deployed. However, these are the traits of a controller (or operator), and not typically associated with applications.

[everflux]

Just a remark: The usage of secrets and configmaps through volume mounts is supported by spring-cloud-kubernetes, that is what the paths configuration is for.

[spencergibb]

See spring-cloud/spring-cloud-commons#604 for srv record integration

[spencergibb]

See #190 for watching volume mount

[spencergibb]

Let's use this issue to inform users of when it is appropriate to use the api (Spring Cloud Gateway for example)

[trisberg]

I would suggest setting default to be disabling the enableApi for both config and secrets and updating the docs to favor mounting config and secrets as volumes.

Also trying the enable reload for config/secret without accessing the API but haven't been successful yet. I think we need to add a filewatcher on the paths for mounted config/secrets to trigger a reload.

[ryanjbaxter]

We could just mount config maps and secrets to the container using a volume. Which would eliminate the reliance on the k8s API.

To solve the refresh issue we could implement a piece of infrastructure that would watch for changes to config maps and secrets the app is concerned about, then use something like Spring Cloud Bus to notify the application that something has changed and refresh. With this approach we get refresh working but also have just a single app using the k8s api.

[ojhughes]

Currently the core module sets up an API client. We can move that into a separate module such as spring-cloud-kubernetes-api-server. This will help people who consciously want a nice way of injecting a Kubernetes API client but it will opt-in.

[johanan1]

Hi!
Any update on this?
We are currently using spring cloud kubernetes but would like to disable the use of tha API completely.

[krunalsabnis]

Hi!
Any update on this?
We are currently using spring cloud kubernetes but would like to disable the use of tha API completely.

spring:
  application:
    name: my-service
  cloud:
    kubernetes:
      config:
        enabled: false
      secrets:
        enabled: true
        enable-api: false

This works as expected. It does not lookup config at bootstrap, also I disabled API call for secret.
I am using
springBootVersion = '2.4.0'
group: 'org.springframework.cloud', name: 'spring-cloud-starter-kubernetes-fabric8-config', version: '2.0.2'