Include Lua filter in cilium envoy for service-mesh

[harshaisgud]

I am in the process of testing port to cilium service mesh and CNI from AWS CNI and Istio. One of the requirements is to enable security headers for all responses out of a cluster. It has been achieved in Istio using an approach similar to https://gist.github.com/kabute/ef8e7198031c8a99212a629a139ac83f .

I am trying to achieve the same on cilium cluster mesh using CiliumEnvoyConfig and noticed that the Lua filter is not activated. Would it be possible to please activate it? I think this will simplify the migration from Istio to Cilium .

[sergeyshevch]

I also find the requirement to use a Lua filter for adding a custom header into all requests to the backend. I need to implement a filter with the same behavior as https://github.com/kubeflow/manifests/blob/3e08dc102059def5a0b0d04560c7d119959bf506/common/istio/add-anonymous-user-filter/base/envoy-filter.yaml

[jtcarnes]

I am looking at porting our services to cilium service mesh. My org does some rather annoying things with headers that we normally use Nginx + Lua to solve. It would make adoption much easier to allow these to be easily ported to the service mesh.

[sergeyshevch]

@jtcarnes I was moved to Istio as a result. I guess you can create PR with these changes. It's really easy to implement

[jtcarnes]

@sergeyshevch I'm absolutely willing to write the MR, but would the project be interested in it or is there stance just use istio?

[sergeyshevch]

@jtcarnes We returned to Istio because Kubeflow deeply integrated into it and it's impossible to use it without Istio. That was the single reason.

I guess MR is always welcome

[dbazhal]

Plussing this, we would love to see this implemented for cilium service mesh. If there could be someone to point out where to start, I'd love to help getting this done.

[sca-]

+1 needing lua filter

[kevinlmadison]

+1 This is super important for my organization as well, I'm not super well versed on what would be needed to implement this but @sergeyshevch mentioned above that it would be really easy to implement and I would be willing to do that if I could have just a tiny bit of guidance!

[sizgiyaev]

Is there any news on that? It's very important for us, our org giving up an envoy because of lack of lua support in cilium envoy

[youngnick]

We're unlikely to add Lua support any time soon, for similar reasons to what John Howard from Istio described on his blog about EnvoyFilter in Istio:

EnvoyFilter is, objectively, the worst feature in Istio for stability. Essentially, it gives arbitrary patching into Envoy code. An analogy would be to provide a fast-moving project a git diff that is patched dynamically and recompiled; EnvoyFilter is only slightly more stable than that. In addition to risks of breakage, particularly around upgrades, safe usage requires a deep understanding of Envoy, which is surprisingly hard.

(from https://blog.howardjohn.info/posts/opinionated-istio/)

We have CiliumEnvoyConfig available as an escape hatch, but it's really an implementation detail, and we can't guarantee that the API will always be stable. It's way better to build the sorts of things you would do with Lua directly into Cilium's Gateway API support.

See also cilium/cilium#27008 for a similar discussion.