Falco Audit Plugin for EKS - HA and sidekick

[cloudec1993]

Motivation
I would like to have High Avaibility for my falco audit eks plugin which works as one pod in eks cluster. I don’t want to miss any of the events from eks if my pod will be failing or crashing due to different reasons (OOM etc)

I would like to utilise Falcosidkick pods for eks audit plugin pod which I have deployed for syscalls etc in order to not waste resources on my cluster. Can I point in configuration the falcosidekick endpoint instead of creating new one for the audit deployment?

HA for audit pods - it’s dangerous to have just one pod for audit eks plugin since the pod can crash or stop working and I will be missing for some time the alerts

Sidekick pods - I’m wasting resources on cluster instead of using already working sidekick pods

Feature

I would like to run the audit plugin pods with 3 replicas in order to provide HA and consume the logs from SQS in order to not have duplicated alerts

I would like to point in the configuration which sidekick I want to use instead of creating new one
Alternatives

Additional context

[robert-pudlowski-mox]

Any information?

[Issif]

The k8saudit-eks plugin pulls the logs from AWS Cloudwatch Logs, based on some filters like the cluster name and the type of the logs (k8saudit). They are only exposed like that by AWS (to make you pay for the access and the storage), and this is why we can't have multiple falco instances pulling the same logs, or they will all fire the same alerts.
A solution would be to have multiple falco + k8saudit-eks plugin pods, they all send their alerts to falcosidekick and you use a third party available as output in sidekick which has deduplication (NATS, kafka, rabbitmq, alertmanager, ...)

Concerning the second question, it's totally possible to re-use the same falcosidekick deployment. Imagine you deployed falco + falcosidekick together, and you want to deploy falco + k8saudit-eks plugin aside. The service for falcosidekick has been created by the first deployment, and you can configure the second falco deployment to forward its alerts to it like this (with helm):

falco:
  json_output: true
  json_include_output_property: true
  json_include_tags_property: true
  http_output:
    enabled: true
    url: "http://falco-falcosidekick.falco:2801"

[Issif]

Can we close this issue? Is my answer enough detailed for you both?