Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build(helm): Make Kueue configurable to REANA admins at the deployment time #801

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

xaviertintin
Copy link

Helm-Controlled Deployment Choice:

Make the behaviour configurable to REANA admins at the deployment time, so that each admin can choose whether to use the classical approach or the new optional Kueue approach, via Helm values.

  • Configuration via Helm Values: REANA administrators can select the desired job submission method (standard or Kueue) during deployment. Helm values provide a user-friendly way to achieve this, aligning with REANA's existing configuration practices.
  • Environment Variable Configuration:
    • reana-workflow-controller: Define the environment variables within its related template, following existing REANA practices (e.g., helm/reana/templates/reana-workflow-controller.yaml).
    • reana-job-controller: This controller is dynamically created by the reana-workflow-controller and not directly from the Helm chart. Therefore, the environment variable needs to be added to the Kubernetes specification created in reana_workflow_controller/workflow_run_manager.py.

Recap:

  1. Add kueue.enabled Value: Introduce a new value named kueue.enabled within the values.yaml file of the Helm chart.
  2. Pass Value as Environment Variable (reana-workflow-controller): In the helm/reana/templates/reana-workflow-controller.yaml template, configure the kueue.enabled value as an environment variable and pass it to the reana-workflow-controller container.
  3. Read Environment Variable (reana-workflow-controller): As suggested by Giuseppe, modify config.py for the reana-workflow-controller to access and utilize this environment variable.
  4. Pass Environment Variable (reana-job-controller): Within reana_workflow_controller/workflow_run_manager.py, modify the Kubernetes specification for the dynamically created reana-job-controller to include the environment variable containing the kueue.enabled value.

By following these steps and leveraging Helm values for configuration, REANA admins gain control over the job submission method during deployment. Additionally, the environment variable approach ensures proper communication between the reana-workflow-controller and the dynamically created reana-job-controller.

Kueue Integration Milestone 2 Discussion here

Automated Kueue Deployment with Customization:

Deploy Kueue cluster automatically during REANA deployment when the admin chooses to use Kueue in their Helm values file. Allow basic parametrisation of Kueue deployment by overriding Helm values and/or chart snippets.

  • Streamlines the process by automatically deploying the Kueue cluster when the admin selects Kueue in their Helm values file.
  • Offers basic configuration options for the Kueue deployment through Helm values and/or chart snippets, allowing admins to tailor it to their specific needs.

Production Instance Considerations:

For production deployments, creating a Kubernetes cluster (e.g., at CERN or Google Cloud) is necessary. While ideally Kueue deployment would be automated through the Helm chart, limitations might exist until a dedicated Kueue Helm chart is available. As a temporary solution, Kueue might need to be manually deployed until a more integrated approach can be implemented.

Kueue Integration Milestone 3 Discussion here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant