Update Flyte components (#5536)

* Update Flyte Components Signed-off-by: Flyte-Bot <admin@flyte.org> * Add changelog and bump version in docs/conf.py Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> --------- Signed-off-by: Flyte-Bot <admin@flyte.org> Signed-off-by: Eduardo Apolinario <eapolinario@users.noreply.github.com> Co-authored-by: eapolinario <eapolinario@users.noreply.github.com>
flyteorg · Jul 3, 2024 · f075b34 · f075b34
1 parent 2b0e8ef
commit f075b34
Show file tree

Hide file tree

Showing 23 changed files with 198 additions and 143 deletions.
diff --git a/CHANGELOG/CHANGELOG-v1.13.0.md b/CHANGELOG/CHANGELOG-v1.13.0.md
@@ -0,0 +1,55 @@
+# Flyte 1.13.0 Release Notes
+
+Flyte 1.13.0 brings a host of new features, optimizations, and fixes, enhancing the platform's functionality and user experience. This release also welcomes several new contributors to the Flyte community. Below are the highlights of this release.
+
+## 🚀 New Features & Improvements
+
+1. **Features**
+   - [Watch agent metadata service](https://github.com/flyteorg/flyte/pull/5017)
+   - [Allow controlling in which task phases log links are shown](https://github.com/flyteorg/flyte/pull/4726)
+   - [Add executionClusterLabel](https://github.com/flyteorg/flyte/pull/5394)
+
+2. **Helm Charts & Manifests**
+   - [Grafana dashboard updates](https://github.com/flyteorg/flyte/pull/5255) for better monitoring and observability.
+   - [Core helm chart updates for propeller configuration of agent service](https://github.com/flyteorg/flyte/pull/5402).
+   - [Helm chart updates related to Prometheus, Webhook HPA, and Flyteconsole probes](https://github.com/flyteorg/flyte/pull/5508).
+
+3. **Documentation Improvements**
+   - Added examples and guides, including [a guide for setting up the OpenAI batch agent backend](https://github.com/flyteorg/flyte/pull/5291) and [updated Optimization Performance docs](https://github.com/flyteorg/flyte/pull/5278).
+   - [Fix doc link to testing agent on local cluster](https://github.com/flyteorg/flyte/pull/5398) and [replace Azure AD OIDC URL](https://github.com/flyteorg/flyte/pull/4075).
+   - [Clarify networking configurations between data plane propeller and control plane data catalog in multi-cluster deployments](https://github.com/flyteorg/flyte/pull/5345).
+
+4. **Performance & Bug Fixes**
+   - [Fix cache and token management issues](https://github.com/flyteorg/flyte/pull/5388), including addressing auto-refresh cache race conditions and improving token refresh mechanisms.
+   - [Fix SQL insert operations](https://github.com/flyteorg/flyte/pull/5482) to handle NULL values appropriately.
+   - [Keep EnvFrom from pod template](https://github.com/flyteorg/flyte/pull/5423)
+   - [Miscellaneous fixes](https://github.com/flyteorg/flyte/pull/5416) for pod templates, Kubeflow webhook errors, and logging configurations.
+
+5. **Flytectl**
+   - [Add prefetch functionality for paginator](https://github.com/flyteorg/flyte/pull/5310)
+   - [Fix `upgrade` and `version` commands](https://github.com/flyteorg/flyte/pull/5470)
+
+
+## 🔧 Housekeeping & Deprecations
+
+1. **Miscellaneous**
+   - [Add OTLP and sampling to otelutils](https://github.com/flyteorg/flyte/pull/5504)
+   - [Refactor distributed job using common ReplicaSpec](https://github.com/flyteorg/flyte/pull/5355)
+   - [Fix typos using codespell CI job](https://github.com/flyteorg/flyte/pull/5418).
+   - [Remove obsolete Flyte config files](https://github.com/flyteorg/flyte/pull/5495).
+   - [Upgraded various dependencies](https://github.com/flyteorg/flyte/pull/5313), including Golang, Docker, and others.
+   - [Move to upstream mockery](https://github.com/flyteorg/flyte/pull/4937), which simplifies the migration to mockery-v2 
+   - [Key-value execution tags](https://github.com/flyteorg/flyte/pull/5453)
+
+## 🆕 New Contributors
+
+- **@zychen5186** for [adding prefetch functionality for paginator](https://github.com/flyteorg/flyte/pull/5310).
+- **@EraYaN** for [replacing Azure AD OIDC URL](https://github.com/flyteorg/flyte/pull/4075).
+- **@Dlougach** for [making BaseURL insensitive to trailing slashes for metadata endpoint redirect](https://github.com/flyteorg/flyte/pull/5458).
+- **@flixr** for [honoring redoc.enabled=false in charts](https://github.com/flyteorg/flyte/pull/5452).
+- **@trevormcguire** for [including group in apiVersion in plugin_collector](https://github.com/flyteorg/flyte/pull/5457).
+- **@va6996** for [inheriting execution cluster label from source execution](https://github.com/flyteorg/flyte/pull/5431).
+- **@mhotan** for [Helm chart updates related to Prometheus, Webhook HPA, and Flyteconsole probes](https://github.com/flyteorg/flyte/pull/5508).
+- **@eltociear** for [updating token_source.go](https://github.com/flyteorg/flyte/pull/5396).
+
+**[Full Changelog](https://github.com/flyteorg/flyte/compare/v1.12.0...v1.13.0)**
diff --git a/charts/flyte-binary/README.md b/charts/flyte-binary/README.md
@@ -42,7 +42,7 @@ Chart for basic single Flyte executable deployment
 | configuration.auth.oidc.clientId | string | `""` |  |
 | configuration.auth.oidc.clientSecret | string | `""` |  |
 | configuration.co-pilot.image.repository | string | `"cr.flyte.org/flyteorg/flytecopilot"` |  |
-| configuration.co-pilot.image.tag | string | `"v1.13.0-rc0"` |  |
+| configuration.co-pilot.image.tag | string | `"v1.13.0"` |  |
 | configuration.database.dbname | string | `"flyte"` |  |
 | configuration.database.host | string | `"127.0.0.1"` |  |
 | configuration.database.options | string | `"sslmode=disable"` |  |

diff --git a/charts/flyte-binary/values.yaml b/charts/flyte-binary/values.yaml
@@ -159,7 +159,7 @@ configuration:
       # repository CoPilot sidecar image repository
       repository: cr.flyte.org/flyteorg/flytecopilot  # FLYTECOPILOT_IMAGE
       # tag CoPilot sidecar image tag
-      tag: v1.13.0-rc0  # FLYTECOPILOT_TAG
+      tag: v1.13.0  # FLYTECOPILOT_TAG
   # agentService Flyte Agent configuration
   agentService:
     defaultAgent:

diff --git a/charts/flyte-core/README.md b/charts/flyte-core/README.md
@@ -95,8 +95,8 @@ helm install gateway bitnami/contour -n flyte
 | configmap.clusters.clusterConfigs | list | `[]` |  |
 | configmap.clusters.labelClusterMap | object | `{}` |  |
 | configmap.console | object | `{"BASE_URL":"/console","CONFIG_DIR":"/etc/flyte/config"}` | Configuration for Flyte console UI |
-| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.0-rc0","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration |
-| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.0-rc0","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) |
+| configmap.copilot | object | `{"plugins":{"k8s":{"co-pilot":{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.0","name":"flyte-copilot-","start-timeout":"30s"}}}}` | Copilot configuration |
+| configmap.copilot.plugins.k8s.co-pilot | object | `{"image":"cr.flyte.org/flyteorg/flytecopilot:v1.13.0","name":"flyte-copilot-","start-timeout":"30s"}` | Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig) |
 | configmap.core | object | `{"manager":{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"},"propeller":{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"},"webhook":{"certDir":"/etc/webhook/certs","serviceName":"flyte-pod-webhook"}}` | Core propeller configuration |
 | configmap.core.manager | object | `{"pod-application":"flytepropeller","pod-template-container-name":"flytepropeller","pod-template-name":"flytepropeller-template"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/manager/config#Config). |
 | configmap.core.propeller | object | `{"downstream-eval-duration":"30s","enable-admin-launcher":true,"leader-election":{"enabled":true,"lease-duration":"15s","lock-config-map":{"name":"propeller-leader","namespace":"flyte"},"renew-deadline":"10s","retry-period":"2s"},"limit-namespace":"all","max-workflow-retries":30,"metadata-prefix":"metadata/propeller","metrics-prefix":"flyte","prof-port":10254,"queue":{"batch-size":-1,"batching-interval":"2s","queue":{"base-delay":"5s","capacity":1000,"max-delay":"120s","rate":100,"type":"maxof"},"sub-queue":{"capacity":100,"rate":10,"type":"bucket"},"type":"batch"},"rawoutput-prefix":"s3://my-s3-bucket/","workers":4,"workflow-reeval-duration":"30s"}` | follows the structure specified [here](https://pkg.go.dev/github.com/flyteorg/flytepropeller/pkg/controller/config). |
@@ -130,7 +130,7 @@ helm install gateway bitnami/contour -n flyte
 | datacatalog.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
 | datacatalog.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
 | datacatalog.image.repository | string | `"cr.flyte.org/flyteorg/datacatalog"` | Docker image for Datacatalog deployment |
-| datacatalog.image.tag | string | `"v1.13.0-rc0"` | Docker image tag |
+| datacatalog.image.tag | string | `"v1.13.0"` | Docker image tag |
 | datacatalog.nodeSelector | object | `{}` | nodeSelector for Datacatalog deployment |
 | datacatalog.podAnnotations | object | `{}` | Annotations for Datacatalog pods |
 | datacatalog.podEnv | object | `{}` | Additional Datacatalog container environment variables |
@@ -166,7 +166,7 @@ helm install gateway bitnami/contour -n flyte
 | flyteadmin.extraArgs | object | `{}` | Appends extra command line arguments to the serve command |
 | flyteadmin.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flyteadmin.image.repository | string | `"cr.flyte.org/flyteorg/flyteadmin"` | Docker image for Flyteadmin deployment |
-| flyteadmin.image.tag | string | `"v1.13.0-rc0"` |  |
+| flyteadmin.image.tag | string | `"v1.13.0"` |  |
 | flyteadmin.initialProjects | list | `["flytesnacks","flytetester","flyteexamples"]` | Initial projects to create |
 | flyteadmin.nodeSelector | object | `{}` | nodeSelector for Flyteadmin deployment |
 | flyteadmin.podAnnotations | object | `{}` | Annotations for Flyteadmin pods |
@@ -238,7 +238,7 @@ helm install gateway bitnami/contour -n flyte
 | flytepropeller.extraArgs | object | `{}` | Appends extra command line arguments to the main command |
 | flytepropeller.image.pullPolicy | string | `"IfNotPresent"` |  |
 | flytepropeller.image.repository | string | `"cr.flyte.org/flyteorg/flytepropeller"` | Docker image for Flytepropeller deployment |
-| flytepropeller.image.tag | string | `"v1.13.0-rc0"` |  |
+| flytepropeller.image.tag | string | `"v1.13.0"` |  |
 | flytepropeller.manager | bool | `false` |  |
 | flytepropeller.nodeSelector | object | `{}` | nodeSelector for Flytepropeller deployment |
 | flytepropeller.podAnnotations | object | `{}` | Annotations for Flytepropeller pods |
@@ -272,7 +272,7 @@ helm install gateway bitnami/contour -n flyte
 | flytescheduler.configPath | string | `"/etc/flyte/config/*.yaml"` | Default regex string for searching configuration files |
 | flytescheduler.image.pullPolicy | string | `"IfNotPresent"` | Docker image pull policy |
 | flytescheduler.image.repository | string | `"cr.flyte.org/flyteorg/flytescheduler"` | Docker image for Flytescheduler deployment |
-| flytescheduler.image.tag | string | `"v1.13.0-rc0"` | Docker image tag |
+| flytescheduler.image.tag | string | `"v1.13.0"` | Docker image tag |
 | flytescheduler.nodeSelector | object | `{}` | nodeSelector for Flytescheduler deployment |
 | flytescheduler.podAnnotations | object | `{}` | Annotations for Flytescheduler pods |
 | flytescheduler.podEnv | object | `{}` | Additional Flytescheduler container environment variables |

diff --git a/charts/flyte-core/values.yaml b/charts/flyte-core/values.yaml
@@ -16,7 +16,7 @@ flyteadmin:
   image:
     # -- Docker image for Flyteadmin deployment
     repository: cr.flyte.org/flyteorg/flyteadmin  # FLYTEADMIN_IMAGE
-    tag: v1.13.0-rc0  # FLYTEADMIN_TAG
+    tag: v1.13.0  # FLYTEADMIN_TAG
     pullPolicy: IfNotPresent
   # -- Additional flyteadmin container environment variables
   #
@@ -144,7 +144,7 @@ flytescheduler:
     # -- Docker image for Flytescheduler deployment
     repository: cr.flyte.org/flyteorg/flytescheduler  # FLYTESCHEDULER_IMAGE
     # -- Docker image tag
-    tag: v1.13.0-rc0  # FLYTESCHEDULER_TAG
+    tag: v1.13.0  # FLYTESCHEDULER_TAG
     # -- Docker image pull policy
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flytescheduler deployment
@@ -210,7 +210,7 @@ datacatalog:
     # -- Docker image for Datacatalog deployment
     repository: cr.flyte.org/flyteorg/datacatalog  # DATACATALOG_IMAGE
     # -- Docker image tag
-    tag: v1.13.0-rc0  # DATACATALOG_TAG
+    tag: v1.13.0  # DATACATALOG_TAG
     # -- Docker image pull policy
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Datacatalog deployment
@@ -309,7 +309,7 @@ flytepropeller:
   image:
     # -- Docker image for Flytepropeller deployment
     repository: cr.flyte.org/flyteorg/flytepropeller  # FLYTEPROPELLER_IMAGE
-    tag: v1.13.0-rc0  # FLYTEPROPELLER_TAG
+    tag: v1.13.0  # FLYTEPROPELLER_TAG
     pullPolicy: IfNotPresent
   # -- Default resources requests and limits for Flytepropeller deployment
   resources:
@@ -805,7 +805,7 @@ configmap:
         # -- Structure documented [here](https://pkg.go.dev/github.com/lyft/flyteplugins@v0.5.28/go/tasks/pluginmachinery/flytek8s/config#FlyteCoPilotConfig)
         co-pilot:
           name: flyte-copilot-
-          image: cr.flyte.org/flyteorg/flytecopilot:v1.13.0-rc0  # FLYTECOPILOT_IMAGE
+          image: cr.flyte.org/flyteorg/flytecopilot:v1.13.0  # FLYTECOPILOT_IMAGE
           start-timeout: 30s
 
   # -- Core propeller configuration