Rebase Monitoring to 14.5.0 and update all dependencies #1072

aiyengar2 · 2021-04-01T03:03:33Z

First commit switches rancher-monitoring to a Git-based package for the next step.

Second commit creates a script that allows commit-by-commit interactive rebases on upstreams for Git-based packages.

Remaining commits were done via scripts in pairs:

First commit per pair creates generated-changes based on the existing base
Second commit per pair simply modifies the package.yaml to the new base and regenerates generated-changes on that base

Some fixups were added after the fact to each chart when I tested deploying them and found bugs from the rebase.

aiyengar2

Since this PR is huge and not reviewable in its original form, I have added 3 commits to make this process way easier to review:

Third To Last Commit

I ran:

a) PACKAGE=rancher-monitoring make charts without any changes

b) mv charts/rancher-monitoring/rancher-monitoring/<VERSION> charts/rancher-monitoring/rancher-monitoring/charts

c) mv charts/rancher-monitoring/rancher-monitoring-crd/<VERSION> charts/rancher-monitoring/rancher-monitoring-crd/charts

This represents the state of the chart today.

Second To Last Commit

Same steps as above once my changes were made.

This represents the chart that will be shipped in this commit.

Reviewers should review this commit for changes since this contains the final diff that will be shipped from this PR.

Last Commit

I removed the charts/ directory since we do not want to commit that.

Modifications

If any modifications are requested, I will add further commits to this PR instead of modifying these charts commits.

At the end of the PR, all of this will be squash and merged.

aiyengar2 · 2021-04-04T03:23:45Z

charts/rancher-monitoring/rancher-monitoring/charts/values.yaml

@@ -2486,18 +2771,6 @@ prometheus:
          runAsUser: 101
          runAsGroup: 101

-    # Additional volumes on the output StatefulSet definition.


I moved this back to the original location, shown above

aiyengar2 · 2021-04-04T03:45:00Z

charts/rancher-monitoring/rancher-monitoring-crd/charts/Chart.yaml

@@ -7,4 +7,4 @@ apiVersion: v1
 description: Installs the CRDs for rancher-monitoring.
 name: rancher-monitoring-crd
 type: application
-version: 9.4.204-rc09


Old chart versions are dropped in #1075

aiyengar2 · 2021-04-04T03:46:12Z

charts/rancher-monitoring/rancher-monitoring/charts/Chart.yaml

@@ -14,8 +14,63 @@ annotations:
  catalog.cattle.io/requests-cpu: 4500m
  catalog.cattle.io/requests-memory: 4000Mi
  catalog.cattle.io/ui-component: monitoring
-apiVersion: v1
-appVersion: 0.38.1
+apiVersion: v2


Since we've moved to apiVersion v2, the dependencies have all been moved to the Chart.yaml

charts/rancher-monitoring/rancher-monitoring/charts/Chart.yaml

aiyengar2 · 2021-04-04T03:48:00Z

charts/rancher-monitoring/rancher-monitoring/charts/charts/grafana/Chart.yaml

+  catalog.cattle.io/hidden: "true"
+  catalog.cattle.io/os: linux
+  catalog.rancher.io/certified: rancher
+  catalog.rancher.io/namespace: cattle-monitoring-system
+  catalog.rancher.io/release-name: rancher-grafana


These annotations were added since all dependent charts that track an upstream were split off into top-level charts. This chart needs to be hidden and is only supported by Linux.

aiyengar2 · 2021-04-04T03:52:02Z

charts/rancher-monitoring/rancher-monitoring/charts/charts/grafana/values.yaml

+  replicas: 1
+  image:
+    # image-renderer Image repository
+    repository: rancher/grafana-grafana-image-renderer


New image repository required and will be added in rancher/image-mirror#83

aiyengar2 · 2021-04-04T03:53:15Z

charts/rancher-monitoring/rancher-monitoring/charts/charts/kube-state-metrics/Chart.yaml

+annotations:
+  catalog.cattle.io/hidden: "true"
+  catalog.cattle.io/os: linux
+  catalog.rancher.io/certified: rancher
+  catalog.rancher.io/namespace: cattle-monitoring-system
+  catalog.rancher.io/release-name: rancher-kube-state-metrics


These annotations were added since all dependent charts that track an upstream were split off into top-level charts. This chart needs to be hidden and is only supported by Linux.

aiyengar2 · 2021-04-04T03:54:11Z

charts/rancher-monitoring/rancher-monitoring/charts/charts/prometheus-adapter/Chart.yaml

+annotations:
+  catalog.cattle.io/hidden: "true"
+  catalog.cattle.io/os: linux
+  catalog.rancher.io/certified: rancher
+  catalog.rancher.io/namespace: cattle-monitoring-system
+  catalog.rancher.io/release-name: rancher-prometheus-adapter


These annotations were added since all dependent charts that track an upstream were split off into top-level charts. This chart needs to be hidden and is only supported by Linux.

aiyengar2 · 2021-04-04T03:54:37Z

charts/rancher-monitoring/rancher-monitoring/charts/charts/prometheus-node-exporter/Chart.yaml

+annotations:
+  catalog.cattle.io/hidden: "true"
+  catalog.cattle.io/os: linux
+  catalog.rancher.io/certified: rancher
+  catalog.rancher.io/namespace: cattle-monitoring-system
+  catalog.rancher.io/release-name: rancher-node-exporter


These annotations were added since all dependent charts that track an upstream were split off into top-level charts. This chart needs to be hidden and is only supported by Linux.

aiyengar2 · 2021-04-04T03:55:14Z

charts/rancher-monitoring/rancher-monitoring/charts/requirements.yaml

@@ -1,145 +0,0 @@
-dependencies:


This was completely removed since we moved to apiVersion v2 charts

aiyengar2 · 2021-04-05T23:21:10Z

Tested deploying in a hardened cluster, a windows cluster, and a normal cluster and successfully deployed all components.

All images and repositories have been created as well.

Signed-off-by: Arvind Iyengar <arvind.iyengar@rancher.com>

aiyengar2 · 2021-04-05T23:30:41Z

Script has been moved to #1079.

prachidamle · 2021-04-07T00:16:42Z

@aiyengar2 In upgrade case, doe the deleted CRDs and the CRs get cleaned up? I see a couple of them deleted by the new version?

aiyengar2 · 2021-04-07T00:35:56Z

@aiyengar2 In upgrade case, doe the deleted CRDs and the CRs get cleaned up? I see a couple of them deleted by the new version?

I believe those should be cleaned up by the CRD chart upgrade

aiyengar2 · 2021-04-07T00:36:40Z

Also, I don't think the CRDs are being deleted, but rather renamed into different files that also have modifications. That's why they appear to show up as deletions

charts/rancher-monitoring/rancher-monitoring/charts/Chart.yaml

charts/rancher-monitoring/rancher-monitoring/charts/README.md

packages/rancher-grafana/generated-changes/exclude/ci/with-dashboard-json-values.yaml

aiyengar2 · 2021-04-07T18:30:04Z

Had to push another commit to patch the CRD version to 14.5.1 (for 14.5.100 to be exposed) as well

brendarearden

this was a huge rebase, well done!!! as far as I could tell everything looks good. You answered all of my questions and it looks like images, versions, and other port or chart.yaml changes were consistent and expected!

* Switch rancher-monitoring to Git-based package Signed-off-by: Arvind Iyengar <arvind.iyengar@rancher.com> * Move Monitoring dependencies to separate packages * Switch rancher-grafana to Git-based and add annotations * Switch rancher-kube-state-metrics to Git-based and add annotations * Switch rancher-prometheus-adapter to Git-based and add annotations * Switch rancher-node-exporter to Git-based and add annotations * Rebase to kube-prometheus-stack-10.0.0 * Update rancher-monitoring to new base kube-prometheus-stack-10.0.0 * Rebase to kube-prometheus-stack-11.0.0 * Update rancher-monitoring to new base kube-prometheus-stack-11.0.0 * Rebase to kube-prometheus-stack-12.0.0 * Update rancher-monitoring to new base kube-prometheus-stack-12.0.0 * Rebase to kube-prometheus-stack-13.0.0 * Update rancher-monitoring to new base kube-prometheus-stack-13.0.0 * Rebase to kube-prometheus-stack-14.0.0 * Update rancher-monitoring to new base kube-prometheus-stack-14.0.0 * Rebase to kube-prometheus-stack-14.5.0 * Update rancher-monitoring to new base kube-prometheus-stack-14.5.0 * fixup rancher-monitoring * Switch nginx port for Prometheus to 8081 This is necessary since prometheusConfigReloaderImage now takes port 8080 in the Prometheus pod due to a change introduced in Prometheus Operator v0.43.0 in prometheus-operator/prometheus-operator@dcf34c5. This value is hard-coded in the operator in https://github.com/prometheus-operator/prometheus-operator/blob/bcf61aacda782d6c8ffc79f55ea1def9cfd97933/pkg/operator/statefulset.go#L25. * Rebase to grafana-5.7.7 * Update rancher-grafana to new base grafana-5.7.7 * Rebase to grafana-6.0.0 * Update rancher-grafana to new base grafana-6.0.0 * Rebase to grafana-6.6.4 * Update rancher-grafana to new base grafana-6.6.4 * fixup rancher-grafana * Add linux annotations to image-renderer * Use default grafana serviceAccount if unavailable for image-renderer * Rebase to kube-state-metrics-helm-chart-2.13.1 * Update rancher-kube-state-metrics to new base kube-state-metrics-helm-chart-2.13.1 * fixup kube-state-metrics * Rebase to prometheus-node-exporter-1.16.2 * Update rancher-node-exporter to new base prometheus-node-exporter-1.16.2 * fixup node-exporter * Rebase to prometheus-adapter-2.12.1 * Update rancher-prometheus-adapter to new base prometheus-adapter-2.12.1 * fixup prometheus-adapter * Update CRD chart version * Add original chart before all commits were made * Add chart after all commits were made * Delete charts * Patch the chart version to 1 and reset packageVersion to 00 * Patch CRD chart version

aiyengar2 force-pushed the add_rebase_script branch 5 times, most recently from 2a28654 to a4b3b80 Compare April 3, 2021 02:35

aiyengar2 changed the title ~~Rebase Monitoring to 14.x.x and introduce new interactive rebase script~~ Rebase Monitoring to 14.5.0 and introduce new interactive rebase script Apr 3, 2021

aiyengar2 force-pushed the add_rebase_script branch from a4b3b80 to 060d3c7 Compare April 3, 2021 19:56

aiyengar2 changed the title ~~Rebase Monitoring to 14.5.0 and introduce new interactive rebase script~~ Rebase Monitoring to 14.5.0, update all dependencies, and introduce new interactive rebase script Apr 3, 2021

aiyengar2 force-pushed the add_rebase_script branch 3 times, most recently from 756f66a to 1f30dbf Compare April 4, 2021 02:07

aiyengar2 marked this pull request as ready for review April 4, 2021 03:11

This was referenced Apr 4, 2021

Add images for rebasing Monitoring rancher/image-mirror#83

Merged

[2.5] Drop all Monitoring 9.4.204 releaseCandidateVersions #1075

Closed

aiyengar2 commented Apr 4, 2021

View reviewed changes

aiyengar2 force-pushed the add_rebase_script branch 4 times, most recently from bc6a1db to 3cbb181 Compare April 5, 2021 23:12

aiyengar2 requested review from prachidamle and brendarearden April 5, 2021 23:21

Arvind Iyengar added 8 commits April 5, 2021 16:30

Switch rancher-monitoring to Git-based package

9b48b7a

Signed-off-by: Arvind Iyengar <arvind.iyengar@rancher.com>

Move Monitoring dependencies to separate packages

f7b3b02

Switch rancher-grafana to Git-based and add annotations

0930423

Switch rancher-kube-state-metrics to Git-based and add annotations

9a08473

Switch rancher-prometheus-adapter to Git-based and add annotations

7558257

Switch rancher-node-exporter to Git-based and add annotations

9c3bdc6

Rebase to kube-prometheus-stack-10.0.0

114dee3

Update rancher-monitoring to new base kube-prometheus-stack-10.0.0

4d52426

Arvind Iyengar added 5 commits April 5, 2021 16:30

fixup prometheus-adapter

fa37dfd

Update CRD chart version

c66a415

Add original chart before all commits were made

4edbcf6

Add chart after all commits were made

e78eff9

Delete charts

ce35c7b

aiyengar2 mentioned this pull request Apr 5, 2021

Add rebase script #1079

Closed

aiyengar2 force-pushed the add_rebase_script branch from 3cbb181 to ce35c7b Compare April 5, 2021 23:31

aiyengar2 changed the title ~~Rebase Monitoring to 14.5.0, update all dependencies, and introduce new interactive rebase script~~ Rebase Monitoring to 14.5.0 and update all dependencies Apr 5, 2021

aiyengar2 mentioned this pull request Apr 6, 2021

Add Cluster Overview dashboards #1086

Merged

prachidamle previously approved these changes Apr 7, 2021

View reviewed changes

brendarearden reviewed Apr 7, 2021

View reviewed changes

charts/rancher-monitoring/rancher-monitoring/charts/Chart.yaml Outdated Show resolved Hide resolved

brendarearden reviewed Apr 7, 2021

View reviewed changes

charts/rancher-monitoring/rancher-monitoring/charts/README.md Outdated Show resolved Hide resolved

brendarearden reviewed Apr 7, 2021

View reviewed changes

charts/rancher-monitoring/rancher-monitoring/charts/README.md Outdated Show resolved Hide resolved

Patch the chart version to 1 and reset packageVersion to 00

98659f5

aiyengar2 dismissed prachidamle’s stale review via 98659f5 April 7, 2021 18:02

aiyengar2 requested a review from brendarearden April 7, 2021 18:03

brendarearden reviewed Apr 7, 2021

View reviewed changes

packages/rancher-grafana/generated-changes/exclude/ci/with-dashboard-json-values.yaml Show resolved Hide resolved

aiyengar2 requested a review from brendarearden April 7, 2021 18:15

Patch CRD chart version

4c99ae2

brendarearden approved these changes Apr 7, 2021

View reviewed changes

Merge branch 'dev-v2.5-source' into add_rebase_script

0163b5e

aiyengar2 merged commit b88bdc6 into rancher:dev-v2.5-source Apr 7, 2021

aiyengar2 mentioned this pull request Jun 9, 2022

Add rebase script #1936

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rebase Monitoring to 14.5.0 and update all dependencies #1072

Rebase Monitoring to 14.5.0 and update all dependencies #1072

aiyengar2 commented Apr 1, 2021 •

edited

Loading

aiyengar2 left a comment •

edited

Loading

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 Apr 4, 2021

aiyengar2 commented Apr 5, 2021

aiyengar2 commented Apr 5, 2021

prachidamle commented Apr 7, 2021 •

edited

Loading

aiyengar2 commented Apr 7, 2021

aiyengar2 commented Apr 7, 2021

aiyengar2 commented Apr 7, 2021

brendarearden left a comment

Rebase Monitoring to 14.5.0 and update all dependencies #1072

Rebase Monitoring to 14.5.0 and update all dependencies #1072

Conversation

aiyengar2 commented Apr 1, 2021 • edited Loading

aiyengar2 left a comment • edited Loading

Choose a reason for hiding this comment

Third To Last Commit

Second To Last Commit

Last Commit

Modifications

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aiyengar2 commented Apr 5, 2021

aiyengar2 commented Apr 5, 2021

prachidamle commented Apr 7, 2021 • edited Loading

aiyengar2 commented Apr 7, 2021

aiyengar2 commented Apr 7, 2021

aiyengar2 commented Apr 7, 2021

brendarearden left a comment

Choose a reason for hiding this comment

aiyengar2 commented Apr 1, 2021 •

edited

Loading

aiyengar2 left a comment •

edited

Loading

prachidamle commented Apr 7, 2021 •

edited

Loading