Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

a test issue from Neo to keep open #193

Open
neo-liang-sap opened this issue Aug 14, 2023 · 8 comments
Open

a test issue from Neo to keep open #193

neo-liang-sap opened this issue Aug 14, 2023 · 8 comments
Labels
component/dashboard Gardener Dashboard component/documentation Gardener Documentation component/gardener Gardener kind/bug Bug kind/post-mortem Bug that requires deeper analysis after immediate issues were resolved (usually after downtime) kind/regression Bug that hit us already in the past and that is reappearing/requires a proper solution os/garden-linux Related to Garden Linux OS os/suse-chost Related to SUSE Container Host OS platform/alicloud Alicloud platform/infrastructure platform/aws Amazon web services platform/infrastructure platform/azure Microsoft Azure platform/infrastructure platform/converged-cloud Converged Cloud (CC) platform/infrastructure platform/gcp Google cloud platform/infrastructure priority/normal Has no particular urgency status/new Issue is new and unprocessed topology/shoot Affects Shoot clusters

Comments

@neo-liang-sap
Copy link
Contributor

Which cluster is affected?

Cluster Details Dashboard Link: URL

What happened?

What you expected to happen?

When did it happen or started to happen?

Absolute:
Relative:

How would we reproduce it (concisely and precisely)?

Anything else we need to know?

Help us categorise this issue for faster resolution:

/area audit-logging auto-scaling backup certification control-plane cost delivery disaster-recovery documentation high-availability logging metering monitoring networking os performance quality security storage usability user-management
/component gardener dashboard documentation
/kind bug regression post-mortem
/os garden-linux suse-chost
/platform alicloud aws azure gcp converged-cloud

/priority normal
@neo-liang-sap neo-liang-sap added the topology/shoot Affects Shoot clusters label Aug 14, 2023
@gardener-robot-dev
Copy link

@neo-liang-sap No more than 5 labels permitted, but 21 labels were given.

@gardener-robot-dev gardener-robot-dev added component/dashboard Gardener Dashboard component/documentation Gardener Documentation component/gardener Gardener kind/bug Bug kind/post-mortem Bug that requires deeper analysis after immediate issues were resolved (usually after downtime) kind/regression Bug that hit us already in the past and that is reappearing/requires a proper solution os/garden-linux Related to Garden Linux OS os/suse-chost Related to SUSE Container Host OS platform/alicloud Alicloud platform/infrastructure platform/aws Amazon web services platform/infrastructure platform/azure Microsoft Azure platform/infrastructure platform/converged-cloud Converged Cloud (CC) platform/infrastructure platform/gcp Google cloud platform/infrastructure priority/normal Has no particular urgency labels Aug 14, 2023
@gardener-robot-dev
Copy link

@neo-liang-sap

Shoot: core/i312126-ha-test v1.26.5
            created at 2023-05-09 07:30 by Hongyan Zhang (BTP FP SRE (CHN))
            on aws in eu-west-1 with purpose development
            at 1 nodes and 28 pods and 35 API server requests/second (max of last 24h each)
Seed: garden/aws

🟢 Last Operation 
description: Shoot cluster has been successfully reconciled.
lastUpdateTime: '2023-08-14T00:07:14Z'
progress: 100
state: Succeeded
type: Reconcile
🟢 Shoot Conditions

          🟢 APIServerAvailable (HealthzRequestSucceeded)
          🟢 ControlPlaneHealthy (ControlPlaneRunning)
          🟢 ObservabilityComponentsHealthy (ObservabilityComponentsRunning)
          🟢 EveryNodeReady (EveryNodeReady)
          🟢 SystemComponentsHealthy (SystemComponentsRunning)
          🟢 HibernationPossible (NoProblematicWebhooks)
          🟢 MaintenancePreconditionsSatisfied (NoProblematicWebhooks)

🟢 Seed Conditions

          🟢 AuditlogServiceAvailability (AuditlogInstanceAttached)
          🟢 GardenletReady (GardenletReady)
          🟢 ExtensionsReady (AllExtensionsReady)
          🟢 BackupBucketsReady (BackupBucketsAvailable)
          🟢 SeedSystemComponentsHealthy (SystemComponentsRunning)

🟠 Control Plane Events Not Healthy
Timestamp Severity Reason Object Details
00:00:09 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-events-client		 Insufficient number of endpoints (0 endpoints, 3 zones), addressType: IPv4
00:00:09 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-main-client		 Insufficient number of endpoints (0 endpoints, 3 zones), addressType: IPv4
00:00:31 on
2023-08-14		 🟠 Warning Unhealthy pod/
etcd-events-0		 Readiness probe failed: command "/bin/sh -ec ETCDCTL_API=3 etcdctl --cacert=/var/etcd/ssl/client/ca/bundle.crt --cert=/var/etcd/ssl/client/client/tls.crt --key=/var/etcd/ssl/client/client/tls.key --endpoints=https://etcd-events-local:2379 get foo --consistency=l" timed out
00:00:33 on
2023-08-14		 🟠 Warning Unhealthy pod/
etcd-events-1		 Readiness probe failed: command "/bin/sh -ec ETCDCTL_API=3 etcdctl --cacert=/var/etcd/ssl/client/ca/bundle.crt --cert=/var/etcd/ssl/client/client/tls.crt --key=/var/etcd/ssl/client/client/tls.key --endpoints=https://etcd-events-local:2379 get foo --consistency=l" timed out
00:00:33 on
2023-08-14		 🟠 Warning Unhealthy pod/
etcd-main-2		 Readiness probe failed: command "/bin/sh -ec ETCDCTL_API=3 etcdctl --cacert=/var/etcd/ssl/client/ca/bundle.crt --cert=/var/etcd/ssl/client/client/tls.crt --key=/var/etcd/ssl/client/client/tls.key --endpoints=https://etcd-main-local:2379 get foo --consistency=l" timed out
00:00:34 on
2023-08-14		 🟠 Warning Unhealthy pod/
etcd-main-0		 Readiness probe failed: command "/bin/sh -ec ETCDCTL_API=3 etcdctl --cacert=/var/etcd/ssl/client/ca/bundle.crt --cert=/var/etcd/ssl/client/client/tls.crt --key=/var/etcd/ssl/client/client/tls.key --endpoints=https://etcd-main-local:2379 get foo --consistency=l" timed out
00:00:39 on
2023-08-14		 🟠 Warning Unhealthy pod/
etcd-events-2		 Readiness probe failed: command "/bin/sh -ec ETCDCTL_API=3 etcdctl --cacert=/var/etcd/ssl/client/ca/bundle.crt --cert=/var/etcd/ssl/client/client/tls.crt --key=/var/etcd/ssl/client/client/tls.key --endpoints=https://etcd-events-local:2379 get foo --consistency=l" timed out
00:00:42 on
2023-08-14		 🟠 Warning Unhealthy pod/
etcd-main-1		 Readiness probe failed: command "/bin/sh -ec ETCDCTL_API=3 etcdctl --cacert=/var/etcd/ssl/client/ca/bundle.crt --cert=/var/etcd/ssl/client/client/tls.crt --key=/var/etcd/ssl/client/client/tls.key --endpoints=https://etcd-main-local:2379 get foo --consistency=l" timed out
00:01:29 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-main-client		 Insufficient number of endpoints (1 endpoints, 3 zones), addressType: IPv4
00:01:30 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-events-client		 Insufficient number of endpoints (1 endpoints, 3 zones), addressType: IPv4
00:01:33 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-events-client		 Insufficient number of endpoints (2 endpoints, 3 zones), addressType: IPv4
00:01:33 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-main-client		 Insufficient number of endpoints (2 endpoints, 3 zones), addressType: IPv4
00:01:45 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
kube-apiserver		 Insufficient number of endpoints (0 endpoints, 3 zones), addressType: IPv4
00:02:02 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-events-client		 Unable to allocate minimum required endpoints to each zone without exceeding overload threshold (3 endpoints, 3 zones), addressType: IPv4
00:02:02 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
etcd-main-client		 Unable to allocate minimum required endpoints to each zone without exceeding overload threshold (3 endpoints, 3 zones), addressType: IPv4
00:02:06 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
kube-apiserver		 Insufficient number of endpoints (1 endpoints, 3 zones), addressType: IPv4
00:02:16 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
kube-apiserver		 Insufficient number of endpoints (2 endpoints, 3 zones), addressType: IPv4
00:02:41 on
2023-08-14		 🟠 Warning FailedGetResourceMetric horizontalpodautoscaler/
kube-apiserver-p87bk		 failed to get cpu utilization: did not receive metrics for any ready pods
00:02:41 on
2023-08-14		 🟠 Warning FailedComputeMetricsReplicas horizontalpodautoscaler/
kube-apiserver-p87bk		 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for any ready pods
00:02:42 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
kube-apiserver		 Unable to allocate minimum required endpoints to each zone without exceeding overload threshold (3 endpoints, 3 zones), addressType: IPv4
00:03:15 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
gardener-resource-manager		 Insufficient number of endpoints (0 endpoints, 3 zones), addressType: IPv4
00:03:19 on
2023-08-14		 🟠 Warning BackOff pod/
kube-controller-manager-b9cb5b5c-l6kgv		 Back-off restarting failed container kube-controller-manager in pod kube-controller-manager-b9cb5b5c-l6kgv_shoot--core--i312126-ha-test(12047520-68f1-462b-a39b-155fb590e24e)
00:03:35 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
gardener-resource-manager		 Insufficient number of endpoints (1 endpoints, 3 zones), addressType: IPv4
00:03:36 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
gardener-resource-manager		 Insufficient number of endpoints (2 endpoints, 3 zones), addressType: IPv4
00:03:37 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
vpa-webhook		 Insufficient number of endpoints (0 endpoints, 3 zones), addressType: IPv4
00:03:39 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
vpa-webhook		 Insufficient number of endpoints (1 endpoints, 3 zones), addressType: IPv4
00:03:39 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
vpa-webhook		 Insufficient number of endpoints (2 endpoints, 3 zones), addressType: IPv4
00:03:40 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
csi-snapshot-validation		 Insufficient number of endpoints (0 endpoints, 3 zones), addressType: IPv4
00:03:42 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
csi-snapshot-validation		 Insufficient number of endpoints (1 endpoints, 3 zones), addressType: IPv4
00:03:43 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
csi-snapshot-validation		 Insufficient number of endpoints (2 endpoints, 3 zones), addressType: IPv4
00:03:44 on
2023-08-14		 🟠 Warning FailedMount pod/
vali-0		 `MountVolume.MountDevice failed for volume "pv-shoot-garden-aws-2c75c15a-fcd5-41d9-9fa7-f1072e526c45" : rpc error: code = Internal desc = Could not determine if volume "vol-08a9d0a87cf273c33" ("/dev/nvme8n1") need to be resized: failed to read size of filesystem on /dev/nvme8n1: exit status 1: dumpe2fs 1.42.9 (28-Dec-2013)
          dumpe2fs: Filesystem has unsupported feature(s) while trying to open /dev/nvme8n1
          Couldn't find valid filesystem superblock.
          `
00:03:46 on
2023-08-14		 🟠 Warning TopologyAwareHintsDisabled service/
gardener-resource-manager		 Unable to allocate minimum required endpoints to each zone without exceeding overload threshold (3 endpoints, 3 zones), addressType: IPv4
00:04:20 on
2023-08-14		 🟠 Warning Unhealthy pod/
aws-custom-route-controller-c5bd6f9fb-5x4b7		 Liveness probe failed: HTTP probe failed with statuscode: 500
00:04:24 on
2023-08-14		 🟠 Warning FailedMount pod/
vali-0		 `MountVolume.MountDevice failed for volume "pv-shoot-garden-aws-2c75c15a-fcd5-41d9-9fa7-f1072e526c45" : rpc error: code = Internal desc = Could not determine if volume "vol-08a9d0a87cf273c33" ("/dev/nvme8n1") need to be resized: failed to read size of filesystem on /dev/nvme8n1: exit status 1: dumpe2fs 1.42.9 (28-Dec-2013)
          dumpe2fs: Filesystem has unsupported feature(s) while trying to open /dev/nvme8n1
          Couldn't find valid filesystem superblock.
          `
00:04:59 on
2023-08-14		 🟠 Warning Unhealthy pod/
aws-custom-route-controller-c5bd6f9fb-2d9vt		 Liveness probe failed: HTTP probe failed with statuscode: 500
00:05:21 on
2023-08-14		 🟠 Warning Unhealthy pod/
aws-custom-route-controller-c5bd6f9fb-2d9vt		 Readiness probe failed: Get "http://10.243.138.162:10259/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
🟢 Worker Pools
Name OS Machine Zones
worker-fvfgh
1:5 -0 +1		 gardenlinux
v934.9.0		 m5.large
50Gi gp3		 eu-west-1a
eu-west-1b
🟠 Worker Events Not Healthy
Timestamp Severity Reason Object Details
11:00:41 on
2023-08-11		 🟠 Warning FailedToUpdateEndpoint endpoints/
kube-dns-upstream		 Failed to update endpoint kube-system/kube-dns-upstream: Operation cannot be fulfilled on endpoints "kube-dns-upstream": the object has been modified; please apply your changes to the latest version and try again
11:00:41 on
2023-08-11		 🟠 Warning FailedToUpdateEndpoint endpoints/
kube-prometheus-stack-coredns		 Failed to update endpoint kube-system/kube-prometheus-stack-coredns: Operation cannot be fulfilled on endpoints "kube-prometheus-stack-coredns": the object has been modified; please apply your changes to the latest version and try again
11:00:41 on
2023-08-11		 🟠 Warning FailedToUpdateEndpoint endpoints/
metrics-server		 Failed to update endpoint kube-system/metrics-server: Operation cannot be fulfilled on endpoints "metrics-server": the object has been modified; please apply your changes to the latest version and try again
11:00:43 on
2023-08-11		 🟠 Warning Unhealthy pod/
apiserver-proxy-zqwrg		 Readiness probe failed: Get "http://10.180.29.156:16910/ready": dial tcp 10.180.29.156:16910: connect: connection refused
11:00:49 on
2023-08-11		 🟠 Warning FailedGetResourceMetric horizontalpodautoscaler/
coredns		 failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io)
11:00:49 on
2023-08-11		 🟠 Warning FailedComputeMetricsReplicas horizontalpodautoscaler/
coredns		 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io)
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
blackbox-exporter-5868c644cd-mczqg		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
blackbox-exporter-5868c644cd-z5fhq		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-node-vertical-autoscaler-7bbd54698f-cbv4f		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-typha-deploy-767cff6988-pznhc		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-typha-horizontal-autoscaler-7b89d5ff97-qpwkj		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-typha-vertical-autoscaler-656479b7b5-5lstt		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
coredns-6477b5dd7d-gr9dh		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
coredns-6477b5dd7d-r7tzr		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
metrics-server-5b5587d858-7f5rz		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
metrics-server-5b5587d858-rstk4		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
vpn-shoot-0		 no nodes available to schedule pods
00:03:39 on
2023-08-14		 🟠 Warning FailedScheduling pod/
vpn-shoot-1		 no nodes available to schedule pods
00:03:51 on
2023-08-14		 🟠 Warning FailedScheduling pod/
vpn-shoot-1		 skip schedule deleting pod: kube-system/vpn-shoot-1
00:03:51 on
2023-08-14		 🟠 Warning FailedScheduling pod/
vpn-shoot-1		 no nodes available to schedule pods
00:03:52 on
2023-08-14		 🟠 Warning FailedScheduling pod/
metrics-server-65bc64b849-f5r6l		 no nodes available to schedule pods
00:04:21 on
2023-08-14		 🟠 Warning FailedGetResourceMetric horizontalpodautoscaler/
coredns		 failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io)
00:04:21 on
2023-08-14		 🟠 Warning FailedComputeMetricsReplicas horizontalpodautoscaler/
coredns		 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io)
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
blackbox-exporter-5868c644cd-mczqg		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
blackbox-exporter-5868c644cd-z5fhq		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-node-vertical-autoscaler-7bbd54698f-cbv4f		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-typha-horizontal-autoscaler-7b89d5ff97-qpwkj		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
calico-typha-vertical-autoscaler-656479b7b5-5lstt		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
coredns-6477b5dd7d-gr9dh		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
coredns-6477b5dd7d-r7tzr		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
metrics-server-5b5587d858-7f5rz		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
metrics-server-5b5587d858-rstk4		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
metrics-server-65bc64b849-f5r6l		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
vpn-shoot-0		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:25 on
2023-08-14		 🟠 Warning FailedScheduling pod/
vpn-shoot-1		 0/1 nodes are available: 1 node(s) had untolerated taint {node.gardener.cloud/critical-components-not-ready: }. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
00:05:26 on
2023-08-14		 🟠 Warning FailedMount pod/
kube-proxy-worker-fvfgh-v1.26.5-4n84v		 MountVolume.SetUp failed for volume "kube-proxy-mode" : open /var/lib/kube-proxy/mode: no such file or directory
00:05:26 on
2023-08-14		 🟠 Warning NetworkNotReady pod/
network-problem-detector-pod-s45hl		 network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
00:05:26 on
2023-08-14		 🟠 Warning NetworkNotReady pod/
node-problem-detector-qqzl7		 network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initialized
00:06:01 on
2023-08-14		 🟠 Warning FailedCreatePodSandBox pod/
network-problem-detector-pod-s45hl		 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "0d3889d10a1208db274bdb6cf29ffa50937dd697a9765343d9f5021ab74f1c86": plugin type="calico" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/
00:06:01 on
2023-08-14		 🟠 Warning FailedCreatePodSandBox pod/
node-problem-detector-qqzl7		 Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "b0f2ff874cc614f29f02076fa45ad0b1b43a4d53ef72a5862690be277ec25401": plugin type="calico" failed (add): stat /var/lib/calico/nodename: no such file or directory: check that the calico/node container is running and has mounted /var/lib/calico/
00:06:46 on
2023-08-14		 🟠 Warning Unhealthy pod/
metrics-server-5b5587d858-7f5rz		 Readiness probe failed: HTTP probe failed with statuscode: 500
00:06:46 on
2023-08-14		 🟠 Warning Unhealthy pod/
metrics-server-5b5587d858-7f5rz		 Readiness probe failed: Get "https://100.64.0.13:8443/readyz": dial tcp 100.64.0.13:8443: connect: connection refused
00:06:51 on
2023-08-14		 🟠 Warning FailedGetResourceMetric horizontalpodautoscaler/
coredns		 failed to get cpu utilization: did not receive metrics for any ready pods
00:06:51 on
2023-08-14		 🟠 Warning FailedComputeMetricsReplicas horizontalpodautoscaler/
coredns		 invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for any ready pods
00:06:56 on
2023-08-14		 🟠 Warning Unhealthy pod/
metrics-server-5b5587d858-4smwm		 Readiness probe failed: HTTP probe failed with statuscode: 500
00:06:56 on
2023-08-14		 🟠 Warning Unhealthy pod/
metrics-server-5b5587d858-4smwm		 Readiness probe failed: Get "https://100.64.0.24:8443/readyz": dial tcp 100.64.0.24:8443: connect: connection refused

@gardener-robot-dev gardener-robot-dev added the status/new Issue is new and unprocessed label Aug 14, 2023
@gardener-robot-dev
Copy link

@gardener-robot-dev You have mentioned internal references in the public. Please check.

@gardener-robot-dev
Copy link

This issue has not been touched since 3 work days. Please add a follow up comment and/or change the status/ label.

@gardener-robot-dev
Copy link

This issue has not been touched since 8 work days. Please add a follow up comment and/or change the status/ label.

@gardener-robot-dev
Copy link

@bd3lage This issue has not been touched since 85 work days. Please add a follow up comment and/or change the status/ label.

@gardener-robot-dev
Copy link

@dguendisch, @DockToFuture This issue has not been touched since 87 work days. Please add a follow up comment and/or change the status/ label.

@gardener-robot-dev
Copy link

@bd3lage This issue has not been touched since 204 work days. Please add a follow up comment and/or change the status/ label.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component/dashboard Gardener Dashboard component/documentation Gardener Documentation component/gardener Gardener kind/bug Bug kind/post-mortem Bug that requires deeper analysis after immediate issues were resolved (usually after downtime) kind/regression Bug that hit us already in the past and that is reappearing/requires a proper solution os/garden-linux Related to Garden Linux OS os/suse-chost Related to SUSE Container Host OS platform/alicloud Alicloud platform/infrastructure platform/aws Amazon web services platform/infrastructure platform/azure Microsoft Azure platform/infrastructure platform/converged-cloud Converged Cloud (CC) platform/infrastructure platform/gcp Google cloud platform/infrastructure priority/normal Has no particular urgency status/new Issue is new and unprocessed topology/shoot Affects Shoot clusters
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@neo-liang-sap @gardener-robot-dev