fix(statefulset): fix maxUnavailable for rolling upgrades not taking …

[Yesphet]

…into account pods that fail later in the updateIndexes

0. Backgroud

Advanced statefulset provides MaxUnavailable guarantee that the number of unavailable pods during the update cannot exceed this value. However, in the current implementation code, there is no consideration for unavailable pods with smaller ordinal or later in the update order. For example, for an asts with maxUnavailable= 1 and replicas=3, if the asts updates while pod-0 is in a crash, it will still trigger a rolling upgrade, and 2 pods will be unavailable because pod-2 is updated and restarted.

It can be easily reproduced in this way：

1. create a asts

apiVersion: apps.kruise.io/v1beta1
kind: StatefulSet
metadata:
  name: test
spec:
  selector:
    matchLabels:
      app: busybox-test
  replicas: 3
  podManagementPolicy: Parallel
  updateStrategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  template:
    metadata:
      labels:
        app: busybox-test
    spec:
      containers:
      - name: busybox
        image: busybox
        env:
        - name: MY_POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: Restart
          value: "2"
        command: ["sh","-c"]
        args:
        - |-
          [ "$MY_POD_NAME" == "test-0" ] && exit 1;tail -f /dev/null

2. List pods, test-0 is in crash

3. Update an env of the container in asts to trigger rolling update

kubectl patch asts test --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/env/1/value", "value":"3"}]'

4. We can see that test-2 is deleted which breaking the maxUnavailable guarantee

I have tested k8s official statefulset with MaxUnavailable feature gate enabled, it can still guarantee that the number of unavailable pods does not exceed in this scenario.

Ⅰ. Describe what this PR does

So in this PR, I separate out the update pods logics to a new function rollingUpdateStatefulsetPods, counts all unavailable pods before update pods.

Ⅱ. Does this pull request fix one issue?

no

Ⅲ. Describe how to verify it

refer to part 0

Ⅳ. Special notes for reviews

In k8s official statefulsets controller, if the unavailable pods count reach the MaxUnavailable limit, the rolling update won't make progress even if the first pod in the update sequence is unavailable. But in my implementation, rolling updates are still process in this case because update one unavailable pod does not increase the total number of unavailable Pods.

[kruise-bot]

Welcome @Yesphet! It looks like this is your first PR to openkruise/kruise 🎉

[codecov]

Codecov Report

Attention: 23 lines in your changes are missing coverage. Please review.

Comparison is base (fa7a1da) 47.66% compared to head (46e4e41) 48.41%.
Report is 4 commits behind head on master.

Files	Patch %	Lines
pkg/controller/statefulset/stateful_set_control.go	66.66%	17 Missing and 6 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1480      +/-   ##
==========================================
+ Coverage   47.66%   48.41%   +0.75%     
==========================================
  Files         157      157              
  Lines       22427    22460      +33     
==========================================
+ Hits        10689    10875     +186     
+ Misses      10570    10391     -179     
- Partials     1168     1194      +26     

Flag	Coverage Δ
unittests	48.41% <66.66%> (+0.75%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Yesphet]

@veophi @Fei-Guo Could you please help to review this PR when you are free?

[Yesphet]

@veophi @Fei-Guo @zmberg It's been two weeks, and I still haven't received any feedback on this PR. I understand that everyone is busy, I just want to ensure this PR hasn't been overlooked. Your time and attention to it would be much appreciated ~

[furykerry]

sorry, guys are fully occupied these days, i will take a look at this patch.

[furykerry]

/lgtm

[zmberg]

@Yesphet good.

[zmberg]

/approve

[kruise-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zmberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [zmberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Yesphet]

@furykerry @zmberg thanks guys~