Advanced StatefulSet Controller

Kubernetes has a built-inStatefulSetthat allocates consecutive serial numbers to Pods. For example, when there are three replicas, the Pods are named as pod-0, pod-1, and pod-2. When scaling out or scaling in, you must add a Pod at the end or delete the last pod. For example, when you scale out to four replicas, pod-3 is added. When you scale in to two replicas, pod-2 is deleted.

When you use local storage, Pods are associated with the Nodes storage resources and cannot be scheduled freely. If you want to delete one of the Pods in the middle to maintain its Node but no other Nodes can be migrated, or if you want to delete a Pod that fails and to create another Pod with a different serial number, you cannot implement such desired function by the built-in StatefulSet.

Theadvanced StatefulSet controlleris implemented based on the built-in StatefulSet controller. It supports freely controlling the serial number of Pods. This document describes how to use the advanced StatefulSet controller in TiDB Operator.

Enable

Load the Advanced StatefulSet CRD file:

For Kubernetes versions < 1.16:


                  
                   kubectl apply -f https://raw.githubusercontent.com/pingcap/tidb-operator/v1.5.0/manifests/advanced-statefulset-crd.v1beta1.yaml

For Kubernetes versions >= 1.16:


                  
                   kubectl apply -f https://raw.githubusercontent.com/pingcap/tidb-operator/v1.5.0/manifests/advanced-statefulset-crd.v1.yaml

Enable theAdvancedStatefulSetfeature invalues.yamlof the TiDB Operator chart:


                
                 features:
                 -
                 AdvancedStatefulSet=true
                 advancedStatefulset:
                 create:
                 true

Upgrade TiDB Operator. For details, refer toUpgrade TiDB Operator.

After upgrading TiDB Operator, check the AdvancedStatefulSet Controller is deployed by the following command:


                
                 kubectl get pods -n${operator-ns}--selector app.kubernetes.io/component=advanced-statefulset-controller

Expected output


                 
                  名字就绪状态RESTARTS AGE advanced-statefulset-controller-67885c5dd9-f522h 1/1 Running 0 10s

Note

If theAdvancedStatefulSetfeature is enabled, TiDB Operator converts the currentStatefulSetobject into anAdvancedStatefulSetobject. However, after theAdvancedStatefulSetfeature is disabled, theAdvancedStatefulSetobject cannot be automatically converted to the built-inStatefulSetKubernetes的对象。

Usage

This section describes how to use the advanced StatefulSet controller.

View the`AdvancedStatefulSet`Object by kubectl

The data format ofAdvancedStatefulSetis the same as that ofStatefulSet, butAdvancedStatefulSetis implemented based on CRD, withastsas the alias. You can view theAdvancedStatefulSetobject in the namespace by running the following command:


              
               kubectl get -n${namespace}asts

Specify the Pod to be scaled in

With the advanced StatefulSet controller, when scaling in TidbCluster, you can not only reduce the number of replicas, but also specify the scaling in of any Pod in the PD, TiDB, or TiKV components by configuring annotations.

对于穰mple:


              
               apiVersion:
               m.rzhenli.com/v1alpha1
               kind:
               TidbCluster
               metadata:
               name:
               asts
               spec:
               版本:
               v7.1.0
               timezone:
               UTC
               pvReclaimPolicy:
               删除
               pd:
               baseImage:
               pingcap/pd
               maxFailoverCount:
               0
               replicas:
               3
               requests:
               storage:
               "1Gi"
               config:{}tikv:
               baseImage:
               pingcap/tikv
               maxFailoverCount:
               0
               replicas:
               4
               requests:
               storage:
               "1Gi"
               config:{}tidb:
               baseImage:
               pingcap/tidb
               maxFailoverCount:
               0
               replicas:
               2
               service:
               type:
               ClusterIP
               config:{}

The above configuration deploys 4 TiKV instances, namelybasic-tikv-0,basic-tikv-1, ...,basic-tikv-3. If you want to deletebasic-tikv-1, setspec.tikv.replicasto3和configure the following annotations:


              
               metadata:
               annotations:
               tikv.tidb.m.rzhenli.com/delete-slots:
               '[1]'

Note

When modifyingreplicas和delete-slots annotation, complete the modification in the same operation; otherwise, the controller operates the modification according to the general expectations.

The complete example is as follows:


              
               apiVersion:
               m.rzhenli.com/v1alpha1
               kind:
               TidbCluster
               metadata:
               annotations:
               tikv.tidb.m.rzhenli.com/delete-slots:
               '[1]'
               name:
               asts
               spec:
               版本:
               v7.1.0
               timezone:
               UTC
               pvReclaimPolicy:
               删除
               pd:
               baseImage:
               pingcap/pd
               maxFailoverCount:
               0
               replicas:
               3
               requests:
               storage:
               "1Gi"
               config:{}tikv:
               baseImage:
               pingcap/tikv
               maxFailoverCount:
               0
               replicas:
               3
               requests:
               storage:
               "1Gi"
               config:{}tidb:
               baseImage:
               pingcap/tidb
               maxFailoverCount:
               0
               replicas:
               2
               service:
               type:
               ClusterIP
               config:{}

The supported annotations are as follows:

pd.tidb.m.rzhenli.com/delete-slots: Specifies the serial numbers of the Pods to be deleted in the PD component.
tidb.tidb.m.rzhenli.com/delete-slots: Specifies the serial number of the Pods to be deleted in the TiDB component.
tikv.tidb.m.rzhenli.com/delete-slots: Specifies the serial number of the Pods to be deleted in the TiKV component.

The value of Annotation is an integer array of JSON, such as[0],[0,1],(1、3).

Specify the location to scale out

You can reverse the above operation of scaling in to restorebasic-tikv-1.

Note

The specified scaling out performed by the advanced StatefulSet controller is the same as the regular StatefulSet scaling, which does not delete the Persistent Volume Claims (PVCs) associated with the Pod. If you want to avoid using the previous data, delete the associated PVCs before scaling out at the original location.

对于穰mple:


              
               apiVersion:
               m.rzhenli.com/v1alpha1
               kind:
               TidbCluster
               metadata:
               annotations:
               tikv.tidb.m.rzhenli.com/delete-slots:
               '[]'
               name:
               asts
               spec:
               版本:
               v7.1.0
               timezone:
               UTC
               pvReclaimPolicy:
               删除
               pd:
               baseImage:
               pingcap/pd
               maxFailoverCount:
               0
               replicas:
               3
               requests:
               storage:
               "1Gi"
               config:{}tikv:
               baseImage:
               pingcap/tikv
               maxFailoverCount:
               0
               replicas:
               4
               requests:
               storage:
               "1Gi"
               config:{}tidb:
               baseImage:
               pingcap/tidb
               maxFailoverCount:
               0
               replicas:
               2
               service:
               type:
               ClusterIP
               config:{}

The delete-slots annotations can be left empty or deleted completely.

Download PDF Request docs changes Ask questions on Discord

Playground

New

One-stop & interactive experience of TiDB's capabilities WITHOUT registration.

Advanced StatefulSet Controller

Enable

Usage

View theAdvancedStatefulSetObject by kubectl

Specify the Pod to be scaled in

Specify the location to scale out

View the`AdvancedStatefulSet`Object by kubectl