Use Resource Control to Achieve Resource Isolation

Note

This feature is not available onTiDB Serverless clusters.

As a cluster administrator, you can use the resource control feature to create resource groups, set quotas for resource groups, and bind users to those groups.

The TiDB resource control feature provides two layers of resource management capabilities: the flow control capability at the TiDB layer and the priority scheduling capability at the TiKV layer. The two capabilities can be enabled separately or simultaneously. See theParameters for resource controlfor details. This allows the TiDB layer to control the flow of user read and write requests based on the quotas set for the resource groups, and allows the TiKV layer to schedule the requests based on the priority mapped to the read and write quota. By doing this, you can ensure resource isolation for your applications and meet quality of service (QoS) requirements.

TiDB flow control: TiDB flow control uses thetoken bucket algorithm. If there are not enough tokens in a bucket, and the resource group does not specify theBURSTABLEoption, the requests to the resource group will wait for the token bucket to backfill the tokens and retry. The retry might fail due to timeout.
TiKV scheduling: You can set the absolute priority(PRIORITY)根据需要。不同的资源将雅高集团ding to thePRIORITYsetting. Tasks with highPRIORITYare scheduled first. If you do not set the absolute priority, TiKV uses the value ofRU_PER_SECof each resource group to determine the priority of the read and write requests for each resource group. Based on the priorities, the storage layer uses the priority queue to schedule and process requests.

Scenarios for resource control

The introduction of the resource control feature is a milestone for TiDB. It can divide a distributed database cluster into multiple logical units. Even if an individual unit overuses resources, it does not crowd out the resources needed by other units.

With this feature, you can:

Combine multiple small and medium-sized applications from different systems into a single TiDB cluster. When the workload of an application grows larger, it does not affect the normal operation of other applications. When the system workload is low, busy applications can still be allocated the required system resources even if they exceed the set quotas, so as to achieve the maximum utilization of resources.
Choose to combine all test environments into a single TiDB cluster, or group the batch tasks that consume more resources into a single resource group. It can improve hardware utilization and reduce operating costs while ensuring that critical applications can always get the necessary resources.
当there are mixed workloads in a system, you can put different workloads into separate resource groups. By using the resource control feature, you can ensure that the response time of transactional applications is not affected by data analysis or batch applications.
当集群中遇到意外SQL性能ormance issue, you can use SQL bindings along with resource groups to temporarily limit the resource consumption of a SQL statement.

In addition, the rational use of the resource control feature can reduce the number of clusters, ease the difficulty of operation and maintenance, and save management costs.

Limitations

Currently, the resource control feature has the following limitations:

This feature only supports flow control and scheduling of read and write requests initiated by foreground clients. It does not support flow control and scheduling of background tasks such as DDL operations and auto analyze.
Resource control incurs additional scheduling overhead. Therefore, there might be a slight performance degradation when this feature is enabled.

What is Request Unit (RU)

Request Unit (RU) is a unified abstraction unit in TiDB for system resources, which currently includes CPU, IOPS, and IO bandwidth metrics. It is used to indicate the amount of resources consumed by a single request to the database. The number of RUs consumed by a request depends on a variety of factors, such as the type of operations, and the amount of data being queried or modified. Currently, the RU contains consumption statistics for the resources in the following table:

Resource type	RU consumption
Read	2 storage read batches consume 1 RU
	8 storage read requests consume 1 RU
	64 KiB read request payload consumes 1 RU
Write	1 storage write batch consumes 1 RU for each replica
	1 storage write request consumes 1 RU
	1 KiB write request payload consumes 1 RU
SQL CPU	3 ms consumes 1 RU

Note

Each write operation is eventually replicated to all replicas (by default, TiKV has 3 replicas). Each replication operation is considered a different write operation.
In addition to queries executed by users, RU can be consumed by background tasks, such as automatic statistics collection.
The preceding table lists only the resources involved in RU calculation for TiDB Self-Hosted clusters, excluding the network and storage consumption. For TiDB Serverless RUs, seeTiDB Serverless Pricing Details.

Estimate RU consumption of SQL statements

You can use theEXPLAIN ANALYZEstatement to get the amount of RUs consumed during SQL execution. Note that the amount of RUs is affected by the cache (for example,coprocessor cache). When the same SQL is executed multiple times, the amount of RUs consumed by each execution might be different. The RU value does not represent the exact value for each execution, but can be used as a reference for estimation.

Parameters for resource control

The resource control feature introduces two new global variables.

TiDB: you can use thetidb_enable_resource_controlsystem variable to control whether to enable flow control for resource groups.

TiKV: you can use theresource-control.enabledparameter to control whether to use request scheduling based on resource groups.

TiKV: For TiDB Self-Hosted, you can use theresource-control.enabledparameter to control whether to use request scheduling based on resource group quotas. For TiDB Cloud, the value of theresource-control.enabledparameter istrueby default and does not support dynamic modification.

Starting from TiDB v7.0.0, both parameters are enabled by default. The results of the combinations of these two parameters are shown in the following table.

`resource-control.enabled`	`tidb_enable_resource_control`= ON	`tidb_enable_resource_control`= OFF
`resource-control.enabled`= true	Flow control and scheduling (recommended)	Invalid combination
`resource-control.enabled`= false	Only flow control (not recommended)	The feature is disabled.

For more information about the resource control mechanism and parameters, seeRFC: Global Resource Control in TiDB.

How to use resource control

This section describes how to use the resource control feature to manage resource groups and control the resource allocation of each resource group.

Estimate cluster capacity

Before resource planning, you need to know the overall capacity of the cluster. TiDB provides the statementCALIBRATE RESOURCEto estimate the cluster capacity. You can use one of the following methods:

You can view theResource Manager pagein TiDB Dashboard. For more information, seeCALIBRATE RESOURCE.

对于TiDB自托管,您可以使用CALIBRATE RESOURCEstatement to estimate the cluster capacity.

For TiDB Cloud, theCALIBRATE RESOURCEstatement is inapplicable.

Manage resource groups

To create, modify, or delete a resource group, you need to have theSUPERorRESOURCE_GROUP_ADMINprivilege.

You can create a resource group for a cluster by using创建RESOURCE GROUP.

For an existing resource group, you can modify theRU_PER_SECoption (the rate of RU backfilling per second) of the resource group by usingALTER RESOURCE GROUP. The changes to the resource group take effect immediately.

You can delete a resource group by usingDROP RESOURCE GROUP.

Create a resource group

The following is an example of how to create a resource group.

Create a resource grouprg1. The resource limit is 500 RUs per second and allows applications in this resource group to overrun resources.


                
                 创建RESOURCEGROUPIFNOT
                 EXISTSrg1 RU_PER_SEC=
                 500BURSTABLE;

Create a resource grouprg2. The RU backfill rate is 600 RUs per second and does not allow applications in this resource group to overrun resources.


                
                 创建RESOURCEGROUPIFNOT
                 EXISTSrg2 RU_PER_SEC=
                 600;

Create a resource grouprg3with the absolute priority set toHIGH. The absolute priority currently supportsLOW|MEDIUM|HIGH. The default value isMEDIUM.


                
                 创建RESOURCEGROUPIFNOT
                 EXISTSrg3 RU_PER_SEC=
                 100PRIORITY=HIGH;

Bind resource groups

TiDB supports three levels of resource group settings as follows.

User level. Bind a user to a specific resource group via the创建USERorALTER USERstatement. After a user is bound to a resource group, sessions created by the user are automatically bound to the corresponding resource group.
Session level. Set the resource group for the current session viaSET RESOURCE GROUP.
Statement level. Set the resource group for the current statement viaRESOURCE_GROUP()Optimizer Hint.

Bind users to a resource group

The following example creates a userusr1and binds the user to the resource grouprg1.rg1is the resource group created in the example inCreate Resource Group.


              
               创建
               USER
               'usr1'@'%'IDENTIFIEDBY
               '123'RESOURCEGROUPrg1;

The following example usesALTER USERto bind the userusr2to the resource grouprg2.rg2is the resource group created in the example inCreate Resource Group.


              
               ALTER
               USERusr2 RESOURCEGROUPrg2;

After you bind users, the resource consumption of newly created sessions will be controlled by the specified quota (Request Unit, RU). If the system workload is relatively high and there is no spare capacity, the resource consumption rate ofusr2will be strictly controlled not to exceed the quota. Becauseusr1is bound byrg1withBURSTABLEconfigured, the consumption rate ofusr1is allowed to exceed the quota.

If there are too many requests that result in insufficient resources for the resource group, the client's requests will wait. If the wait time is too long, the requests will report an error.

Note

当you bind a user to a resource group by using创建USERorALTER USER, it will not take effect for the user's existing sessions, but only for the user's new sessions.
TiDB automatically creates adefaultresource group during cluster initialization. For this resource group, the default value ofRU_PER_SECisUNLIMITED(equivalent to the maximum value of theINTtype, that is,2147483647) and it is inBURSTABLEmode. Statements that are not bound to a resource group are automatically bound to this resource group. This resource group does not support deletion, but you can modify the configuration of its RU.

Bind the current session to a resource group

By binding a session to a resource group, the resource usage of the corresponding session is limited by the specified usage (RU).

The following example binds the current session to the resource grouprg1.


              
               SETRESOURCEGROUPrg1;

Bind the current statement to a resource group

By adding theRESOURCE_GROUP(resource_group_name)提示一个SQL语句,您可以指定resource group to which the statement is bound. This hint supportsSELECT,INSERT,UPDATE, andDELETEstatements.

The following example binds the current statement to the resource grouprg1.


              
               SELECT
               /*+ RESOURCE_GROUP(rg1) */
               *
               FROMt limit10;

Disable resource control

Execute the following statement to disable the resource control feature.


                
                 SET
                 GLOBALtidb_enable_resource_control=
                 'OFF';

Set the TiKV parameterresource-control.enabledtofalseto disable scheduling based on the RU of the resource group.

Execute the following statement to disable the resource control feature.


                
                 SET
                 GLOBALtidb_enable_resource_control=
                 'OFF';

对于TiDB自托管,您可以使用resource-control.enabledparameter to control whether to use request scheduling based on resource group quotas. For TiDB Cloud, the value of theresource-control.enabledparameter istrueby default and does not support dynamic modification. If you need to disable it for TiDB Dedicated clusters, contactTiDB Cloud Support.

Monitoring metrics and charts

TiDB regularly collects runtime information about resource control and provides visual charts of the metrics in Grafana'sTiDB>Resource Controldashboard. The metrics are detailed in theResource Controlsection ofTiDB Important Monitoring Metrics.

TiKV also records the request QPS from different resource groups. For more details, seeTiKV Monitoring Metrics Detail.

You can view the data of resource groups in the currentRESOURCE_GROUPStable in TiDB Dashboard. For more details, seeResource Manager page.

Note

This section is only applicable to TiDB Self-Hosted. Currently, TiDB Cloud does not provide resource control metrics.

TiDB regularly collects runtime information about resource control and provides visual charts of the metrics in Grafana'sTiDB>Resource Controldashboard.

TiKV also records the request QPS from different resource groups in Grafana'sTiKVdashboard.

Tool compatibility

The resource control feature does not impact the regular usage of data import, export, and other replication tools. BR, TiDB Lightning, and TiCDC do not currently support processing DDL operations related to resource control, and their resource consumption is not limited by resource control.

FAQ

Do I have to disable resource control if I don't want to use resource groups?
No. Users who do not specify any resource groups will be bound to thedefaultresource group that has unlimited resources. When all users belong to thedefaultresource group, the resource allocation method is the same as when the resource control is disabled.
Can a database user be bound to several resource groups?
No. A database user can only be bound to one resource group. However, during the session runtime, you can useSET RESOURCE GROUPto set the resource group used by the current session. You can also use the optimizer hintRESOURCE_GROUP()to set the resource group for the running statement.
What happens when the total resource allocation (RU_PER_SEC) of all resource groups exceeds the system capacity?
TiDB does not verify the capacity when you create a resource group. As long as the system has enough available resources, TiDB can meet the resource requirements of each resource group. When the system resources exceed the limit, TiDB prioritizes satisfying requests from resource groups with higher priority. If requests with the same priority cannot all be met, TiDB allocates resources proportionally according to the resource allocation (RU_PER_SEC).

Use Resource Control to Achieve Resource Isolation

Scenarios for resource control

Limitations

What is Request Unit (RU)

Estimate RU consumption of SQL statements

Parameters for resource control

How to use resource control

Estimate cluster capacity

Manage resource groups

Create a resource group

Bind resource groups

Bind users to a resource group

Bind the current session to a resource group

Bind the current statement to a resource group

Disable resource control

Monitoring metrics and charts

Tool compatibility

FAQ

See also