TiDB 5.4.1 Release Notes

Release Date: May 13, 2022

TiDB version: 5.4.1

Compatibility changes

TiDB v5.4.1 does not introduce any compatibility changes in product design. But note that bug fixes in this release might result in compatibility changes, too. For more information, seeBug Fixes.

Improvements

• TiDB

  • Support using the PointGet plan for queries that read the_tidb_rowidcolumn#31543
  • Add more logs and metrics for theApplyoperator to show whether it is parallel#33887
  • Improve theTopNpruning logic for Analyze Version 2 used for collecting statistics#34256
  • Support displaying multiple Kubernetes clusters in the Grafana dashboard#32593

• TiKV

  • Support displaying multiple Kubernetes clusters in the Grafana dashboard#12104

• PD

  • Support displaying multiple Kubernetes clusters in the Grafana dashboard#4673

• TiFlash

  • Support displaying multiple Kubernetes clusters in the Grafana dashboard#4129

• Tools

  • TiCDC

    • Support multiple Kubernetes clusters in Grafana dashboards#4665
    • Expose configuration parameters of the Kafka producer to make them configurable in TiCDC#4385

  • TiDB Data Migration (DM)

    • Support Syncer using the working directory of the DM-worker rather than/tmpto write internal files, and cleaning the directory after the task is stopped#4107

Bug Fixes

• TiDB

  • 解决的是sue thatdate_formatin TiDB handles'\n'in a MySQL-incompatible way#32232
  • 解决的是sue that TiDB writes wrong data due to the wrong encoding of theENUMorSETcolumn#32302
  • 解决的是sue that the Merge Join operator gets wrong results in certain cases#33042
  • 解决的是sue that TiDB gets a wrong result when a correlated subquery returns a constant#32089
  • 解决的是sue that TiDB gets the wrong result when using TiFlash to scan tables with empty range although TiFlash does not support reading tables with empty range yet#33083
  • 解决的是sue that theMAXorMIN函数on theENUMorSETcolumn returns a wrong result when the new collation is enabled in TiDB#31638
  • Fix a bug that CTE might be blocked when a query reports errors#31302
  • Fix wrong range calculation results for Nulleq function on Enum values#32428
  • Fix TiDB OOM when exporting data using ChunkRPC#31981#30880
  • Fix a bug thattidb_super_read_onlyis not automatically enabled whentidb_restricted_read_onlyis enabled#31745
  • 解决的是sue that thegreatestorleast函数with collation gets a wrong result#31789
  • Fix load data panic if the data is broken at an escape character#31589
  • Fix theinvalid transactionerror when executing a query using index lookup join#30468
  • Fix wrong results of deleting data of multiple tables usingleft join#31321
  • Fix a bug that TiDB may dispatch duplicate tasks to TiFlash#32814
  • 解决的是sue that granting theallprivilege might fail in clusters that are upgraded from v4.0#33588
  • 修复会话恐慌发生在执行the prepared statement after table schema change with the MySQL binary protocol#33509
  • 解决的是sue that executing SQL statements that have thecompress()expression withtidb_enable_vectorized_expressionenabled will fail#33397
  • 解决的是sue of high CPU usage by thereArrangeFallback函数#30353
  • 解决的是sue that the table attributes are not indexed when a new partition is added and the issue that the table range information is not updated when the partition changes#33929
  • Fix a bug that theTopNstatistical information of a table during the initialization is not correctly sorted#34216
  • Fix the error that occurs when reading from theINFORMATION_SCHEMA.ATTRIBUTEStable by skipping the unidentifiable table attributes#33665
  • Fix a bug that even if@@tidb_enable_parallel_applyis set, theApplyoperator is not paralleled when anorderproperty exists#34237
  • Fix a bug that'0000-00-00 00:00:00'can be inserted into adatetimecolumn whensql_modeis set toNO_ZERO_DATE#34099
  • 解决的是sue that the TiDB server might run out of memory when theINFORMATION_SCHEMA.CLUSTER_SLOW_QUERYtable is queried. This issue can be triggered when you check slow queries on the Grafana dashboard#33893
  • Fix a bug that in theNOWAITstatement, a transaction being executed does not return immediately when encountering a lock#32754
  • Fix a bug that causes a failure when creating a table with theGBKcharset andgbk_bincollation#31308
  • Fix a bug that whenenable-new-charsetison, creating aGBKcharset table with collation fails with the "Unknown character set" error#31297

• TiKV

  • 解决的是sue that TiKV panics and destroys peers unexpectedly because the target Region to be merged is invalid#12232
  • Fix a bug that stale messages cause TiKV to panic#12023
  • 解决的是sue of intermittent packet loss and out of memory (OOM) caused by the overflow of memory metrics#12160
  • Fix the potential panic issue that occurs when TiKV performs profiling on Ubuntu 18.04#9765
  • Fix a bug that replica reads might violate the linearizability#12109
  • Fix the TiKV panic issue that occurs when the target peer is replaced with the peer that is destroyed without being initialized when merging a Region#12048
  • Fix a bug that TiKV might panic if it has been running for 2 years or more#11940
  • Reduce the TiCDC recovery time by reducing the number of the Regions that require the Resolve Locks step#11993
  • Fix the panic issue caused by deleting snapshot files when the peer status isApplying#11746
  • 解决的是sue that destroying a peer might cause high latency#10210
  • Fix the panic issue caused by invalid assertion in resource metering#12234
  • 解决的是sue that slow score calculation is inaccurate in some corner cases#12254
  • Fix the OOM issue caused by theresolved_tsmodule and add more metrics#12159
  • 解决的是sue that successfully committed optimistic transactions may report theWrite Conflicterror when the network is poor#34066
  • Fix the TiKV panic issue that occurs when replica read is enabled on a poor network#12046

• PD

  • 解决的是sue thatDurationfields ofdr-autosynccannot be dynamically configured#4651
  • 解决的是sue that when there exists a Store with large capacity (2T for example), fully allocated small Stores cannot be detected, which results in no balance operator being generated#4805
  • 解决的是sue that the label distribution has residual labels in the metrics#4825

• TiFlash

  • Fix the panic issue that occurs when TLS is enabled#4196
  • Fix possible metadata corruption caused by Region merge on a lagging Region peer#4437
  • 解决的是sue that a query containingJOINmight be hung if an error occurs#4195
  • Fix a bug that MPP tasks might leak threads forever#4238
  • Fix the overflow that occurs when castingFLOATtoDECIMAL#3998
  • 解决的是sue that expired data is recycled slowly#4146
  • Fix a bug that canceled MPP queries might cause tasks to hang forever when the local tunnel is enabled#4229
  • 解决的是sue of memory leak that occurs when a query is canceled#4098
  • Fix the wrong result that occurs when castingDATETIMEtoDECIMAL#4151
  • Fix the potential issue of TiFlash panic whenSnapshotis applied simultaneously with multiple DDL operations#4072
  • Fix the bug that invalid storage directory configurations lead to unexpected behaviors#4093
  • Fix the bug that some exceptions are not handled properly#4101
  • 解决的是sue that castingINTtoDECIMALmight cause overflow#3920
  • 解决的是sue that the result ofINis incorrect in multi-value expressions#4016
  • 解决的是sue that the date format identifies'\n'as an invalid separator#4036
  • Fix the potential query error after adding columns under heavy read workload#3967
  • Fix the panic issue that occurs when the memory limit is enabled#3902
  • Fix potential data corruption in DTFiles#4778
  • Fix potential errors when querying on a table with many delete operations#4747
  • Fix a bug that TiFlash reports many "Keepalive watchdog fired" errors randomly#4192
  • Fix a bug that data not matching any region range remains on a TiFlash node#4414
  • Fix a bug that empty segments cannot be merged after GC#4511

• Tools

  • Backup & Restore (BR)

    • Fix a bug that causes the restore operation to fail when the encryption information is lost during backup retry#32423
    • 解决的是sue that BR fails to back up RawKV#32607
    • Fix duplicate primary keys when inserting a record into a table after incremental restoration#33596
    • Fix a bug that BR incremental restore returns errors mistakenly due to DDL jobs with empty query#33322
    • Fix the potential issue that Regions might be unevenly distributed after a restore operation is finished#31034
    • 解决的是sue that BR does not retry enough times when Regions are not consistent during restoration#33419
    • 解决的是sue that BR might panic occasionally when merging small files is enabled#33801
    • 解决的是sue that schedulers do not resume after BR or TiDB Lightning exits abnormally#33546

  • TiCDC

    • Fix incorrect metrics caused by owner changes#4774
    • Fix the TiCDC panic issue that might occur becauseCanal-JSONdoes not support nil#4736
    • 修复工作者池Unif所使用的稳定性问题ied Sorter#4447
    • Fix a bug that sequence is incorrectly replicated in some cases#4563
    • Fix the TiCDC panic issue that might occur whenCanal-JSONincorrectly handlesstring#4635
    • Fix a bug that a TiCDC node exits abnormally when a PD leader is killed#4248
    • Fix a bug that MySQL sink generates duplicatedreplaceSQL statements whenbatch-replace-enableis disabled#4501
    • Fix the DML construct error caused by therename tablesDDL#5059
    • 解决的是sue that in rare cases replication can be stuck if the owner is changed and the new scheduler is enabled (disabled by default)#4963
    • 解决的是sue that the error ErrProcessorDuplicateOperations is reported when the new scheduler is enabled (disabled by default)#4769
    • 解决的是sue that TiCDC fails to start when the first PD set in--pdis not available after TLS is enabled#4777
    • 解决的是sue that the checkpoint metrics are missing when tables are being scheduled#4714

  • TiDB Lightning

    • Fix the checksum error "GC life time is shorter than transaction duration"#32733
    • 解决的是sue that TiDB Lightning gets stuck when it fails to check empty tables#31797
    • Fix a bug that TiDB Lightning may not delete the metadata schema when some import tasks do not contain source files#28144
    • 解决的是sue that the precheck does not check local disk resources and cluster availability#34213

  • TiDB Data Migration (DM)

    • 解决的是sue that hundreds of "checkpoint has no change, skip sync flush checkpoint" print in the log and the replication is very slow#4619
    • Fix a bug that long varchars report an errorColumn length too big#4637
    • 解决的是sue that execution errors of the update statement in safemode may cause the DM-worker panic#4317
    • 解决的是sue that in some cases manually executing the filtered DDL in the downstream might cause task resumption failure#5272
    • Fix a bug that no data is returned for thequery-statuscommand when binlog is not enabled in the upstream#5121
    • Fix the DM worker panic issue that occurs when the primary key is not first in the index returned by theSHOW CREATE TABLEstatement#5159
    • 解决的是sue that CPU usage may increase and a large amount of log is printed when GTID is enabled or when the task is automatically resumed#5063