Concurrency issue between rename partitioned table and `applyTable`

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

2. What did you expect to see? (Required)

3. What did you see instead (Required)

https://ci.pingcap.net/blue/organizations/jenkins/tiflash-ghpr-integration-tests/detail/tiflash-ghpr-integration-tests/16561/pipeline/

fullstack-test2-logs.tar.gz

[2024-07-12T04:46:15.000Z] fullstack-test2/ddl/rename_table_across_databases.test: Running
[2024-07-12T04:46:33.020Z]   File: fullstack-test2/ddl/rename_table_across_databases.test
[2024-07-12T04:46:33.020Z]   Error line: 117
[2024-07-12T04:46:33.020Z]   Error: set session tidb_isolation_read_engines='tiflash'; select * from test_new.part4 order by id;
[2024-07-12T04:46:33.020Z]   Result:
[2024-07-12T04:46:33.020Z]     ERROR 1105 (HY000) at line 1: other error for mpp stream: Code: 107, e.displayText() = DB::Exception: Cannot open file /tmp/tiflash/data/db/metadata/db_708/t_713.sql, errno: 2, strerror: No such file or directory, e.what() = DB::Exception,
[2024-07-12T04:46:33.020Z]   Expected:
[2024-07-12T04:46:33.020Z]     +----+----------+------+
[2024-07-12T04:46:33.020Z]     | id | store_id | c1   |
[2024-07-12T04:46:33.020Z]     +----+----------+------+
[2024-07-12T04:46:33.020Z]     |  1 |        1 | NULL |
[2024-07-12T04:46:33.020Z]     |  2 |        2 | NULL |
[2024-07-12T04:46:33.020Z]     |  3 |        3 | NULL |
[2024-07-12T04:46:33.020Z]     | 11 |       11 | NULL |
[2024-07-12T04:46:33.020Z]     | 16 |       16 | NULL |
[2024-07-12T04:46:33.020Z]     +----+----------+------+

4. What is your TiFlash version? (Required)

release-7.5

Start with fullstack-test2/ddl/rename_table_across_databases.test, especially the failure at line 117, and review the attached logs for the missing metadata file. Trace the concurrent partitioned-table rename and applyTable operations. Done means the test query succeeds and no missing metadata-file error occurs.