Lightning replace conflict resolution fails with Error 1390: Prepared statement contains too many placeholders in v8.5.4
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
Please answer these questions before submitting your issue. Thanks!
### 1. Minimal reproduce step (Required)
When importing with TiDB Lightning (conflict.strategy = "replace"), the import reaches 100% data restore but fails during post-processing with:
insert data conflict record for conflict detection 'replace' mode failed: Error 1390 (HY000): Prepared statement contains too many placeholders
This happened on TiDB v8.5.4 with ~1.88M conflict records.
Progress reached 100% (tables/chunks/engines = 100%)
Then failed in post-processing:
restore all tables data failed
run failed
the whole procedure failed
Error stack points to:
[errormanager.go] in ReplaceConflictKeys conflict-record insert path
Warning also shows:
Detect 1885595 conflict errors in total ... conflict_view
Environment
TiDB Lightning: v8.5.4
Conflict strategy: replace
Large dataset (~1.385 TiB restored, billions of rows)
In replace conflict handling, conflict rows are inserted with a single dynamically built [INSERT ... VALUES (...)]statement that can grow too large and exceed placeholder limits (65535).
Additionally, in ReplaceConflictKeys, buffered insertRows can keep accumulating across loop iterations, making the SQL even larger.
### 2. What did you expect to see? (Required)
Lightning should split conflict-record inserts into safe batches and finish post-processing without placeholder overflow.
### 3. What did you see instead (Required)
Lightning fails in post-processing with Error 1390, and the whole task reports failure even though data restore reached 100%.
### 4. What is your TiDB version? (Required)
v8.5.4
Contributor guide
Research direction
Start in errormanager.go at ReplaceConflictKeys and trace the conflict-record insert path used by TiDB Lightning's replace strategy. Check how insertRows is buffered across loop iterations and how the dynamically built INSERT statement is formed. Done means conflict records are inserted in safe batches and post-processing completes without Error 1390.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100