matrixorigin / matrixorigin/matrixone
[Bug]: ISCP index consumer deadlocks after downstream SQL execution fails
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Summary
`IndexConsumer.Consume` can deadlock permanently after the downstream index SQL executor returns an error. The execution goroutine reports the error and exits, but the producer continues flushing to an unbuffered `sqlBufSendCh` with no remaining receiver.
This affects asynchronous index maintenance, including FULLTEXT and IVF_FLAT; HNSW uses the same producer/channel pattern with a different consumer loop.
## Version
Commit: `7f7de555c`
## Code path
`pkg/iscp/index_consumer.go`:
```go
func (c *IndexConsumer) sendSql(writer IndexSqlWriter) error {
// ...
c.sqlBufSendCh <- sql
writer.Reset()
return nil
}
```
The send does not select on `ctx.Done()` or the error channel. In `runIndex`, an `ExecWithResult` error is sent to `errch` and the goroutine returns. The producer does not read `errch` while generating data, so its next `sendSql` blocks forever.
There is a second error-propagation bug at the end of `sinkTail`:
```go
c.flushCdc()
return nil
```
Any final flush error is discarded.
## White-box reproduction
A focused test used a writer that flushes after every row and an `ExecWithResult` stub that fails the first SQL with `injected index SQL failure`:
1. The producer sends row 1.
2. `runIndex` receives it, the executor returns the injected error, `runIndex` writes `errch` and exits.
3. The producer processes row 2 and blocks sending to `sqlBufSendCh` because no receiver exists.
The test runs `Consume` in a goroutine and requires the injected error within two seconds.
## Actual result
```text
Consume deadlocked after the SQL executor returned an error
```
The original executor error is never returned to the caller.
## Expected result
- A downstream execution error must cancel/stop the producer and be returned by `Consume` promptly.
- Channel sends must select on context cancellation and consumer failure.
- `sinkTail` must return `flushCdc()` errors.
- Closing or abandoning the channel must not leave either goroutine blocked.
## Suggested verification
Cover execution failure on the first and middle batch for snapshot and tail consumers, final-flush failure, context cancellation during a blocked send, and the HNSW consumer variant.
Contributor guide
Assessment
This issue has not been assessed yet.