datastore.Iterator leaks database/Firestore resources on decode errors
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 364
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 84
Description
Title
MySQL/Firestore datastore.Iterator leaks connections on decode error
Description
pkg/datastore/datastore.go(Iteratorinterface has noClose()),pkg/datastore/mysql/iterator.go:38-52List()calls across the datastore layer can return early on a row/document decode error without releasing the underlyingsql.Rows/Firestore iterator.- No
SetMaxOpenConnsis configured, so repeated failures can accumulate unreleased database resources.ListNotCompletedDeploymentsand similar methods are polled continuously by every piped instance, meaning a single malformed/incompatible row could eventually exhaust available MySQL connections and impact datastore operations across the control plane. - This is distinct from the related but narrower open issue [#6699](https://github.com/pipe-cd/pipecd/issues/6699), which concerns a swallowed
json.Unmarshalerror in the same MySQL iterator but does not address iterator/resource cleanup.
Suggested fix
Add Close() error to the datastore.Iterator interface and ensure every caller closes the iterator immediately after obtaining it:
iterator, err := ...
if err != nil {
return err
}
defer iterator.Close()
This ensures the underlying database/Firestore iterator is released both on successful iteration and when iteration exits early because of a decode error.
Verification
The MySQL implementation can return from iteration when row decoding fails without closing the underlying sql.Rows. The Firestore implementation should be verified for the same lifecycle issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Iterator interface in pkg/datastore/datastore.go and the MySQL implementation in pkg/datastore/mysql/iterator.go:38-52. Inspect List() callers across the datastore layer and verify the Firestore implementation for the same lifecycle issue. Done means every obtained iterator is closed on success and decode-error paths, with no underlying sql.Rows or Firestore iterator left unreleased.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, mysql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100