litestar-org / litestar-org/pytest-databases
fix: serialize cross-service Docker port allocation under xdist
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 59
- Forks
- 11
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
`DockerService.run()` protects each service name with its own xdist file lock, but different services can still call `client.containers.run(... ports={container_port: None})` concurrently on separate workers.
On rootless Docker, one SQLSpec `pytest -n 4 --dist=loadgroup` run started several native pytest-databases services at once. Docker allocated adjacent host ports 33313-33322, while the MSSQL container failed on 33316:
```text
RootlessKit PortManager.AddPort(): listen tcp4 0.0.0.0:33316:
bind: address already in use
```
There was no second pytest process. The concurrent starts were different service names from the same run. Because `mssql_service` is session-scoped, the one startup failure cascaded into every MSSQL test.
Issue #131 / PR #133 added `host_port` support to `DockerService.run`, but only the PostgreSQL-family fixture surface exposes port overrides. MSSQL, MySQL, Oracle, GizmoSQL, CockroachDB, RustFS, and other native service fixtures still use dynamic allocation and cannot opt into a pinned port.
## Suggested fix
Serialize only the container-creation/port-allocation critical section across service names when xdist is active, using a global file lock under the pytest-databases temp directory. Keep the existing per-name lock for service reuse and lifecycle behavior.
Alternatively, expose an optional host-port fixture consistently for every Docker service, though a global creation lock is less configuration and protects all current and future services.
## Environment
- pytest-databases 0.19.0
- pytest-xdist 3.8.0
- docker-py 7.x
- Linux rootless Docker / RootlessKit
- Python 3.10
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at DockerService.run and trace the existing per-service xdist file lock and the native service fixtures that request dynamic ports. Preserve service reuse and lifecycle behavior while serializing container creation across service names when xdist is active. Validate with a pytest -n 4 --dist=loadgroup run covering MSSQL and other native Docker services.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- databases, devops, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100