elastic / elastic/rally

Preserve Rally dependency constraints when installing track dependencies

Open
#2,180 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

:Packaging bug dependencies
Dominant language
Python
Stars
2k
Forks
348
Avg merge
6d 17h
Merged PRs (30d)
5

Description

Rally installs track dependencies with pip install --upgrade --target ~/.rally/libs. This can upgrade transitive dependencies independently of Rally’s pinned environment.

For example, the elastic/security track installs geneve, which depends on anyio without any constraints. Two days ago, anyio 4.15.0 was released, and that version imports sentinel from typing_extensions, while Python resolves Rally’s pinned typing_extensions==4.12.2, which only provides Sentinel. (This is a recent breaking change in typing extensions.) The resulting mix of packages from ~/.rally/libs and Rally’s virtualenv fails at runtime:

File ".../.rally/libs/anyio/abc/_streams.py", line 8, in <module>
from .._core._typedattr import TypedAttributeProvider

File ".../.rally/libs/anyio/_core/_typedattr.py", line 10, in <module>
from typing_extensions import sentinel

ImportError: cannot import name 'sentinel' from 'typing_extensions' (.../rally/.venv/lib/python3.13/site-packages/typing_extensions.py)

(Notice how anyio comes from .rally/libs, while typing_extensions comes from .rally/.venv.)

Track dependency installation should use Rally’s resolved environment as a pip constraints file. This would allow track-specific packages to be installed or upgraded while forcing shared dependencies such as elasticsearch, anyio, and typing_extensions to versions compatible with Rally.

As a short-term mitigation, bumping Rally’s pinned dependencies, particularly typing_extensions, would restore compatibility with the newer anyio. However, without applying Rally’s constraints during track dependency installation, similar incompatibilities can recur whenever a track introduces or upgrades a shared transitive dependency.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the track dependency installation path that runs pip install --upgrade --target ~/.rally/libs, and review how Rally’s resolved environment can be supplied as a pip constraints file. Done means track-specific packages can still be installed or upgraded while shared dependencies such as elasticsearch, anyio, and typing_extensions remain compatible with Rally.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.