openedx / openedx/openedx-platform
Unpin elasticsearch and numpy in edx-platform
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.2k
- Forks
- 4.4k
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 42
Description
There is a pin for elasticsearch<7.14.0 in the common_constraints file but it's not clear if this is a problem specific to course-discovery or a problem with edx-platform as well. Looking at the changelog, the biggest difference with the newer versions of the elastichsearch client is that you have to match the python client version to the corresponding elasticsearch client version or it will fail to startup.
Looking at the PR that added this change, it's not clear what exactly was the error being seen. After speaking with Jeremy at 2U, it looks like 2U is constrained with running elasticsearch 7.10 or opensearch and the newer version of the elasticsearch python library will break with both of those versions which explains the pin.
In the meantime, we'll have to pin numpy<2.0.0 no not block other upgrades because the current pin of elasticsearch depends on a function in numpy that is dropped in numpy 2.0.0.
Once the elasticsearch python package is at version 7.15.0 or greater or it has been removed from the platform, the numpy pin in requirements/constraints.txt can be removed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with requirements/constraints.txt and the common_constraints file, then review the Elasticsearch changelog and PR 183 to establish the compatibility assumptions for Elasticsearch 7.10 or OpenSearch. Check the pinned versions against those deployments; done means the Elasticsearch pin can be safely relaxed or removed, and the numpy pin is removed once Elasticsearch is at least 7.15 or no longer used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, numpy, python
- Domain
- backend, databases, search
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100