openedx / openedx/openedx-platform

Unpin elasticsearch and numpy in edx-platform

Open
#35,126 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.2k
Forks
4.4k
Avg merge
6d 18h
Merged PRs (30d)
42

Description

There is a pin for elasticsearch<7.14.0 in the common_constraints file but it's not clear if this is a problem specific to course-discovery or a problem with edx-platform as well. Looking at the changelog, the biggest difference with the newer versions of the elastichsearch client is that you have to match the python client version to the corresponding elasticsearch client version or it will fail to startup.

Looking at the PR that added this change, it's not clear what exactly was the error being seen. After speaking with Jeremy at 2U, it looks like 2U is constrained with running elasticsearch 7.10 or opensearch and the newer version of the elasticsearch python library will break with both of those versions which explains the pin.

image

In the meantime, we'll have to pin numpy<2.0.0 no not block other upgrades because the current pin of elasticsearch depends on a function in numpy that is dropped in numpy 2.0.0.

Once the elasticsearch python package is at version 7.15.0 or greater or it has been removed from the platform, the numpy pin in requirements/constraints.txt can be removed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with requirements/constraints.txt and the common_constraints file, then review the Elasticsearch changelog and PR 183 to establish the compatibility assumptions for Elasticsearch 7.10 or OpenSearch. Check the pinned versions against those deployments; done means the Elasticsearch pin can be safely relaxed or removed, and the numpy pin is removed once Elasticsearch is at least 7.15 or no longer used.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch, numpy, python
Domain
backend, databases, search
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.