dask / dask/dask-gateway

Decision: allow Hadoop Yarn to be phased out due to missing maintenance capacity?

Open
#882 3 comments 0 reactions 0 assignees View on GitHub
codebase:dask-gateway-server maintenance
Dominant language
Python
Stars
148
Forks
93
PR merge metrics
No merged PRs in 30d

Description

I'd like to propose we stop attempting to support Hadoop Yarn in this project by:
- documenting it is being phased out unless maintenance effort beyond current capacity arrives, specifically maintenance around:
- maintenance of the dependency [`skein`](https://github.com/jcrist/skein) (https://github.com/jcrist/skein/issues/255)
- making a new CI testing image not based on CentOS 7, which no longer builds (https://github.com/dask/dask-gateway/issues/837)
- allowing it to be cut away from this project over time when we encounter issues maintaining functional tests etc.

As a first step, I'd like to merge https://github.com/conda-forge/dask-gateway-feedstock/pull/43 and drop the conda-forge feedstock's output `dask-gateway-server-yarn` that makes the entire feedstock not get upgraded to emit Python 3.13 outputs because its dependency `skein` isn't updated for Python 3.13.

_**I'm looking for a conclusion in this issue around whether it is okay to proceed with phasing out Hadoop Yarn as friction arises in maintenance while waiting for eventual maintenance capacity to arise.**_

/cc: @TomAugspurger @jacobtomlinson @martindurant @jcrist

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.