cupy / cupy/cupy-release-tools

Improve `WHEEL_LINUX_CONFIGS` structure in `dist_config.py`

Open
#166 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10
Forks
13
Avg merge
50m
Merged PRs (30d)
4

Description

Currently, wheel config is defined like this:

```py
'10.0': {
'name': 'cupy-cuda100',
'kind': 'cuda',
'image': 'nvidia/cuda:10.0-devel-centos6',
'libs': [
],
'includes': [
],
'preloads': ['nccl', 'cudnn'],
'verify_image': 'nvidia/cuda:10.0-devel-{system}',
'verify_systems': ['ubuntu16.04'],
'system_packages': '',
},
```

I'd like to change to this:

```py
'cuda-10.0': {
'name': 'cupy-cuda100',
'platform': 'cuda',
'platform_version': '10.0',
'preloads': ['nccl', 'cudnn'],
'builder': {
'image': 'nvidia/cuda:10.0-devel-centos6',
'template': 'cuda', # template Dockerfile
'envs': {'CUPY_USE_CUDA_PYTHON': '0'},
'libs': [],
},
'verifier': {
'images': [
{'image': 'nvidia/cuda:10.0-devel-ubuntu16.04', 'template': 'cuda'}
]
},
},
```

* Add `cuda-*` prefix to configuration names
* Merge `WHEEL_LINUX_CONFIGS` and `SDIST_CONFIG`
* Allow specifying Dockerfile template for builder/verifier so that we can use different Dockerfile for builder/verifier (currently Dockerfiles are very hacky to handle CUDA/ROCm, CentOS 6/7 in one Dockerfile)
* Allow specifying environment variables for each build configuration (e.g., can be used to support CUDA Python, HIP platform list, etc.)
* Rename `kind` to `platform`
* Remove `includes` which is no longer used (originally used for ChainerX)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in dist_config.py by examining WHEEL_LINUX_CONFIGS and SDIST_CONFIG and how their fields are consumed. Trace the builder and verifier Dockerfile generation paths, then verify that the requested merged configuration supports platform metadata, separate templates, per-configuration environment variables, and the removal of includes.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
build-system, devops, release
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.