Can not download defaultVersion.jar in Ray
- Dominant language
- Java
- Stars
- 808
- Forks
- 188
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 2
Description
**Describe the bug**
When submitting a job to Ray via geaflow-console, the `defaultVersion.jar` cannot be downloaded.
There are two issues:
1. **URL parsing in Ray's `packaging.py` excludes query parameters**:
The download URL provided by GeaFlow is in the format:
`https://127.0.0.1:8443/api/tasks/%s/files?path=/tmp/geaflow/files/versions/defaultVersion/defaultVersion.jar.zip`
However, Ray's default implementation in `packaging.py` strips the query part when generating the cache key, resulting in a key like:
`https_127_0_0_1_8443_api_tasks_%s_files`
In contrast, GeaFlow's `RayRuntime` generates a key that includes the query parameters (with special characters replaced), resulting in:
`https_127_0_0_1_8443_api_tasks_%s_files?path=_tmp_geaflow_files_versions_defaultVersion_defaultVersion_jar`
This inconsistency causes a mismatch in `runtime_env`, preventing proper reuse of cached files.
2. **Authentication issue**:
Accessing the console's download endpoint requires a valid `geaflow-token` header, but this token is not passed to Ray during the file download process, resulting in unauthorized access and download failure.
**Expected behavior**
The `defaultVersion.jar` should be downloaded successfully and the job should run normally.
**Additional context**
The above issue is not verified by hdfs or oss, and may have the same problem
Contributor guide
Research direction
Start by tracing the GeaFlow RayRuntime download setup and comparing its cache-key handling with Ray's packaging.py. Then follow the geaflow-token requirement for the console download endpoint; done means defaultVersion.jar downloads successfully and the submitted job runs normally, including the noted HDFS or OSS cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, python
- Domain
- authentication, backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100