apache / apache/geaflow

Can not download defaultVersion.jar in Ray

Open
#590 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
808
Forks
188
Avg merge
3d 22h
Merged PRs (30d)
2

Description

**Describe the bug**
When submitting a job to Ray via geaflow-console, the `defaultVersion.jar` cannot be downloaded.

There are two issues:

1. **URL parsing in Ray's `packaging.py` excludes query parameters**:
The download URL provided by GeaFlow is in the format:
`https://127.0.0.1:8443/api/tasks/%s/files?path=/tmp/geaflow/files/versions/defaultVersion/defaultVersion.jar.zip`
However, Ray's default implementation in `packaging.py` strips the query part when generating the cache key, resulting in a key like:
`https_127_0_0_1_8443_api_tasks_%s_files`
In contrast, GeaFlow's `RayRuntime` generates a key that includes the query parameters (with special characters replaced), resulting in:
`https_127_0_0_1_8443_api_tasks_%s_files?path=_tmp_geaflow_files_versions_defaultVersion_defaultVersion_jar`
This inconsistency causes a mismatch in `runtime_env`, preventing proper reuse of cached files.

2. **Authentication issue**:
Accessing the console's download endpoint requires a valid `geaflow-token` header, but this token is not passed to Ray during the file download process, resulting in unauthorized access and download failure.

**Expected behavior**
The `defaultVersion.jar` should be downloaded successfully and the job should run normally.

**Additional context**

Image

Image

The above issue is not verified by hdfs or oss, and may have the same problem

Contributor guide

Open the contributing guide

Research direction

Start by tracing the GeaFlow RayRuntime download setup and comparing its cache-key handling with Ray's packaging.py. Then follow the geaflow-token requirement for the console download endpoint; done means defaultVersion.jar downloads successfully and the submitted job runs normally, including the noted HDFS or OSS cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, python
Domain
authentication, backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.