project-codeflare / project-codeflare/codeflare-sdk
job.status and job.log returns error for certain domain name pattern
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 35
- Forks
- 66
- Avg merge
- 48m
- Merged PRs (30d)
- 7
Description
The job.status() and job.logs() Are failing, they're showing
gaierror Traceback (most recent call last)
File /opt/app-root/lib64/python3.8/site-packages/urllib3/connection.py:174, in HTTPConnection._new_conn(self)
173 try:
--> 174 conn = connection.create_connection(
175 (self._dns_host, self.port), self.timeout, **extra_kw
176 )
178 except SocketTimeout:
....
ConnectionError: Failed to connect to Ray at address: http://ray/.
Cause:
In torchx The app_handle is a string. On my cluster it looks like
ray://torchx/ray-dashboard-mnisttest-default.tedchang-codeflare-test-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.us-south.containers.appdomain.cloud-mnisttest-trwmgjb2s041jc
This method uses a regx that not parse the app_id(mnisttest-trwmgjb2s041jc) and ray dashboard(ray-dashboard-mnisttest-default.tedchang-codeflare-test-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.us-south.containers.appdomain.cloud) property
":\d+-|.com-|.org-|.net-|.gov-|.io-"
/opt/app-root/lib64/python3.8/site-packages/torchx/schedulers/ray_scheduler.py
330 def _parse_app_id(self, app_id: str) -> Tuple[str, str]:
331 # find index of '-' in the first :\d+-
332 m = re.search(r":\d+-|.com-|.org-|.net-|.gov-|.io-" , app_id)
333 if m:
334 sep = m.span()[1]
335 addr = app_id[: sep-1]
336 app_id = app_id[sep:]
337 return addr, app_id
338 ~~~~~~~~~~~~~~~~
339 addr, _, app_id = app_id.partition("-")
340 return addr, app_id
something like this worked for my cluster but may not be robust
":\d+-|.com-|.org-|.net-|.gov-|.io-|.cloud-"
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in torchx/schedulers/ray_scheduler.py at _parse_app_id, then trace how job.status() and job.logs() use the parsed address and application ID. Reproduce the failure with the shown cloud hostname pattern and verify both operations connect successfully without the Ray connection error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100