sometimes submit job failed.
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 2.7k
- Forks
- 554
- Avg merge
- 20h 42m
- Merged PRs (30d)
- 14
Description
Organization Name:HIT
Short summary about the issue/question:
sometimes submit job failed.
Brief what process you are following:
when I submit ad job, it occurs error as follow:
[Exit Trigger Info]
ExitTriggerMessage: FailedTaskCount 1 has reached MinFailedTaskCount 1 in the TaskRole
ExitTriggerTaskRole: taskrole
ExitTriggerTaskIndex: 0
--------------------------------------------------------------------------------
[Exit Spec]
code: 1
phrase: PAIRuntimeExitAbnormally
issuer: PAI_RUNTIME
causer: PAI_RUNTIME
type: PLATFORM_FAILURE
stage: UNKNOWN
behavior: UNKNOWN
reaction: RETRY_TO_MAX
reason: 'PAI Runtime exit abnormally with undefined exitcode, it may have bugs'
repro:
- PAI Runtime exits with exitcode 1
solution:
- Contact PAI Dev to fix PAI Runtime bugs
--------------------------------------------------------------------------------
[Exit Diagnostics]
Pod failed: PodPattern unmatched:
containers:
- name: init
reason: Completed
code: 0
- name: app
reason: Error
message: >
standard_init_linux.go:228: exec user process caused: no such file or
directory
code: 1
what 's the reason about this error? I need help, thanks!
PLATFORM_FAILURE
How to reproduce it:
submit a new job.
OpenPAI Environment:
-
OpenPAI version: v1.8.0
-
Cloud provider or hardware configuration:
-
OS (e.g. from /etc/os-release):
-
Kernel (e.g.
uname -a): Linux rsgpuserver154 4.15.0-166-generic 174-Ubuntu SMP Wed Dec 8 19:07:44 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux -
Hardware (e.g. core number, memory size, storage size, GPU type etc.): A40
-
Others:
Anything else we need to know:
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with docs/user/troubleshooting_job.md and the job-debugging section of docs/job_tutorial.md, then reproduce the failure by submitting a new job. Use the reported pod diagnostics, especially the app container's exit message, to identify the cause; done means documenting or fixing the confirmed cause and verifying that a job submits successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, linux
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100