`zfs` and `chown` subprocesses calls are failing to `internal error` in ocluster-worker
Nobody has claimed this yet.
- Dominant language
- OCaml
- Stars
- 42
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Description
Noticed in https://github.com/tarides/infrastructure/issues/375#issuecomment-2329726772
ocluster-worker: internal error, uncaught exception:
Failure("\"zfs\" \"create\" \"--\" \"obuilder/state\" failed with exit status 1")
2024-09-04 20:58.56 worker [INFO] Prune threshold not set and docker max df size is not. Will not check for low disk-space!
cannot open 'obuilder/cache-tmp': dataset does not exist
cannot open 'obuilder/state': dataset does not exist
2024-09-04 20:58.56 application [INFO] Exec "zfs" "create" "--" "obuilder/state"
cannot create 'obuilder/state': no such pool 'obuilder'
ocluster-worker: internal error, uncaught exception:
Failure("\"zfs\" \"create\" \"--\" \"obuilder/state\" failed with exit status 1")
2024-09-04 20:59.06 worker [INFO] Prune threshold not set and docker max df size is not. Will not check for low disk-space!
2024-09-04 20:59.06 application [INFO] Exec "zfs" "destroy" "-R" "-f" "--" "obuilder/cache-tmp"
2024-09-04 20:59.07 application [INFO] Exec "chown" "0:0" "/Volumes/obuilder/state"
chown: /Volumes/obuilder/state: No such file or directory
ocluster-worker: internal error, uncaught exception:
Failure("\"chown\" \"0:0\" \"/Volumes/obuilder/state\" failed with exit status 1")
ocluster-worker: internal error, uncaught exception:
Failure("\"chown\" \"0:0\" \"/Volumes/obuilder/state\" failed with exit status 1")
administrator@m1-worker-03 ~ % grep -A 2 'internal error' ./ocluster.log | tail -n 10
Failure("\"zfs\" \"create\" \"--\" \"obuilder/state\" failed with exit status 1")
I suspect this may be causing, contributing to, or masking problems that lead the macos builders to stop being able to build jobs. But proper error handling for these sub process calls should be put in place regardless.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the ocluster-worker subprocess handling for the failing zfs and chown calls, using the reported logs as the reproduction context. Determine how these non-zero exits are currently propagated and define the expected handling for missing datasets or paths; done means the failures no longer surface as uncaught internal errors and the relevant behavior is covered by tests, if existing tests are found.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, ocaml
- Domain
- devops, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100