opensafely-core / opensafely-core/backend-server
Support ad-hoc docker execution in the VM's docker
Nobody has claimed this yet.
- Dominant language
- Shell
- Stars
- 1
- Forks
- 5
- Avg merge
- 1h 36m
- Merged PRs (30d)
- 2
Description
The docker service in the Ubuntu VM is used by job-runner and can execute jobs submitted by job-runner.
However, some Bennett staff (researchers and tech team) needs to sometimes manually run docker. The tooling and documentation for this is based on using Docker for Windows.
So, we have kept Docker for Windows (with a much smaller resource allocation) around to continue to support this workflow.
This is not ideal:
a) Docker for Windows doesn't always come up after a reboot.
b) So that researchers can release outputs from these notebooks, we have to run hacky sync script from Windows host into the VM so that the TPP level 4 sync script can move them to level 4.
c) Would be nice to remove Docker for Windows and its resource usage.
The long term solution probably involves sqlrunner, but it's likely we'll still need an ad-hoc query mechanism.
We could point the docker client on the Windows host to the docker service running on the VM, perhaps over SSH auth. This would mean all users would need SSH set up.
However, while this would work for executing docker containers, we may need some trickery to mount workspaces in.
Another option is to just move all the tooling to require working over SSH in the VM. That might be reasonble, but add some additional friction to an already awkward process for researchers using this.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the VM's Docker service, its job-runner usage, and the linked notebook tooling and documentation. Compare the proposed host-to-VM Docker access and SSH-based workflow, including workspace mounts; done should provide ad-hoc container execution without Docker for Windows and its sync workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, ubuntu
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100