lablup / lablup/backend.ai

Per-user dotfiles

Open
#98 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
15h 13m
Merged PRs (30d)
368

Description

There are many customer requests to preserve the home directory contents, mainly user-installed packages.

The problem is, since our home directories are bind-mounted from each agent's scratch directory, they are not present in Docker images even we perform `docker commit` against running sessions. The technical reasons to use scratch directories are: 1) better I/O performance, 2) agent customization of home directory before starting containers, and 3) enforcement of quota limits.

Another problem is that some customer sites do not allow use of per-user vfolders due to their organizational policy, though we are offering auto-mounts of dot-prefixed per-user vfolders (e.g., `.local` and `.ssh`) if present to all sessions (#57) as an alternative to the above problem.

**This issue tackles the second problem: per-user dotfiles.**

For small configuration files aka dotfiles, we could just keep them in the user database and pass them to the agent whenever creating a new container, like SSH keypairs stored in the user database (lablup/backend.ai-manager#211).This will be exposed as "user settings" in the GUI and CLI.

- [x] manager: Add a blob column `dotfiles` (max-size limited to 64 KiB) which holds a msgpack'ed list of `{"name": "", "perm": "644", "data": ""`} to the `keypairs` table.
- [x] manager: Add a new set of REST API to CRUD the dotfiles.\* The total msgpack-encoded size of dotfiles must be less than 64 KiB and each file must be also smaller than 64 KiB. The total number of dotfiles is limited to 100.
- Allow only valid UTF-8 encoded text files (not arbitrary binary data).
- Prevent adding files under `.local` directory and the file `.ssh/authorized_keys` as they are managed by other means (#99 and lablup/backend.ai-manager#211).

- [x] manager/agent: Pass the value of `dotfiles` from manager to agent via `internal_data` field when creating new kernels.
- [x] agent: Populate the dotfiles when preparing the scratch directory.
- [ ] client: Add CLI & function interfaces for CRUD-ing dotfiles as `backend.ai user-config dotfiles ...` command set.
┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1159751085623729/1159756969567148) by [Unito](https://www.unito.io/learn-more)

JIRA Issue: BA-312

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing client interfaces and the manager's dotfiles REST API. Implement the `backend.ai user-config dotfiles ...` CLI and function interfaces for CRUD operations, then verify that they handle the documented dotfile limits and restrictions. Done means users can create, read, update, and delete dotfiles through the client.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, cli
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.