nebari-dev / nebari-dev/data-science-pack
Set jupyterhub.cull.maxAge as a backstop for activity-based culling
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5
- Forks
- 7
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 13
Description
jupyterhub.cull.maxAge is 0 (unset) in values.yaml, so nothing bounds how long a user pod can live. Every culler we run is activity-based, and activity is a signal we don't fully control.
Concretely, the hub culler can't be fixed from inside the pod. CHP updates route activity on any request or reply data including websocket frames:
and the hub folds that into spawner.last_activity with max():
https://redirect.github.com/jupyterhub/jupyterhub/blob/5.5.0/jupyterhub/app.py#L3595-L3602
so any proxied app holding a websocket keeps the hub's view of the pod fresh no matter what the in-pod server reports. That's the VS Code case in #208, but it'll be true of anything else we put behind jupyter-server-proxy.
max_age is the one hub-level lever that doesn't care. It's evaluated in its own branch keyed off started, independent of the inactivity check:
2i2c runs timeout: 1800 / every: 300 / maxAge: 43200 in production:
https://infrastructure.2i2c.org/sre-guide/manage-k8s/culling/
and minrk's guidance in https://redirect.github.com/jupyterhub/jupyterhub-idle-culler/issues/55 is the same shape, short in-pod deadline and a long hub one.
Proposal is to set maxAge to something in the 12h range and document it as a hard cap rather than an idle timeout. Open question is what number actually fits our users. 12h kills a pod mid-session for anyone doing a long training run who isn't touching the browser, so I'd want to know if that's a real pattern here before picking it.
This is independent of #226. That PR fixes what counts as activity, this bounds what happens when the answer is still wrong.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with values.yaml and review how the existing culler settings are configured. Confirm what maxAge value fits users' workloads, then set it as a hard pod lifetime cap and document that it is independent of the idle timeout. Check the resulting configuration against the referenced production settings and culling behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- yaml
- Domain
- devops
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100