nebari-dev / nebari-dev/data-science-pack
jhub-apps service does not receive CA trust env vars: nebi environment selector silently empty behind a private CA
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5
- Forks
- 7
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 13
Description
Symptom
On a private-CA deployment, the "Software Environment" dropdown never appears in the jhub-apps create-app form. No error surfaces in the UI; the japps log shows:
token-exchange step 1 FAILED: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate ...> (url=https://keycloak.<domain>/realms/nebari/protocol/openid-connect/token, ...)
nebi-envs: token exchange returned no JWT for <user>
Found conda environments: []
Root cause
get_nebi_environments (03-nebi-envs.py) runs inside the japps managed service, and JupyterHub managed services spawn with a minimal environment — they do not inherit the hub container's env. 02-jhub-apps.py already forwards JUPYTERHUB_OIDC_CLIENT_SECRET into the japps service environment for exactly this reason, but the CA trust env vars are not forwarded. Verified on-cluster: hub pid 1 has SSL_CERT_FILE, the japps uvicorn subprocess has no CA vars at all. The exchange's urllib calls to the external Keycloak URL (which advertises itself even in-cluster) therefore fail verification, get_conda_envs swallows the empty result, and the dropdown silently vanishes.
Same gap class as #182 (singleuser/app pods, fixed by #102 / released in 0.1.1) — the japps service is one more process that needs the CA and doesn't get it. Note the trust-bundle feature from #102 does not cover it either: _setup_trust_bundle wires spawner pods, not the japps service env.
Suggested fix
In 02-jhub-apps.py, forward the CA vars alongside the client secret, e.g.:
for var in ("SSL_CERT_FILE", "CURL_CA_BUNDLE", "REQUESTS_CA_BUNDLE"):
val = os.environ.get(var)
if val:
svc.setdefault("environment", {})[var] = val
(or, once trust-bundle-enabled is on, point the japps env at the merged bundle path the same way _setup_trust_bundle does for pods).
Workaround
A late hub.extraConfig entry appending SSL_CERT_FILE to the japps service environment (runs after 02-jhub-apps.py registers the service) — deployed successfully as openteams-ai/NIC-argocd-tyler-dev#12.
Environment: nebari-data-science-pack 0.1.1, jhub-apps 2026.5.1rc1, k3s/Hetzner, org CA distributed via trust-manager.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read 02-jhub-apps.py and compare its service environment setup with _setup_trust_bundle; then inspect get_nebi_environments in 03-nebi-envs.py to understand the failing token exchange. Forward the available CA trust variables to the japps service and verify that the Software Environment dropdown populates on a private-CA deployment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 74/100