kubeflow / kubeflow/notebooks

[TASK] Enable frontend API regeneration during Tilt live development with uncommitted backend changes

Open
#857 3 comments 0 reactions 1 assignee Claimed by @prakharsingh-74 View on GitHub
area/ci area/frontend area/v2 kind/plan-task
Dominant language
No language data
Stars
84
Forks
149
Avg merge
5d 15h
Merged PRs (30d)
29

Description

### Certification

- [x] I certify I am an Epic Owner for Kubeflow Notebooks 2.0 and expected to create planning-related issues.

### Description

The `swagger.version` mechanism prevents the core benefit of Tilt—**live iterative development across components**—from working when backend API changes are introduced.

### Tilt's Purpose

Tilt provides a live-reload development environment where:
- Code changes are synced into running containers
- Components rebuild/restart automatically
- Developers iterate rapidly without manual rebuild cycles

The Kubeflow Notebooks Tiltfile (`developing/Tiltfile`) implements this well for most changes:
- Controller: Full Docker rebuild on changes
- Backend: Full Docker rebuild on changes (regenerates swagger.json via `make swag`)
- Frontend: Live sync of `src/` and `config/` with webpack HMR

### The Problem

When a developer modifies a **backend API handler** during a Tilt session:

1. Backend container rebuilds → new `swagger.json` is generated inside the container
2. Developer wants to update frontend TypeScript types to use the new API
3. Developer runs `npm run generate:api` (locally or needs it in container)
4. **Generation fails** because `swagger.version` points to a git commit hash that doesn't contain the new swagger.json

This breaks the Tilt development loop. The developer cannot:
- Test frontend code against new API endpoints
- Iterate on API design with immediate frontend feedback
- Develop a feature that spans backend and frontend in a single session

### Current Tilt Frontend Configuration

From `developing/Tiltfile` (lines 127-158):
```python
docker_build(
"ghcr.io/kubeflow/notebooks/workspaces-frontend",
dockerfile=os.path.join(frontend_dir, "Dockerfile.dev"),
context=frontend_dir,
live_update=[
# Sync source files into the container
sync(os.path.join(frontend_dir, "src"), "/app/src"),
sync(os.path.join(frontend_dir, "config"), "/app/config"),

# If package.json changes, reinstall dependencies
run(
"cd /app && npm install",
trigger=[...],
),
],
...
)
```

The `src/` sync includes `src/generated/`, but there's no mechanism to regenerate these files when backend changes occur.

### Why This Violates Tilt's Core Value

Tilt's documentation emphasizes:
> "Tilt watches your files for edits, automatically builds your container images, and applies any changes to bring your environment up-to-date in
real-time."

The `swagger.version` git-hash requirement creates a **manual synchronization step** that interrupts this automatic flow. Developers must:
1. Stop iterating
2. Commit backend changes (even if incomplete)
3. Update `swagger.version` with the commit hash
4. Regenerate frontend types
5. Resume development

This defeats the purpose of using Tilt for rapid iteration.

### Key Files Involved

| File | Role in Tilt Workflow |
|------|----------------------|
| `developing/Tiltfile` | Orchestrates multi-component development |
| `workspaces/frontend/Dockerfile.dev` | Dev container running webpack-dev-server |
| `workspaces/frontend/scripts/swagger.version` | Git hash blocking local generation |
| `workspaces/frontend/scripts/generate-api.sh` | Uses `git show` to fetch swagger.json |
| `workspaces/backend/openapi/swagger.json` | Generated in backend container on rebuild |
| `workspaces/frontend/src/generated/*` | TypeScript client synced via live_update |

### Concepts a Solution Should Consider

1. **Tilt-aware generation**: A way for `generate-api.sh` to detect it's running in a Tilt context and use local/container files instead of git history
2. **Backend-to-frontend file sharing**: Mechanism to make backend's generated swagger.json available to frontend container
3. **Tilt resource dependencies**: Leverage Tilt's `resource_deps` or custom triggers to regenerate frontend types when backend rebuilds
4. **Development vs CI modes**: Preserve the git-hash-based approach for CI while enabling local-file-based approach for Tilt
5. **Developer experience**: Solution should be automatic—no manual steps between backend change and frontend type availability

### Acceptance Criteria

- [ ] When backend API handlers are modified during a Tilt session, the frontend can access updated TypeScript types without manual intervention
- [ ] The `npm run generate:api` command (or equivalent) works within the Tilt development workflow using local/container swagger.json
- [ ] Frontend webpack-dev-server picks up regenerated types and triggers HMR
- [ ] The existing CI workflow (git-hash-based generation) continues to function
- [ ] Documentation in `DEVELOPMENT_GUIDE.md` is updated to explain the Tilt API development workflow

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.