local-dev.sh: schema-currency check only probes the `feedback` table, silently skips DDL on a stale DB → jOOQ codegen fails with a misleading compile error
- Dominant language
- Scala
- Stars
- 314
- Forks
- 187
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 214
Description
### What happened?
`bin/local-dev.sh up` decides whether the Postgres schema needs (re)loading by probing for a **single hard-coded table, `feedback`** (`bin/local-dev/main.sh`, `infra_ensure_db_schema`, ~L1346-1357):
```bash
# `feedback` is one of the newer tables; if it exists we assume the
# whole schema is current.
has_feedback=$(docker exec "$pg" psql -U texera -d texera_db -tAc \
"SELECT 1 FROM pg_tables WHERE schemaname='texera_db' AND tablename='feedback'" ...)
if [[ "$has_feedback" == "1" ]]; then
tui_skip "postgres: schema already current"
return 0
fi
```
The probe assumes `feedback` is the newest table. Any table added *after* `feedback` is invisible to this check. When the Postgres data volume was created before such a table existed (a very common case for a long-lived local dev volume), the volume already has `feedback`, so the script reports **"postgres: schema already current"** and skips applying `sql/texera_ddl.sql` — even though the schema is actually stale.
The downstream effect is nasty: the `dao` module runs jOOQ codegen against this stale live DB (`common/dao/build.sbt`, wired into `Compile / sourceGenerators`), so the generated `Tables`/`Keys` are missing the new relation, and `sbt dist` then fails with a **misleading compile error** that points at application Scala code, not at the schema. In my case the missing table was `workflow_cover_image` (added in `sql/updates/27.sql`), and the build failed with 11 errors like:
```
not found: value WORKFLOW_COVER_IMAGE
```
Expected: the schema-currency check should not silently skip DDL when the schema is out of date. Since `sql/texera_ddl.sql` is already idempotent (`CREATE TABLE IF NOT EXISTS`), the safest fix is to **always apply it** (the comment in the code even acknowledges re-applying is safe), or to probe the newest expected table / a schema-version marker instead of `feedback`.
### How to reproduce?
1. Have a `texera-local-dev` Postgres volume created before `workflow_cover_image` was added (i.e. an older volume that already contains `feedback`).
2. Check out a branch whose code references the newer table (references `WORKFLOW_COVER_IMAGE`).
3. Run `bin/local-dev.sh up`.
4. Observe `postgres: schema already current` is printed (DDL skipped), then `sbt: dist FAILED` with `not found: value WORKFLOW_COVER_IMAGE` — an error that gives no hint that the real cause is a stale DB schema.
Workaround: manually apply the missing migration (e.g. `docker exec -i texera-postgres psql -U texera -d texera_db < sql/updates/27.sql`) and re-run `up`.
### Version/Branch
1.3.0-incubating-SNAPSHOT (main)
### Commit Hash (Optional)
be99b3661 (observed; the check lives in `bin/local-dev/main.sh`)
### Relevant log output
```shell
▸ Infra
✓ infra: 5 containers up
○ postgres: schema already current
▸ Build
→ sbt dist (log: /tmp/texera-local-dev/logs/sbt-dist.log) (no-TTY)
✗ sbt: dist FAILED
# sbt-dist.log:
[error] .../WorkflowResource.scala:729:15: not found: value WORKFLOW_COVER_IMAGE
[error] .../WorkflowResource.scala:730:13: not found: value WORKFLOW_COVER_IMAGE
...
[error] 11 errors found
[error] (WorkflowExecutionService / Compile / compileIncremental) Compilation failed
```
Contributor guide
Research direction
Start in bin/local-dev/main.sh around infra_ensure_db_schema and inspect how bin/local-dev.sh up decides whether to apply sql/texera_ddl.sql. Check the dao codegen wiring in common/dao/build.sbt and reproduce with a stale Postgres volume containing feedback but missing workflow_cover_image. Done means schema currency is detected reliably and sbt dist no longer fails because generated jOOQ relations are missing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, postgresql, scala, shell
- Domain
- build-system, database, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100