astrodbtoolkit / astrodbtoolkit/AstrodbKit

Empty JSON array ([]) as a lookup-table reference file causes a confusing IntegrityError in load_table()

Offen Anfängerfreundlich
#110 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
12
Forks
5
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

**Where:** `astrodbkit.astrodb.Database.load_table()` does `conn.execute(self.metadata.tables[table].insert().values(data))` where `data` comes straight from `json.load()`. If the JSON file's top-level array is empty (`[]`), SQLAlchemy's `insert().values([])` is interpreted as "insert one row using column defaults" (a known SQLAlchemy quirk: an empty list to a multi-row `.values()` isn't a no-op), not "insert nothing."

**What happened:** After emptying `Publications.json`, `SourceTypeList.json`, and `ParameterList.json` to `[]` (clearing stale template-example placeholder rows before real data existed), `build_db_from_json()` failed with `sqlite3.IntegrityError: NOT NULL constraint failed: Publications.reference` — a single default-valued row was attempted, which violated the non-nullable `reference` column. The error message gives no hint that the root cause was an empty-array JSON file rather than a real data problem.

**Workaround:** Deleted the three now-empty JSON files entirely instead of leaving them as `[]` — `load_table()`'s existing `if os.path.exists(filename):` guard cleanly skips a missing file (with an optional verbose "not found" message), which is the actual correct way to represent "no data yet" for a reference table.

**Suggested change:** `Database.load_table()` should special-case `if not data: return` before calling `insert().values(data)`, so an empty-array reference file is a documented no-op instead of an inserted row of column defaults that then trips downstream NOT NULL constraints.

**Cross-filed:** Also filed against the [astrodb-bot](https://github.com/astrodbtoolkit/astrodb-bot) skills repo as [astrodbtoolkit/astrodb-bot#99](https://github.com/astrodbtoolkit/astrodb-bot/issues/99), which documents the same gotcha from the skill-user workaround side; this issue is for the actual `astrodbkit` fix.

---
_Reported from a gotchas.md log filed by a skill user (2026-08-28)._

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Rechercherichtung

Start in astrodbkit.astrodb.Database.load_table(), where data from json.load() is passed to insert().values(data). Reproduce the behavior with a top-level [] reference-table file and follow build_db_from_json(); done means the empty file is treated as a no-op and no default-valued row or IntegrityError is produced.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, sqlalchemy
Bereich
databases
Issue-Typ
Bug
Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Aktivitätsstatus
Aktiv
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
84/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.