libredb / libredb/libredb-studio
[EPIC] Database object model: containers, kinds, and a lazy object tree across all 17 engines
- Dominant language
- TypeScript
- Stars
- 726
- Forks
- 119
- Avg merge
- 7h 47m
- Merged PRs (30d)
- 265
Description
Four open requests point at one missing abstraction. This issue plans the end-to-end work that answers all of them with a single object model, and tracks it.
Related: #710, #765, [discussion 778](https://github.com/orgs/libredb/discussions/778), #773.
## The gap
The schema surface is one flat `TableSchema[]` with a `name: string` (`src/lib/types.ts:195`). It has no namespace level, no object kind, and no lazy boundary. Each of the four requests needs one of those three.
| Request | Asks for | Blocked by |
| --- | --- | --- |
| #710 | Show views and materialised views on PostgreSQL | No object-kind concept |
| #765 | Oracle with 43,512 tables and 910,000 columns freezes the UI on connect | No container level, no lazy loading |
| [discussion 778](https://github.com/orgs/libredb/discussions/778) | Edit procedures and functions | No source text, no DDL apply path |
| #773 | Double-click a table for a data preview | No tree interaction model |
## Measured current state
Measured on `main` at `ea5730c9`.
- `name` embeds the schema on PostgreSQL when it is not `public` (`postgres.ts:1478`), and `query-generators.ts:132` splits it back on `.`. A table literally named `a.b` in `public` therefore generates `"a"."b"`.
- PostgreSQL filters `table_type IN ('BASE TABLE', 'MATERIALIZED VIEW')` (`postgres.ts:189`). The materialised-view entry exists for Materialize and RisingWave. Stock PostgreSQL does not report materialised views through `information_schema.tables` at all, so on real PostgreSQL neither views nor materialised views are listed. That is #710.
- MySQL filters `TABLE_TYPE = 'BASE TABLE'` in four places.
- Oracle hard-scopes introspection to `OWNER = connection.user` (`oracle.ts:778`), so the app shows exactly one schema on Oracle with no way to reach another. PostgreSQL sweeps every non-system schema. The two engines have opposite scoping and the user controls neither.
- Two-phase loading exists in 3 of 17 providers. The other 14 fall back to a full `getSchema()`.
- `schemaRefreshPattern` triggers a full schema refetch for any editor statement matching `CREATE|DROP|ALTER|TRUNCATE` (`use-query-execution.ts:598`).
- `docker/postgres-init/02-sample-data.sql` already creates four views in schema `app`. #710 is reproducible in this repo's own dev environment today.
## The model
Providers declare their container levels and object kinds as data on `ProviderCapabilities`. Core code never reads a kind id or a type id.
- `ObjectRole` is a closed set the UI derives behaviour from: `relation`, `routine`, `group`, `attached`, `config`.
- `ObjectKindSpec.id` is an open string. DBeaver, CloudBeaver, Azure Data Studio and pgAdmin all reached the same answer independently, and JDBC's closed model is the counter-example: it cannot express a trigger, a sequence, a materialized view or a package at all. A ClickHouse dictionary, a Druid lookup and an Oracle package therefore reach the tree through a provider-local declaration and no core change.
- `ContainerLevelSpec` gives zero, one or two levels, each carrying the engine's own word. Five engines have no container, seven have one, five have two.
- `DatabaseObject.path` is a `readonly string[]`, never a joined string, which retires the dot-splitting defect above.
- `KindCount` is `{ count } | { unavailable }`, so three facts stay separate: the engine has no such concept (kind not declared, no folder drawn), the engine holds none (`count: 0`, zero badge), the read was refused (the engine's own sentence).
Four provider methods replace `getSchema` / `getSchemaList` / `getSchemaRelations`:
```
listContainers(parent?) countObjects(container)
listObjects(container, kind) describeObject(path)
```
Plus a bulk `inventory` surface for the agent, the diagram and schema diff, which reports its own truncation rather than handing a slice over as a whole inventory.
## Decisions
| Decision | Choice |
| --- | --- |
| Scope | One model, three phases |
| Tree shape | Full tree: containers, kind folders, objects, lazy per node |
| First paint | Container list plus the active container expanded with kind counts |
| Escape hatch | Connection-level `skipObjectScan` plus an explicit load action, which is #765's reporter's own stated preference |
| Migration | One pull request: `DatabaseObject` replaces `TableSchema`, `getSchema()` removed, major version |
| Acceptance | All 17 type-ids verified against a live instance before merge |
| DDL authority | The database is the boundary, not the UI role. A user can already run `CREATE OR REPLACE` in the query editor. |
## Phases
**Phase 1, this pull request.** The model, the tree, the migration. Closes #710 and #765, and makes #773 cheap.
**Phase 2.** Source reading. `hasSource` gates a Source tab; `sourceLanguage` picks the Monaco language. An unreadable source shows the engine's own sentence and never an empty editor, because an empty editor reads as "no source" and a user who types over it deletes the object.
**Phase 3.** Editing and DDL apply, which answers [discussion 778](https://github.com/orgs/libredb/discussions/778). In-place editing behind a mandatory preview. The provider builds the statement, core never does. MySQL 8.4 has no `CREATE OR REPLACE PROCEDURE` and its DDL implicitly commits, so its strategy is `DROP` then `CREATE` with the current source retained for restore; MariaDB 11.x has `CREATE OR REPLACE` and uses it. Oracle's `CREATE OR REPLACE` can return success and leave the object `INVALID` without node-oracledb throwing, so `result.warning` is checked and `USER_ERRORS` supplies the line and position.
## Acceptance for Phase 1
1. Every type-id has a fixture containing one instance of every kind its provider declares. PostgreSQL already has four views; sixteen fixtures are new.
2. An end-to-end spec walks the tree per engine and asserts each declared kind's folder and count against the fixture.
3. An end-to-end spec asserts the three absences render differently.
4. #710 closes on evidence: the four seeded PostgreSQL views appear in the tree.
5. #765 closes on evidence: a large Oracle owner paints without freezing, measured, and `skipObjectScan` issues zero catalog queries.
6. Coverage stays at 100 percent lines.
7. `docs/providers/.md` and `tests/integration/db/-provider.test.ts` sync for all 17, per the triad rule.
## Out of scope
Debugging PL/SQL, object-level version control, creating objects from a form, cross-database references, and #773's data preview, which becomes cheap once the tree exists but is its own change.
## Design and plan
The full design and the 28-task implementation plan were written before any code and live in the working tree under `docs/superpowers/`, which is git-ignored. Their substance is reproduced above; ask if you want either posted in full.
Contributor guide
Assessment
This issue has not been assessed yet.