Add a "Working with JDBC / databases" section to the 1.0 Migration Guide (#1782)
- Dominant language
- Kotlin
- Stars
- 1.1k
- Forks
- 83
- Avg merge
- 4d 12h
- Merged PRs (30d)
- 30
Description
# Add a "Working with JDBC / databases" section to the 1.0 Migration Guide (#1782)
## Motivation
`dataframe-jdbc` changed a lot between 0.15 and 1.0 — new entry points, a changed validation API, a redesigned custom-`DbType` extension surface, new database support, and many type-mapping fixes. Users upgrading need a single, concise section describing what changed and what they must adjust. This is the detailed plan for #1782 ("Add info about changes in work with JDBC to Migration Guide", milestone 1.0, assigned to zaleslaw).
## Where it goes
`docs/StardustDocs/topics/MigrationTo_1_0.md` already has a commented-out placeholder `## Changes in working with JDBC` (in the TODO block tied to #1630). Flesh out that section; coordinate with #1630 and #1779 (artifacts/modules) so JDBC-as-a-module vs JDBC-API changes don't overlap.
## User-facing changes to document (gathered from the `dataframe-jdbc` public API + commit history)
Grouped by who is affected. Each item notes the source where known.
### A. Reading API (affects everyone)
- **New generic `DataFrame.readDataFrame(...)` entry points** alongside `readSqlTable` / `readSqlQuery` / `readResultSet` / `readAllSqlTables`, available for `Connection`, `DataSource`, and `DbConnectionConfig`.
- **`SqlValidation` enum replaces the old `strictValidation: Boolean`** on query reads (#1881). Values: `None`, `ReadOnly`. → migration mapping needed (`strictValidation = true/false` → `SqlValidation.ReadOnly/None`).
- **`DbConnectionConfig` gained a `readOnly` flag (default `true`)**: the connection is set read-only, `autoCommit = false`, and rolled back after reading; even with `readOnly = false`, only `SELECT` is permitted internally.
- **New `configureStatement` lambda** on the read functions to tune the `PreparedStatement` (fetch size, query timeout, etc.).
- **`limit` is now nullable `Int?`** (`null` = no limit).
- **Column naming is index/order-based with automatic name repair**: missing/duplicate SQL column names no longer break reading; they are auto-renamed (documented via the shared `AutoRenameInputSnippet`).
- **Unsupported operations now throw real exceptions instead of `error()`**, and `importDataSchema()` gives a clearer message in notebooks (the `importDataSchema: String` overload was removed).
### B. Database support (affects users of specific DBs)
- **DuckDB support added** (`DuckDb`), including STRUCT / composite / `STRUCT[]` → `ColumnGroup`/`FrameColumn` conversion.
- **`H2` now has a `Mode` enum** (`Regular`, `MySql`, `MariaDb`, `MsSqlServer`, `PostgreSql`) with `toDbType()` / `fromDbType()`.
- **SQLite custom types**: `Sqlite.withCustomTypes(map)` / `Sqlite(customTypesMap)` to map non-standard SQLite type names, with nullability handling.
- **Type-mapping fixes** that change resulting column types: `BIGINT UNSIGNED` for MySQL/MariaDB; expanded PostgreSQL type & extension-type support (#537); scoped table retrieval for MySQL/MariaDB (#1901); Postgres driver classes no longer loaded when unused (#1773).
### C. Custom `DbType` authors (affects extenders)
- **`DbType` redesigned + new `AdvancedDbType`** ("simple" vs "advanced" DB types using JDBC type mapping).
- **New `JdbcToDataFrameConverter` builder API** with `withResultReader` / `withPreprocessor` / `withColumnBuilder` / `withTargetSchema`, plus the `DbResultSetReader`, `DbValuePreprocessor`, `DbColumnBuilder` interfaces.
- Anyone who implemented a custom `DbType` in 0.15 will need to migrate to the new abstract methods.
### D. Deprecations
- `SupportedDataFrameFormat` (and the `Jdbc` format's use of it) deprecated.
## Proposed section structure
1. Short intro + link to [Read from SQL databases](https://kotlin.github.io/dataframe/readsqldatabases.html) and the Modules page.
2. A `0.15 → 1.0` mapping table for the concrete API changes (at least `strictValidation` → `SqlValidation`, and any renamed reads).
3. Bullet notes for behavioral changes (read-only safety, name repair, type-mapping fixes, new DB support).
4. A short "Custom `DbType`" subsection for extenders.
## Open questions
- Baseline: strictly `0.15 → 1.0`, or also note `1.0-Beta` intermediate JDBC changes?
- Confirm the exact old signature/behavior of `strictValidation` for an accurate mapping row.
- Are `readSqlTable`/`readSqlQuery` themselves renamed vs 0.15, or only extended with new params/defaults?
- How much custom-`DbType` migration detail belongs in the guide vs the SQL docs page?
- Split responsibilities with #1630 (JDBC as a *module*) vs this issue (JDBC *API/behavior*).
## Sources to cite in the PR
Public API dump `dataframe-jdbc/api/dataframe-jdbc.api`; KDocs in `readJdbc.kt`; PRs/commits: #1881 (SqlValidation), #1901 (scoped table retrieval), #537 (Postgres extension types), #1773 (Postgres lazy loading), #1735/#1714 (JDBC types, importDataSchema), and the `DbType2` / `AdvancedDbType` / custom-SQLite-types commit series.
## Acceptance criteria
- `MigrationTo_1_0.md` has a concise **Working with JDBC** section replacing the commented-out placeholder.
- It contains a `0.15 → 1.0` mapping for API changes (incl. `strictValidation` → `SqlValidation`) and bullet notes for behavioral/type-mapping changes and new DB support.
- It has a short custom-`DbType` migration note for extenders.
- It cross-links the SQL docs page and Modules page instead of duplicating them.
- Open questions resolved; alignment with #1630 / #1779 confirmed.
Contributor guide
Assessment
This issue has not been assessed yet.