feat(fabric): TINYINT source columns emit unsupported `tinyint` DDL and fail the load step
- Dominant language
- Python
- Stars
- 5.9k
- Forks
- 600
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 38
Description
### Problem
Fabric Warehouse has no `tinyint`. Its narrowest integer type is `smallint`. The Fabric destination emits `tinyint` anyway, so `CREATE TABLE` fails:
```
dlt.destinations.exceptions.DatabaseTerminalException: Driver Error: Syntax error or access violation;
[Microsoft][SQL Server]The data type 'tinyint' in column 'foo' is not supported in this edition of SQL Server.
```
Because it is a `DatabaseTerminalException` the load is not retried and the package stays pending, so every following run of that pipeline fails on the same table. A single `TINYINT` column anywhere in a source schema blocks the whole schema from loading.
### Root cause
`sql_database` reflects `TINYINT` as dlt type `bigint` with `precision=8`. On the way out, `MsSqlTypeMapper.to_db_integer_type` maps that precision to the narrowest SQL Server integer type:
```python
# dlt/destinations/impl/mssql/factory.py
if precision <= 8:
return "tinyint"
```
That is correct for SQL Server and Synapse but not for Fabric. `FabricTypeMapper` inherits from `SynapseTypeMapper`, which inherits from `MsSqlTypeMapper`, and currently overrides only the datetime and string mappings -- so the `tinyint` branch is reached unchanged.
Verified against `devel`:
```
precision=8 -> tinyint
precision=16 -> smallint
precision=32 -> int
precision=64 -> bigint
```
This is the same class of problem as the already-handled `datetimeoffset` -> `datetime2` and `nvarchar` -> `varchar` mismatches: an inherited SQL Server type that Fabric Warehouse rejects.
### Reproduction
1. Create a SQL Server source table with a `TINYINT` column:
```sql
CREATE TABLE dbo.Foo (Id int NOT NULL, Bar tinyint NOT NULL);
```
2. Run a `sql_database` source with `reflection_level="full_with_precision"` into a `fabric` destination.
3. Extract and normalize succeed; load fails on `CREATE TABLE`.
### Current workaround
Use `reflection_level="full"` so no precision is reflected and the column becomes `bigint`, losing the width information for every integer column in the schema.
### Proposed fix
Override `to_db_integer_type` in `FabricTypeMapper` so any precision at or below 8 widens to `smallint`, then delegate to the parent. Widening is safe: `TINYINT` values (0-255) all fit in `smallint`.
A test asserting that a `bigint` column with `precision=8` yields `smallint` on the Fabric destination would guard the inheritance chain, which is where this class of regression keeps appearing.
Contributor guide
Assessment
This issue has not been assessed yet.