MemberJunction / MemberJunction/MJ
Integration staging DDL drops declared field types: integer/decimal/boolean land as varchar(255)/text
- Dominant language
- TSQL
- Stars
- 29
- Forks
- 6
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 308
Description
## Symptom
A connector declares real types in its catalog, and the staging table is created with everything
as a string.
Observed on a PostgreSQL instance with the EO BMF connector:
| Field | Connector declares | Landed column |
|---|---|---|
| `Subsection`, `Affiliation`, `RulingYear` | `integer` | `character varying(255)` |
| `BmfAssetAmt`, `BmfRevenueAmt` | `decimal(19,2)` | `character varying(255)` |
| `IsAssociation` | `boolean` | `text` |
The declarations are correct in both the TS catalog (`EOBMFObjects.ts`) and the seeded
`IntegrationObjectField` rows.
## Where it is NOT
`packages/Integration/schema-builder/src/TypeMapper.ts:13-27` maps these correctly —
`integer → INTEGER`, `decimal → NUMERIC`, `boolean → BOOLEAN`. So the mapper is fine and the loss
happens upstream of it, somewhere in the RSU / `IntegrationApplyAll` discovery path that feeds it.
## Impact
- every downstream consumer must cast at the boundary before it can filter or aggregate
- numeric and date range filters against staging are wrong or impossible
- indexes on those columns are near-useless
- it is silent: no warning, no error, and the sync reports success
## Note
This is distinct from a connector declaring a type badly. We fixed one genuine instance of that
separately (`BmfSnapshotDate` really was declared as a bounded string). The six above are declared
correctly and still land as strings.
## Environment
MJ 5.51.0, PostgreSQL.
Contributor guide
Research direction
Start with packages/Integration/schema-builder/src/TypeMapper.ts:13-27 to confirm the mappings, then trace the RSU / IntegrationApplyAll discovery path that supplies its input. Compare the declarations in EOBMFObjects.ts and the seeded IntegrationObjectField rows with the generated staging schema; done means the six declared integer, decimal, and boolean fields land with their PostgreSQL types rather than string types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, typescript
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100