metabase / metabase/metabase

Incorrect json type narrowing in the presence of lists

Open
#59,381 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

.Backend .Team/Graphy Administration/Metadata & Sync Priority:P2 Semantic Layer Type:Bug
Dominant language
Clojure
Stars
49.3k
Forks
6.8k
Avg merge
1d 13h
Merged PRs (30d)
653

Description

### Describe the bug

Unfolding of json fields with incompatible array and primitive types produces incorrect and inconsistent types.

Concretely, after syncing a table with a json column "my_column", with entries containing:
```json
{"field": [1]}
{"field": [2]}
{"field": 1}
{"field": 2}
```

sync will unfold `jsoncol -> field` and assign it a database_type/base_type pair of either `"text", :type/Array`, or `"decimal", :type/Integer`, depending on the order of the rows.

Both are wrong, and should be at best `"text", :type/Text`

### To Reproduce

Sync a table with a json field with values
```javascript
{"field": [1]}
{"field": [2]}
{"field": 1}
{"field": 2}
```

### Expected behavior

_No response_

### Logs

_No response_

### Information about your Metabase installation

```JSON
metabase v1.55
```

### Severity

P2

### Additional context

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce table sync with a JSON column containing both numeric arrays and numeric primitives under the same field. Trace JSON field unfolding and type inference for that field, then verify that incompatible array and primitive values consistently resolve to text/Text rather than depending on row order.

Written by the indexing model from the issue text.

Assessment

Tech stack
clojure
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.