apache / apache/superset

[BUG] Table chart crashes with "TypeError: Cannot convert a BigInt value to a number" for integers beyond Number.MAX_SAFE_INTEGER

Open
#44,007 1 comment 0 reactions 0 assignees View on GitHub
#bug
Dominant language
Python
Stars
74.8k
Forks
18.3k
Avg merge
2d 5h
Merged PRs (30d)
685

Description

### Bug description

A Table chart renders an empty cell and logs a `TypeError` when a numeric column
holds an integer larger than `Number.MAX_SAFE_INTEGER` (2^53 - 1) and the column
has a number format set.

The crash is quite specific and occurs only when we hit the cache.
```
TypeError: Cannot convert a BigInt value to a number
at Math.abs ()
at Function.formatFunc (static/assets/616.94c331d4202698eb2aa9.entry.js:5:9152)
at Function.format (static/assets/616.94c331d4202698eb2aa9.entry.js:5:6732)
at ef (static/assets/ddaae12625f5b9aeb7e0.chunk.js:213:479)
at Cell (static/assets/ddaae12625f5b9aeb7e0.chunk.js:225:4829)
```
1. In SQL Lab, run one of the queries below and save it as a virtual dataset. Any database works, the only requirement is a 64-bit integer column with a value above 2^53.

ClickHouse:

```sql
SELECT toInt64(1425300509404304697) AS big_bytes
```

PostgreSQL:

```sql
SELECT 1425300509404304697::bigint AS big_bytes
```

2. Create a Table chart on it. Query mode: Aggregate. Metric:
`MAX(big_bytes)`.
3. In *Customize*, set the number format of the metric column to
`MEMORY_BINARY` .
4. Open the chart. Open it a second time
so the result comes synchronously from the cache (the first, asynchronous
load goes through `fetchCachedData`, which uses plain `JSON.parse` and does
not crash).
5. The cell is empty and the browser console shows the `TypeError` above.

From what I can see, https://github.com/apache/superset/pull/42594 fixed similar for echarts.
We face it in ClickHouse, `Int64` byte counters above 8 PiB.

### Screenshots/recordings

_No response_

### Superset version

master / latest-dev

### Python version

3.12

### Node version

18 or greater

### Browser

Chrome

### Additional context

```js
test('formats bigint values decoded by json-bigint', () => {
const big = BigInt('1425300509404304697') as unknown as number;
expect(createMemoryFormatter({ binary: true })(big)).toBe('1.24EiB');
});

```

### Checklist

- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

Contributor guide

Open the contributing guide

Research direction

Start at the Table chart Cell path and the createMemoryFormatter entry point, then compare the cached-data path using fetchCachedData with the related ECharts fix in PR 42594. Run or extend the shown formatter test for the json-bigint value and verify that a cached Table chart displays the formatted value without a TypeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
data-visualization, frontend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.