[BUG] Table chart crashes with "TypeError: Cannot convert a BigInt value to a number" for integers beyond Number.MAX_SAFE_INTEGER
- Dominant language
- Python
- Stars
- 74.8k
- Forks
- 18.3k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 685
Description
### Bug description
A Table chart renders an empty cell and logs a `TypeError` when a numeric column
holds an integer larger than `Number.MAX_SAFE_INTEGER` (2^53 - 1) and the column
has a number format set.
The crash is quite specific and occurs only when we hit the cache.
```
TypeError: Cannot convert a BigInt value to a number
at Math.abs ()
at Function.formatFunc (static/assets/616.94c331d4202698eb2aa9.entry.js:5:9152)
at Function.format (static/assets/616.94c331d4202698eb2aa9.entry.js:5:6732)
at ef (static/assets/ddaae12625f5b9aeb7e0.chunk.js:213:479)
at Cell (static/assets/ddaae12625f5b9aeb7e0.chunk.js:225:4829)
```
1. In SQL Lab, run one of the queries below and save it as a virtual dataset. Any database works, the only requirement is a 64-bit integer column with a value above 2^53.
ClickHouse:
```sql
SELECT toInt64(1425300509404304697) AS big_bytes
```
PostgreSQL:
```sql
SELECT 1425300509404304697::bigint AS big_bytes
```
2. Create a Table chart on it. Query mode: Aggregate. Metric:
`MAX(big_bytes)`.
3. In *Customize*, set the number format of the metric column to
`MEMORY_BINARY` .
4. Open the chart. Open it a second time
so the result comes synchronously from the cache (the first, asynchronous
load goes through `fetchCachedData`, which uses plain `JSON.parse` and does
not crash).
5. The cell is empty and the browser console shows the `TypeError` above.
From what I can see, https://github.com/apache/superset/pull/42594 fixed similar for echarts.
We face it in ClickHouse, `Int64` byte counters above 8 PiB.
### Screenshots/recordings
_No response_
### Superset version
master / latest-dev
### Python version
3.12
### Node version
18 or greater
### Browser
Chrome
### Additional context
```js
test('formats bigint values decoded by json-bigint', () => {
const big = BigInt('1425300509404304697') as unknown as number;
expect(createMemoryFormatter({ binary: true })(big)).toBe('1.24EiB');
});
```
### Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [x] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
Contributor guide
Research direction
Start at the Table chart Cell path and the createMemoryFormatter entry point, then compare the cached-data path using fetchCachedData with the related ECharts fix in PR 42594. Run or extend the shown formatter test for the json-bigint value and verify that a cached Table chart displays the formatted value without a TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- data-visualization, frontend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 72/100