[JS] Decimal-type conversions result in errors/inaccuracies in Arrow JS
- Dominant language
- TypeScript
- Stars
- 112
- Forks
- 23
- Avg merge
- 21h 18m
- Merged PRs (30d)
- 8
Description
### Describe the bug, including details regarding any error messages, version, and platform.
Thanks for all the work on Arrow! Unfortunately, I've run into problems in the JS API around both encoding and decoding of Decimal-typed data.
**Encoding:**
```js
arrow = require('apache-arrow')
arrow.vectorFromArray([1, 12, 34], new arrow.Decimal(3, 18))
```
Results in `TypeError: d.subarray is not a function`
(Incidentally it also seems that the Decimal type constructor uses a different argument order than pyarrow.)
**Decoding:**
Since encoding failed in JS, let's do it in pyarrow:
```py
import pyarrow as pa
v = pa.array([1, 12, 34], type=pa.decimal128(18, 3))
batch = pa.record_batch([v], names=['d'])
sink = pa.BufferOutputStream()
with pa.ipc.new_stream(sink, batch.schema) as writer:
writer.write_batch(batch)
sink.getvalue().hex()
```
and then we can carry the resulting byte string over to JavaScript and try to decode it:
```js
const hex = 'FFFFFFFF780000001000000000000A000C000600050008000A000000000104000C000000080008000000040008000000040000000100000014000000100014000800060007000C00000010001000000000000107100000001C0000000400000000000000010000006400000008000C0004000800080000001200000003000000FFFFFFFF8800000014000000000000000C0016000600050008000C000C0000000003040018000000300000000000000000000A0018000C00040008000A0000003C00000010000000030000000000000000000000020000000000000000000000000000000000000000000000000000003000000000000000000000000100000003000000000000000000000000000000E8030000000000000000000000000000E02E0000000000000000000000000000D0840000000000000000000000000000FFFFFFFF00000000';
const bytes = Uint8Array.from(hex.match(/.{1,2}/g).map(s => parseInt(s, 16)));
const vec = arrow.tableFromIPC(bytes).getChild('d');
vec.get(0) // [1000, 0, 0, 0] <- byte array looks correct
`${vec.get(0)}` // "1000" <- incorrect, should be 1; ignores scale
Number(vec.get(0)) // 2.2136092888451462e+23 <- incorrect, should be 1
+vec.get(0) // TypeError: can't convert BigInt to number <- ???
```
While the basic byte encoding looks OK, all of the built-in conversions appear to be faulty.
Related: apache/arrow-js#100, apache/arrow-js#81
cc: @domoritz
### Component(s)
JavaScript
Contributor guide
Assessment
This issue has not been assessed yet.