[JS] Table.fromStruct() silently truncates vectors to the first chunk
- Dominant language
- TypeScript
- Stars
- 112
- Forks
- 23
- Avg merge
- 21h 18m
- Merged PRs (30d)
- 8
Description
Table.fromStruct() only uses the first chunk from the input vector.
```javascript
import { Bool, Field, Int32, Struct, Table, Vector } from "apache-arrow";
const myStruct = new Struct([
Field.new({ name: "over", type: new Int32() }),
Field.new({ name: "out", type: new Bool() })
]);
const data = [];
for(let i=0;i<1500;i++) {
data.push({ over:i, out:i%2 === 0 });
// create a vector with two chunks
const victor = Vector.from({
type: myStruct,
/*highWaterMark: Infinity,*/
values: data
});
console.log(victor.length); // 1500
const table = Table.fromStruct(victor);
console.log(table.length); // 1000
```
The workaround is to set highWaterMark to Infinity
Table.new() works as expected
```javascript
const int32Array = new Int32Array(1500);for(let i=0;i<1500;i++) int32Array[i] = i;
const intVector = Vector.from({ type: new Int32(), values: int32Array});
console.log(intVector.length); // 1500
const intTable = Table.new({ intColumn:intVector });
console.log(intTable.length); // 1500
```
The origin seems to be in Chunked.data() but I don't understand the code enough to propose a fix.
**Reporter**: [David Saslawsky](https://issues.apache.org/jira/browse/ARROW-10450)
**Note**: *This issue was originally created as [ARROW-10450](https://issues.apache.org/jira/browse/ARROW-10450). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Assessment
This issue has not been assessed yet.