apache / apache/arrow-js

[JS] tableFromJSON cannot handle nested objects containing strings

Open
#86 5 comments 7 reactions 0 assignees View on GitHub
Type: bug
Dominant language
TypeScript
Stars
112
Forks
23
Avg merge
21h 18m
Merged PRs (30d)
8

Description

```

$ node

const g = require('apache-arrow')

g.tableFromJSON([\{a: [ { b: "hi" } ]}])

```

 

The dictionary types:

 

TYPE Dictionary \{indices: Int32, dictionary: Utf8, isOrdered: false, id: 12}dictionary: Utf8 {}id: 12indices: Int32 \{isSigned: true, bitWidth: 32}isOrdered: falseArrayType: (...)children: (...)typeId: (...)valueType: (...)[[Prototype]]: Dictionary
typecomparator.ts:191 OTHER 

 

OTHER Dictionary \{indices: Int32, dictionary: Utf8, isOrdered: false, id: 14}dictionary: Utf8typeId: (...)[[Prototype]]: Utf8id: 14indices: Int32 \{isSigned: true, bitWidth: 32}isOrdered: falseArrayType: (...)children: (...)typeId: (...)valueType: (...)[[Prototype]]: Dictionary

 

This happens here:

    else if (arraysCount + nullsCount === value.length) {
        const array = value;
        const childType = inferType(array[array.findIndex((ary) => ary != null)]);
        if (array.every((ary) => ary == null || (0, typecomparator_js_1.compareTypes)(childType, inferType(ary)))) {
            return new dtypes.List(new schema_js_1.Field('', childType, true));
        }
    }

 

So we're always instantiating a new dictionary type, with a new id, when we do inferType(ary), so this is never going to succeed.

**Reporter**: [Samuel Schneck](https://issues.apache.org/jira/browse/ARROW-18208)
**Assignee**: [Samuel Schneck](https://issues.apache.org/jira/browse/ARROW-18208)
#### PRs and other links:
- [GitHub Pull Request apache/arrow#14554](https://github.com/apache/arrow/pull/14554)

**Note**: *This issue was originally created as [ARROW-18208](https://issues.apache.org/jira/browse/ARROW-18208). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.