rapidsai / rapidsai/node

BUG: `nullCount` related regression in `getJSONObject`

Open
#401 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
211
Forks
29
Avg merge
17h 30m
Merged PRs (30d)
1

Description

nullCount and countNonNulls are not lining up. The nullmask after performing getJSONObject appears, perhaps, to be uninitialized.

rapids@tcomer-NVIDIA:~/node/modules/demo/api-server$ node null_count_test.js public/small-graph.json 
0
100
rapids@tcomer-NVIDIA:~/node/modules/demo/api-server$ node null_count_test.js public/large-graph.json 
0
100
rapids@tcomer-NVIDIA:~/node/modules/demo/api-server$ node null_count_test.js public/graphology.json 
1918976
81023
rapids@tcomer-NVIDIA:~/node/modules/demo/api-server$ cat null_count_test.js 
#! node

const file = process.argv[2];

const {Series} = require('@rapidsai/cudf');
const small = Series.readText(file);
const s2 = small.split('"edges":');
const edges2 = s2.gather([1], false);
const E2 = edges2.split('[\n').gather([1], false).split('},\n')
keys = E2.getJSONObject('.key')
console.log(keys.nullCount)
console.log(keys.countNonNulls())

There are no nulls in graphology.json, and the format of the numbers does not appear to change. This isn't trivial to track down.

This nullMask affects the use of gather directly, and most likely other cases. It caused a regression in my #397 work. I'll work on producing a minimal reproducer.

Now that I have my regression fixed (with setNullMask(1, 0)) I'm moving on for now.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with null_count_test.js and reproduce the differing nullCount and countNonNulls results for graphology.json. Trace the nullmask behavior through getJSONObject and gather, using the setNullMask workaround and regression context from #397 as clues. Done means the counts agree for the reproducer and the regression is covered by a test.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, typescript
Domain
backend, data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.