microsoft / microsoft/SPTAG

Bug for balanceddatapartition

Open
#437 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
5k
Forks
622
Avg merge
8h 37m
Merged PRs (30d)
5

Description

Describe the bug
I tried to use Release/balanceddatapartition to partition the dataset by the following command.
./balanceddatapartition -d 420 -v float -f XVEC -i msong/msong_base.fvecs -c 200 -m L2
Then, I got the following output.

[1] Using AVX512 InstructionSet!
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
[1] rank 0  data:(994185,420) machines:1 clusters:200 type:3 threads:10 lambda:0.000000 samples:1000 maxcountperpartition:9941
[1] rank 0 init centers
[server:3290477] *** Process received signal ***
Segmentation fault (core dumped)

After add -t 1, the I got more error messages.

[1] Using AVX512 InstructionSet!
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
Authorization required, but no authorization protocol specified
[1] rank 0  data:(994185,420) machines:1 clusters:200 type:3 threads:1 lambda:0.000000 samples:1000 maxcountperpartition:9941
[1] rank 0 init centers
[server:3292645] *** Process received signal ***
[server:3292645] Signal: Segmentation fault (11)
[server:3292645] Signal code: Address not mapped (1)
[server:3292645] Failing at address: 0xc8000000f0
[server:3292645] [ 0] /lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x716f2fc42520]
[server:3292645] [ 1] ./balanceddatapartition(+0x24ab0)[0x5a2b66ecfab0]
[server:3292645] [ 2] /home/cur_usr/miniconda3/envs/test/lib/libgomp.so.1(GOMP_parallel+0x43)[0x716f2ff77c58]
[server:3292645] [ 3] ./balanceddatapartition(+0x8b46a)[0x5a2b66f3646a]
[server:3292645] [ 4] ./balanceddatapartition(+0x22e5c)[0x5a2b66ecde5c]
[server:3292645] [ 5] /lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x716f2fc29d90]
[server:3292645] [ 6] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x716f2fc29e40]
[server:3292645] [ 7] ./balanceddatapartition(+0x23f25)[0x5a2b66ecef25]
[server:3292645] *** End of error message ***
Segmentation fault (core dumped)

I am not sure if I can use the Release/balanceddatapartition program for partitioning the data. Or can you please offer me some alternative choices to partition the dataset. The python binding seems to work well.

Thank you.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the reported command with Release/balanceddatapartition, first with the default thread count and then with -t 1, and inspect the recorded segmentation-fault stack trace. Compare this behavior with the Python binding, which the report says works. Done means identifying and fixing the crash or documenting a working partitioning alternative.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.