machinelearnjs / machinelearnjs/libsvm-ts
libsvm instances during the unit tests are influenced by other coexisting instances and cause test failures
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 4
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
Context:
While trying to cover tests for WASM loaded SVM's `predict`, I've encountered an issue of incorrect prediction results between running a test by itself vs running tests in-band.
Problematic test:
```
it('should predict bodyfat scale dataset using C_SVC and LINEAR', () => {
const rawData = fs.readFileSync(path.join(__dirname, '../samples/bodyfat_scale.txt'), 'utf-8');
const data = rawData.split('\n').map((line) => line.split(' ').filter((el) => el));
const labels = data.map((line) => +line.splice(0, 1)[0]);
const features = data.map((line) => line.map((el) => +el.split(':')[1]));
const svm = new SVM({
type: SVMTypes.C_SVC,
kernel: KernelTypes.LINEAR,
epsilon: 0.001,
quiet: true,
probabilityEstimates: true,
});
return svm.loadWASM().then((loadedSVM) => {
loadedSVM.train({ samples: features, labels });
const predResult = loadedSVM.predict({
samples: [features[0], features[1]],
});
loadedSVM.free();
// TODO: It seems like other tests are influencing the result of this.
// I don't know why so far, we should figure it out
expect(predResult).toEqual([1, 1]);
});
});
```
When you run this test on its own, it passes without an issue since the `loadedSVM` instance is isolated. However, if you run this test in parallel/sequentially along with other tests, the prediction starts to return different values.
We should investigate why this is happening and devise a solution.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the unit test for the bodyfat_scale.txt dataset and the SVM loadWASM(), train(), predict(), and free() flow shown in the issue. Run that test alone and alongside the other tests to reproduce the differing predictions, then trace shared WASM or libsvm state. Done means predictions remain consistent when tests run in parallel or sequentially, with the regression covered by a passing test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, typescript, wasm
- Domain
- machine-learning, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100