machinelearnjs / machinelearnjs/libsvm-ts

libsvm instances during the unit tests are influenced by other coexisting instances and cause test failures

Open
#12 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug good first issue help wanted
Dominant language
C++
Stars
4
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Context:
While trying to cover tests for WASM loaded SVM's `predict`, I've encountered an issue of incorrect prediction results between running a test by itself vs running tests in-band.

Problematic test:

```
it('should predict bodyfat scale dataset using C_SVC and LINEAR', () => {
const rawData = fs.readFileSync(path.join(__dirname, '../samples/bodyfat_scale.txt'), 'utf-8');
const data = rawData.split('\n').map((line) => line.split(' ').filter((el) => el));
const labels = data.map((line) => +line.splice(0, 1)[0]);
const features = data.map((line) => line.map((el) => +el.split(':')[1]));

const svm = new SVM({
type: SVMTypes.C_SVC,
kernel: KernelTypes.LINEAR,
epsilon: 0.001,
quiet: true,
probabilityEstimates: true,
});
return svm.loadWASM().then((loadedSVM) => {
loadedSVM.train({ samples: features, labels });
const predResult = loadedSVM.predict({
samples: [features[0], features[1]],
});
loadedSVM.free();
// TODO: It seems like other tests are influencing the result of this.
// I don't know why so far, we should figure it out
expect(predResult).toEqual([1, 1]);
});
});
```

When you run this test on its own, it passes without an issue since the `loadedSVM` instance is isolated. However, if you run this test in parallel/sequentially along with other tests, the prediction starts to return different values.

We should investigate why this is happening and devise a solution.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the unit test for the bodyfat_scale.txt dataset and the SVM loadWASM(), train(), predict(), and free() flow shown in the issue. Run that test alone and alongside the other tests to reproduce the differing predictions, then trace shared WASM or libsvm state. Done means predictions remain consistent when tests run in parallel or sequentially, with the regression covered by a passing test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, typescript, wasm
Domain
machine-learning, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.