rust-ml / rust-ml/linfa

linfa-svm learning speed and memory allocation

Open
#308 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted
Dominant language
Rust
Stars
4.7k
Forks
334
Avg merge
39m
Merged PRs (30d)
1

Description

Hello,

I cannot get linfa-svm to produce any result as the operations take way too much memory and time:

Multi-class classifier full data:

[2023-07-11T08:18:38Z INFO  be] Fit SVM classifier with #204242 training points
memory allocation of 333718356512 bytes failed
Aborted (core dumped)

Multi-class classifier limited data:

[2023-07-11T09:16:38Z INFO  be] Fit SVM classifier with #10214 training points

^C

real    120m7.427s
user    119m52.661s
sys     0m2.573s

Regressor:

[2023-07-11T11:46:22Z INFO  be] Fit SVM regressor with #10214 training points

(still running after 20 mins and will be probably running much, much longer)

I guess I could trim the input data set further but 10K input data points doesn't seem like way too many.
Is there any other way to speed things up?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linfa-svm SVM classifier and regressor fitting paths using the reported 10,214 and 204,242 training-point cases. Measure runtime and memory, then identify why fitting is impractical at those sizes. Done means multi-class classification and regression complete without the reported allocation failure or excessive runtime.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.