src-d / src-d/lapjv

Possible solution to large data sets?

Open
#46 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
278
Forks
36
PR merge metrics
No merged PRs in 30d

Description

This has already been mentioned in another issue, but I thought it merits its own thread, and besides the other one hasn't been updated for quite a while.

Since the algorithm complexity is $O(n^3)$, the computation can get quite costly quite quickly, both time-wise and memory-wise. Are there any solutions or approximation algorithms to address this problem?

In the other thread a data size of 55K was already considered a big data set, and I have a data set that is thousands of times its size... Any hope?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading this issue and the referenced issue #7 to understand the existing large-data discussion. Investigate whether the requested result is an approximation algorithm, a more scalable implementation, or guidance for large inputs; done would require a decided approach with clear performance expectations and documented or tested behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.