Randomized sampling can fail for very large tensors
- Dominant language
- Julia
- Stars
- 7
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Right now the randomized methods index the random sample by the column position of the matricized tensor using Int64. When a tensor has a very large column space, greater than 10^20, the number of columns overflows Int64 and returns negative values. The system needs to be modified to index by the position in each mode of the tensor. This would then only overflow when the dimension of a single mode is greater than 10^20.
The solution is that all the pivot structures need to be modified to hold onto matrices of positions instead of vectors. Multi-index positions can easily be converted into other values.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the randomized methods and pivot structures described in the issue, then inspect how matricized column positions are represented. Reproduce the overflow with a very large-column tensor; the work is done when pivot structures retain per-mode position matrices and only single-mode dimensions can overflow.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100