Gabriella439 / Gabriella439/Haskell-Succinct-Vector-Library
Indexing a 78MB bit-string blows up memory
- Dominant language
- Haskell
- Stars
- 20
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
I encountered the following problem while using the library.
The following sequence of commands in `ghci` will start thrashing.
```
:set -XBangPatterns
import Data.Vector
import System.IO.MMap
import Data.Word
import Data.Vector.Storable.ByteString
import qualified Data.Vector.Storable as VS
fp <- mmapFileByteString "78mb.json" Nothing
let !v = (byteStringToVector fp) :: VS.Vector Word64
let !w = convert v :: Vector Word64
let !bv = prepare w
```
I replaced code for `v1` with dummy vector:
```
v1 :: Unboxed.Vector Int
v1 = Unboxed.singleton 1
-- flip Unboxed.snoc (len * 64)
-- ( Unboxed.map (\(_, i) -> i)
-- ( Unboxed.filter (\(j, _) -> j `rem` 512 == 0)
-- ( Unboxed.imap (,)
-- ( oneIndices 0 (Unboxed.length v)
-- v ))))
```
And the problem went away.
Note that calling `oneIndices 0 (Unboxed.length w) w` alone is sufficient to blow up memory.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.