Performance issues when reading multiple packed structures
Nobody has claimed this yet.
- Dominant language
- Julia
- Stars
- 27
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
I've encountered this in replacing manually written out read and write methods with StructIO in visr/LasIO.jl#10. Although using StructIO is elegant, it's also slower.
I've put up a gist here: https://gist.github.com/evetion/2b57d6105cca39b2d3c6ef670a5cc393 with the following results for reading a thousand TwoUInt64s.
➜ julia performance.jl
Using StructIO:
4.143 ms (2000 allocations: 62.50 KiB)
Using read_generic_array:
6.138 ms (12000 allocations: 406.25 KiB)
Using read_generic_tuple:
480.796 μs (8000 allocations: 265.63 KiB)
Using read_written_out:
29.229 μs (2000 allocations: 31.25 KiB)
Using generated_read:
29.704 μs (2000 allocations: 31.25 KiB)
The handwritten read version, which also can be generated, is ~200 times faster.
I know this is an unfair comparison, as StructIO has much more functionality than these simple read functions, but it seems it could be faster, especially if you look at the allocations, which are on par.
Let me know if you can't duplicate these results, or if I'm missing a StructIO method for reading multiple packed structures.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the comparison in the referenced performance.jl gist and inspect the StructIO paths behind read_generic_array, read_generic_tuple, read_written_out, and generated_read. Compare their allocation and timing results for multiple packed structures, then document or implement a measured improvement over the current StructIO behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100