jasonrogena / jasonrogena/php-excel-reader
Perfomance issues with big files
- Dominant language
- PHP
- Stars
- 1
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
```
I've got a problem when using reader2 for parsing big Excel files (from 5Mb
filesize)
Seems like it takes more then 75Mb of RAM to build full structure of all
raws and columns. And actually it's more then most of memory limits for
general php settings (mine is 64Mb). I was surprised because first reader
(from Vadim Tkachenko) takes about 55Mb RAM for the same file parsing.
Which is also a lot, but still within limit.
So first idea was to add functionality of to skip some sheets and skip
unneeded cell info (links, colors etc.). I've tried it myself but it gave
me not more then 5Mb RAM saving.
The second idea is to go the same way with XML parsers where beside full
getting of DOM there are bunch of agile readers which get only what we need.
So idea is when we work with big file, like mine. First we parse document
and get only meta data: number of sheets, of columns and rows. And then
call something like $reader->getCell(sheet, col, row) and parse data from
that row.
So in the memory will be only pointers to needed bits, but not all the document
I have not time now to work on it, but if I came up with smth I'll post it here
Thanks,
Nick
```
Original issue reported on code.google.com by `nikolai....@gmail.com` on 10 Jun 2009 at 2:14
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing how reader2 currently builds the full structure for a large Excel file and reproduce the reported memory usage with a file around 5 MB. Compare its usage with the first reader, then evaluate the proposed metadata and getCell-style access pattern. Done should mean large files can be read incrementally without loading the full document into memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100