Insanely slow on large files
- Dominant language
- JavaScript
- Stars
- 5k
- Forks
- 577
- PR merge metrics
- No merged PRs in 30d
Description
I tried `html-minifier` but it was insanely slow on one of the first test files I fed it. I looked at #546 but that slowdown was on invalid input and supposedly fixed. As far as I can tell, the input file was well formed, but quite large: 3,450,556 bytes. Still, much smaller than [ES6 table](https://kangax.github.io/compat-table/es6/) listed in the README as being successfully minimized.
Just reading in the file and doing the default (which is basically nothing) took 3.5 minutes (220 seconds):
```
$ time html-minifier -o x.html index.html
real 3m30.062s
user 3m29.046s
sys 0m0.537s
```
Running `html-minifier` with real minification options took, well, I don't know, I gave up after 25 hours (90,000 seconds) and stopped it.
```
$ time html-minifier --collapse-whitespace --conservative-collapse --collapse-inline-tag-whitespace --case-sensitive \
--continue-on-parse-error --preserve-line-breaks --prevent-attributes-escaping --remove-optional-tags \
--keep-closing-slash --remove-tag-whitespace -o x.html index.html
```
## workaround
Giving up on `html-minifier` in favor of [minify](https://github.com/tdewolff/minify) which minified the same page in well under 0.1 seconds:
```
$ time minify --html-keep-end-tags --html-keep-document-tags --html-keep-whitespace -v -o x.html index.html
infer mimetype from file extensions
minify to output file x.html
(61.852901ms, 3.5 MB, 223 kB, 6.4%, 56 MB/s) - index.html to x.html
finished in 65.434042ms
real 0m0.072s
user 0m0.028s
sys 0m0.053s
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the html-minifier CLI commands against the reported 3,450,556-byte index.html file, first with default options and then with the listed minification options. Compare runtime with the supplied baseline and identify where processing becomes excessively slow; done means large, well-formed input no longer takes minutes or hangs for hours.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- cli, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100