ezyang / ezyang/htmlpurifier

Cache memory leak when using PHP 7.4.x

Open
#270 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
PHP
Stars
3.4k
Forks
361
Avg merge
3d 41m
Merged PRs (30d)
2

Description

I am currently investigation a memory leak with PHP 7.4.x. I upgraded from 7.2.26 to 7.4.10 and subsequently htmlpurifier from 4.10 to 4.13 as 4.10 is not PHP 7.4 compatible. Since then I have a huge issue with memory leaks, in my application a couple dozen of calls can leak ~512MB. I am still investigating the root cause, but I hope somebody has an idea what might happen. I will try to strip down my application to the minimum code required for reproducing the issue.

Things I found out so far:

  • PHP 7.4.x is affected, I tested 7.4.0 and 7.4.10. 7.3.22 and 7.2.26 do not have this issue
  • HTML-Purifier 4.12 and 4.13 have this issue, 4.10 is not PHP 7.4 compatible and 4.11 either had this issue or was not 7.4. compatible, but I tested it.
  • With Cache.DefinitionImpl set to null no memory leaking occurs
  • The cache directory does not grow.
  • Running php mem profiler shows that the unserialize statement cause the most memory allocations. I am running a lot of de/serialization and have no issues there.
  • It always happens at the same location.
Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/DefinitionCache/Serializer.php on line 73
...
   45.9637  516876080  16. HTMLPurifier->purify(string(7), ???) /home/someproject/Classes/Util/HTMLSanitizer.php:164
   45.9637  516879240  17. HTMLPurifier_Generator->__construct(class HTMLPurifier_HTML5Config, class HTMLPurifier_Context) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier.php:158
   45.9637  516879240  18. HTMLPurifier_HTML5Config->getHTMLDefinition(???, ???) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/Generator.php:74
   45.9637  516879240  19. HTMLPurifier_HTML5Config->getDefinition(string(4), false, false) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/Config.php:415
   45.9637  516879240  20. HTMLPurifier_HTML5Config->getDefinition(string(4), true, true) /home/someproject/libs/htmlpurifier-html5-master/library/HTMLPurifier/HTML5Config.php:86
   45.9637  516879240  21. HTMLPurifier_DefinitionCache_Decorator_Cleanup->get(class HTMLPurifier_HTML5Config) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/Config.php:579
   45.9637  516879240  22. HTMLPurifier_DefinitionCache_Decorator_Cleanup->get(class HTMLPurifier_HTML5Config) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/DefinitionCache/Decorator/Cleanup.php:70
   45.9637  516879240  23. HTMLPurifier_DefinitionCache_Serializer->get(class HTMLPurifier_HTML5Config) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/DefinitionCache/Decorator.php:81
   45.9640  517026992  24. unserialize(string(132328)) /home/someproject/libs/htmlpurifier-4.12.0-lite/library/HTMLPurifier/DefinitionCache/Serializer.php:73

config:

"AutoFormat.AutoParagraph" => true,
"AutoFormat.Linkify" => true,
"AutoFormat.RemoveEmpty" => true, 
"AutoFormat.RemoveSpansWithoutAttributes" => true,
"Core.RemoveProcessingInstructions" => true,
"URI.AllowedSchemes" => array (
    'http' => true,
    'https' => true,
    'mailto' => true
),
"URI.DefaultScheme" => "https",
"Output.TidyFormat" => true,
"HTML.ForbiddenAttributes" => array("class", "@data-community-tooltip"),
"HTML.ForbiddenElements" => [ "iframe", "form", "button", "input", "body", "html", "frameset", "head", "meta", "script", "style" ],
"Attr.ForbiddenClasses" => array("bb_ul", "bb_tag"),
"Core.CollectErrors" => true,
"Cache.SerializerPath" => "/some/path"

More info will follow.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with HTMLPurifier/DefinitionCache/Serializer.php at line 73 and the Cache.DefinitionImpl and Cache.SerializerPath settings. Reproduce the leak across the PHP and HTML-Purifier versions listed, focusing on repeated unserialize calls in the DefinitionCache path. Done means the cause is isolated with a minimal reproduction and the memory behavior is verified after the change.

Written by the indexing model from the issue text.

Assessment

Tech stack
php
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.