Out of memory errors on App Engine

We use feedparser 5.1 in our app running on Google App Engine, and we parse 
hundreds of thousands of feeds on a regular basis. But I've come across a few 
feeds that have proved problematic. For example, this feed (copy attached):

http://www.moneysupermarket.com/shopping/blog/feed/

    The feed size is about 0.5MB. It's big but not huge. When I run the parser on my local machine with plenty of RAM it parses without problems. But when I run it on App Engine instances with 128MB of RAM (i.e. the amount of RAM available to the app), the instance runs out of memory and gets killed by the system. I tried bigger instances with 256MB of RAM and those failed as well. In comparison, when parsing other feeds, the instance memory usage rarely goes above 60MB, but with this feed it jumps to 300MB or more before getting killed. 

    I don't think feedparser typically needs as much RAM as 50X the feed size, so I'm guessing there might be a problem somewhere.

Original issue reported on code.google.com by wal...@ninua.com on 16 Dec 2011 at 9:19

Attachments:

• out_of_memory_feed.xml

Start by reproducing the failure with the attached out_of_memory_feed.xml using feedparser 5.1 in a 128MB App Engine environment, then compare it with the reported local parse. Done means identifying and fixing the cause of the excessive memory growth without breaking parsing of this feed.