xml serialize optimizations

	jack reported
Trac ID	trac#2477
Type	enhancement
Created	2007-02-24 03:48:01Z
Branch	https://github.com/twisted/twisted/tree/faster-domish-serialization-2477

twisted.words.xish's xml serialization is quite slow. It is routinely showing up as the top cycle eater in profiles. The following two patches attempt to address this.

The first one gets rid of all string formatting operations in favor of more calls to write(). This results in about a 17% speedup.

The second patch caches the results of escapeToXml and achieves about another 25% performance boost at the cost of memory. This patch is usable but could probably be done better.

The performance is still not great, but I'm unsure how further gains can be made. I tried amny things includes cStringIO and a non-recursive implementation (both made the code slower, not faster). I think the real solution is to make domish a light wrapper around some fast underlying C library.

Attachments:

• domish.diff (5010 bytes) - added by jack on 2007-02-24 03:48:57Z - replace string formatting with more calls to write()
• domish2.diff (1136 bytes) - added by jack on 2007-02-24 03:49:43Z - cache results of escapeToXml
• test.log.bz2 (130005 bytes) - added by jack on 2007-02-25 04:00:44Z - test data from our xmpp component
• timer.py (1591 bytes) - added by jack on 2007-02-25 04:01:08Z - benchmark script
• cserialize.diff (1785 bytes) - added by thijs on 2010-01-19 02:24:42Z - diff from #3633

trac-id__2477 2477
type__enhancement enhancement
reporter__jack jack
priority__normal normal
milestone__ 
branch__branches_faster_domish_serialization_2477 branches/faster-domish-serialization-2477
branch_author__exarkun exarkun
status__new new
resolution__None None
component__words words
keywords__xish_domish_xml_performance xish domish xml performance
time__1172288881000000 1172288881000000
changetime__1300471392000000 1300471392000000
version__None None
owner__ 
cc__jack cc__twonds cc__ralphm cc__exarkun cc__thijs

Start with the faster-domish-serialization-2477 branch and review the attached domish.diff, domish2.diff, cserialize.diff, and timer.py. Compare the proposed serialization and escapeToXml changes with current behavior and benchmarks. Done means a measured serialization improvement without breaking XML output or imposing an unacceptable memory cost.