truncated utf-8 byte sequences force an encoding override

What steps will reproduce the problem?

1. # telnet www.arnaudmontebourg.fr 80
Trying 213.186.33.16...
Connected to www.arnaudmontebourg.fr.
Escape character is '^]'.
GET /?feed=rss2 HTTP/1.1
Host: www.arnaudmontebourg.fr 

Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
...
<?xml version="1.0" encoding="UTF-8"?>
...
<title>Retour d’Algérie : sortir des querelles du passé</title>
...

2.
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import feedparser
>>> d = feedparser.parse("http://www.arnaudmontebourg.fr/?feed=rss2")
>>> print d
...
'title': u'Retour d\xe2\x80\x99Alg\u0102\u0160rie : sortir des querelles du 
pass\u0102\u0160'

3.
print u'Retour d\xe2\x80\x99Alg\u0102\u0160rie : sortir des querelles du 
pass\u0102\u0160'

What is the expected output? What do you see instead?
Expected output: Retour d’Algérie : sortir des querelles du passé
Instead: Retour dâAlgĂŠrie : sortir des querelles du passĂŠ


What version of the product are you using? On what operating system?
Debian GNU/Linux 6.0
Linux 2.6.32-5-amd64 #1 SMP Mon Oct 3 03:59:20 UTC 2011 x86_64 GNU/Linux
python-feedparser - Universal Feed Parser for Python 

Python 2.6.6

Original issue reported on code.google.com by pelat.la...@gmail.com on 26 Nov 2011 at 3:01

Reproduce the problem with feedparser.parse("http://www.arnaudmontebourg.fr/?feed=rss2") using the reported Python 2.6 environment, then trace how the response encoding is handled. Done means the title is decoded as “Retour d’Algérie : sortir des querelles du passé” without requiring an encoding override.