imposm / osm-bright fails with UnicodeEncodeError
Nobody has claimed this yet.
- Dominant language
- CartoCSS
- Stars
- 703
- Forks
- 222
- PR merge metrics
- No merged PRs in 30d
Description
Using imposm fails with an UnicodeEncodeError when processing relations.
I processed an OSM file from Geofabrik (alps.osm.pbf). At the end it failed with the following error repeated 4 times, and the process hanged.
Last message was: relations: 10k/79k. TileMill complains about missing relations.
$ imposm -U postgres -d osm -m mapbox-osm-bright-c9bc5ee/imposm-mapping.py --read --write --optimize --deploy-production-tables alps_20130208.osm.pbf
loading mapbox-osm-bright-c9bc5ee/imposm-mapping.py as mapping
password for postgres at localhost:
[11:38:55] ## reading alps_20130208.osm.pbf
[11:38:55] coords: 78954k nodes: 1579k ways: 11279k relations: 78k (estimated)
[11:51:19] coords: 103636k nodes: 365k ways: 12045k relations: 73k
[11:51:21] reading took 12m 25s
[11:51:21] ## dropping/creating tables
[11:51:25] ## writing data
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 505, in run
self.__target(*self.__args, **self.__kwargs)
File "/usr/local/lib/python2.7/dist-packages/imposm/dbimporter.py", line 87, in db_importer
db.insert(mapping, insert_data)
File "/usr/local/lib/python2.7/dist-packages/imposm/db/postgis.py", line 104, in insert
self.cur.executemany(insert_stmt, insert_data)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe0' in position 9: ordinal not in range(128)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the traceback locations in imposm/dbimporter.py and imposm/db/postgis.py, then reproduce the failure with the shown imposm command and alps_20130208.osm.pbf input. Trace the relation-writing path and verify that processing completes without the UnicodeEncodeError or hang, with relations available to TileMill.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100