Having trouble reading UTF-8 encoded GeoJSON on Windows
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 141
- Forks
- 27
- PR merge metrics
- No merged PRs in 30d
Description
I can't figure out what I'm doing wrong. I have a LF delimited geojson I'm trying to upload as a source. The GeoJson contains some non-ANSI characters and so is encoded in UTF-8.
The command I am using looks something like this:
tilesets add-source {account} source-name source-file.geojson
I can't figure out how to get the JSON to parse using UTF-8, it seems like it always tries to use cp1252.py.
So I'm getting this error:
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5413: character maps to undefined
Full stack trace:
load_entry_point('tilesets-cli==0.3.2.dev0', 'console_scripts', 'tilesets')()
File "c:\tools\python\python37\lib\site-packages\click\core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "c:\tools\python\python37\lib\site-packages\click\core.py", line 717, in main
rv = self.invoke(ctx)
File "c:\tools\python\python37\lib\site-packages\click\core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\tools\python\python37\lib\site-packages\click\core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\tools\python\python37\lib\site-packages\click\core.py", line 555, in invoke
return callback(*args, **kwargs)
File "c:\tools\python\python37\lib\site-packages\click\decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\tools\python\python37\lib\site-packages\tilesets\scripts\cli.py", line 320, in add_source
for feature in features:
File "c:\tools\python\python37\lib\site-packages\cligj\features.py", line 30, in normalize_feature_inputs
for feature in iter_features(iter(src)):
File "c:\tools\python\python37\lib\site-packages\cligj\features.py", line 121, in iter_features
text = "".join(chain([first_line], geojsonfile))
File "c:\tools\python\python37\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 5413: character maps to <undefined>```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at tilesets/scripts/cli.py line 320, where add_source iterates features, then trace the cligj normalize_feature_inputs path shown in the stack trace. Reproduce the Windows UTF-8 GeoJSON failure and determine the intended input-decoding behavior. Done means the command parses the supplied UTF-8 GeoJSON without the cp1252 UnicodeDecodeError, with coverage for the failing case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100