matplotlib / matplotlib/basemap

Python 3: make decoding shapefile fields more lenient...

Open
#187 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
817
Forks
395
PR merge metrics
No merged PRs in 30d

Description

In python 3, field values of shapefile are parsed in shapefile.py, as shown below:

def u(v):
    if PYTHON3:
        if isinstance(v, bytes):
            # For python 3 decode bytes to str.
            return v.decode('utf-8')
        elif isinstance(v, str):
            # Already str.
            return v
        else:
            # Error.
            raise Exception('Unknown input type')
    else:
        # For python 2 assume str passed in and return str.
        return v

But in the case v.decode fails in the first if (when v is an instance of bytes), then the file won't be drawn. I propose to change it as follows (to make it work):

def u(v):
    if PYTHON3:
        if isinstance(v, bytes):
            # For python 3 decode bytes to str.
            return v.decode('utf-8', errors='ignore')
        elif isinstance(v, str):
            # Already str.
            return v
        else:
            # Error.
            raise Exception('Unknown input type')
    else:
        # For python 2 assume str passed in and return str.
        return v

Here is the link to the shape on which the original version fails and the changed one works:
https://dl.dropboxusercontent.com/u/4629759/network.zip

Cheers

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in shapefile.py at the Python 3 field-decoding helper and reproduce the failure with the linked network.zip shape. Verify that fields containing undecodable bytes no longer prevent the shape from being drawn, while existing string and non-byte handling remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.