Basemap readshapefile should read shapefile for the long/lat specified in the Basemap instance.

@wxguy commented a day ago

I need to plot more than 100 images for a specific area which are not included in default Basemap. I have read lot of documentation regarding speeding up the matplotlib/basemap plotting examples. Most of them lead to this example https://github.com/matplotlib/basemap/blob/master/examples/hires.py . I agree that picklling does improve the code for Basemap instance. However, I also required to read the shape file once the Basemap instance is created like following:-

south = 0
north = 5
west = 70
east = 85

following is a global instance and read once only

m = Basemap(projection='merc', llcrnrlat=south, urcrnrlat=north,
llcrnrlon=west, urcrnrlon=east, resolution='c')

the following line a part of plotting function and not a continuous line from above

m.readshapefile('data/gis-data/world_countries/ne_10m_admin_0_countries_lakes', 'ne_10m_admin_0_countries_lakes', linewidth=0.7)

Please see that I am reading shape file from NE 10 resolution file (~ 8 mb). When I read this shape file it takes about 10-15 sec which is too much for just map. This is a big problem as the code (m.readshapefile) reside inside the map plotting function and it has to be called to plot boundaries each time.

I have a question and a proposal (sort of) here.

Question:-
How to improve the plotting time by reducing the reading time of shape file using the default m.readshapefile code. There could be some solution similar to pickle which I am not aware. Best people can share their experience here (with some example code).

Proposal (sort of):-
The major problem with the reading shape file which I think is basemap reading the entire shapefile. Why should it read all the data from shapefile if you the lat/lon which are already specified in the basemap instance. Therefore, readshapefile should only read data within the lon/lat specified from the basemap instance. I hope that this way shapefile read time may be reduced greatly.

I may or may not be correct somewhere above. Forgive my ignorance. I hopr that some one may help me on reading shapefile/ plotting the map faster.

Note ::: Please don't suggest me to use lower resolution basemap/shapefile.
@WeatherGod
Matplotlib Developers member
WeatherGod commented a day ago
I think the difficulty with this request is that Basemap uses the pyshp
package for handling reading of the shapefiles. The last time I checked,
pyshp does not support any sort of constraint-based loading, so all of the
data has to be read in anyway. Now, perhaps it might be possible to find
some optimizations in the overall pipeline, I would have to do some
profiling to find out where the bottlenecks are. It is indeed, too slow and
painful at the moment.
…
@wxguy
wxguy commented 20 hours ago

Thanks for the clarification.

Is it not possible to pickle the shapefile and use it again. At least this would improve the plotting speed. Also is it possible to use contour function so that shapefile is not required to be called many times?
@wxguy
wxguy commented 20 hours ago

It would be great and highly appreciated if this request is accepted.

Moved from https://github.com/matplotlib/matplotlib/issues/5993

Basemap の readshapefile メソッドから始め、shapefile データの読み込みとプロットに pyshp パッケージをどのように使用しているかを調査します。既存のパイプラインをプロファイリングし、その後、繰り返しのプロットでファイル全体の再読み込みを避けるか、読み込みを Basemap の範囲に限定できるかを判断します。低解像度の shapefile を必要とせず、測定可能な改善が得られれば完了です。