GenericMappingTools / GenericMappingTools/gmtserver-admin

Changes to @earth_day and @earth_night grids from GMT 6.4 to 6.5?

Open
#257 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
7
Forks
3
Avg merge
9h 22m
Merged PRs (30d)
1

Description

Just checking something that I noticed at https://github.com/GenericMappingTools/pygmt/pull/2963#discussion_r1444072938. Are the Blue Marble `@earth_day_01d_p` and Black Marble `@earth_night_01d_p` datasets only downloadable with GMT 6.5.0 (and not GMT 6.4.0 or older) now?

E.g. doing `gmt which @earth_day_01d_p -Vd` with GMT 6.4.0 gives:

```
gmt [DEBUG]: GMT_Create_Session: Terminal width = 190
gmt [DEBUG]: Obtained the ppid from parent: 3763
gmt [DEBUG]: Enter: gmtinit_new_GMT_ctrl
gmt [DEBUG]: GMT->session.SHAREDIR = ~/mambaforge/envs/pygmt/share/gmt
gmt [DEBUG]: GMT->session.HOMEDIR = ~
gmt [DEBUG]: GMT->session.USERDIR = ~/.gmt [created]
gmt [DEBUG]: GMT->session.CACHEDIR = ~/.gmt/cache [created]
gmt [DEBUG]: GMT: 0. Will try to find subdir=postscriptlight stem = PSL_custom_fonts suffix=.txt
gmt [DEBUG]: GMT: 1. gmt_getsharepath trying current dir
gmt [DEBUG]: GMT: 2. gmt_getsharepath trying USERDIR ~/.gmt
gmt [DEBUG]: GMT: 3. gmt_getsharepath trying USERDIR subdir ~/.gmt/postscriptlight
gmt [DEBUG]: GMT: 4. gmt_getsharepath trying SHAREDIR subdir ~/mambaforge/envs/pygmt/share/gmt/postscriptlight
gmt [DEBUG]: GMT: 5. gmt_getsharepath trying SHAREDIR ~/mambaforge/envs/pygmt/share/gmt
gmt [DEBUG]: GMT: 6. gmt_getsharepath failed
gmt [DEBUG]: Map distance calculation will be Cartesian
gmt [DEBUG]: Exit: gmtinit_new_GMT_ctrl
gmt [DEBUG]: Enter: New_PSL_Ctrl
gmt [DEBUG]: Exit: New_PSL_Ctrl
gmt [DEBUG]: Enter: gmt_manage_workflow
gmt [DEBUG]: Exit : gmt_manage_workflow
gmt [DEBUG]: Enter: PSL_beginsession
gmt [DEBUG]: Exit : PSL_beginsession
gmt [DEBUG]: Enter: PSL_setdefaults
gmt [DEBUG]: Exit : PSL_setdefaults
gmt [DEBUG]: Enter: gmtlib_io_init
gmt [DEBUG]: Exit : gmtlib_io_init
gmt [DEBUG]: Enter: gmt_hash_init
gmt [DEBUG]: Exit: gmt_hash_init
gmt [DEBUG]: Enter: gmt_hash_init
gmt [DEBUG]: Exit: gmt_hash_init
gmt [DEBUG]: Enter: gmt_reload_settings
gmt [DEBUG]: The PROJ_GEODESIC set to Vincenty
gmt [DEBUG]: Look for file ~/gmt.conf
gmt [DEBUG]: Look for file ~/.gmt/gmt.conf
gmt [DEBUG]: Look for file ~/.gmt/server/gmt.conf
gmt [DEBUG]: Look for file ~/.gmt/cache/gmt.conf
gmt [DEBUG]: Could not find file gmt.conf
gmt [DEBUG]: Exit: gmt_reload_settings
gmt [DEBUG]: Enter: gmtlib_plot_C_format
gmt [DEBUG]: Exit: gmtlib_plot_C_format
gmt [DEBUG]: Enter: gmtinit_get_history
gmt [DEBUG]: Initialize FFTW with 16 threads.
gmt [DEBUG]: GMT_Create_Session initialized GMT structure
gmt [DEBUG]: Loading core GMT shared library: libgmt.so
gmt [DEBUG]: Shared Library # 0 (core). Path = libgmt.so
gmt [DEBUG]: Loading GMT plugins from: ~/mambaforge/envs/pygmt/lib/gmt/plugins
gmt [DEBUG]: Shared Library # 1 (supplements). Path = ~/mambaforge/envs/pygmt/lib/gmt/plugins/supplements.so
gmt [DEBUG]: Local file ~/.gmt/server/gmt_data_server.txt found
gmt [DEBUG]: File ~/.gmt/server/gmt_data_server.txt less than 24 hours old, refresh is premature.
gmt [DEBUG]: Load contents from ~/.gmt/server/gmt_data_server.txt
gmt [DEBUG]: Local file ~/.gmt/server/gmt_hash_server.txt found
gmt [DEBUG]: File ~/.gmt/server/gmt_hash_server.txt less than 24 hours old, refresh is premature.
gmt [WARNING]: Remote dataset given to a data processing module but no registration was specified - default to gridline registration (if available)
gmt [DEBUG]: Input remote grid modified to have registration: @earth_day_01d_p
gmt [DEBUG]: Map distance calculation will be using great circle approximation with authalic auxiliary latitudes and authalic (R_2) radius = 6371007.1809 m, in meter.
gmt [DEBUG]: Revised options: @earth_day_01d_p -Vd
gmtwhich [DEBUG]: gmtapi_init_export: Passed family = Data Table and geometry = Non-Geographical
gmtwhich [DEBUG]: Object ID 0 : Registered Data Table Stream 7fd32ddf2780 as an Output resource with geometry Non-Geographical [n_objects = 1]
gmtwhich [DEBUG]: gmtapi_init_export: Added stdout to registered destinations
gmtwhich [DEBUG]: GMT_Init_IO: Returned first Output object ID = 0
gmtwhich [DEBUG]: GMT_Begin_IO: Initialize record-by-record access for Output
gmtwhich [DEBUG]: gmtapi_next_io_source: Selected object 0
gmtwhich [INFORMATION]: Writing Data Table to Standard Output stream
gmtwhich [DEBUG]: GMT_Begin_IO: Output resource access is now enabled [record-by-record]
gmtwhich [DEBUG]: Look for file earth_day_01d_p.tif in ~/.gmt
gmtwhich [DEBUG]: Look for file earth_day_01d_p.tif in ~/.gmt/cache
gmtwhich [DEBUG]: Look for file earth_day_01d_p.tif in ~/.gmt/server
gmtwhich [ERROR]: File earth_day_01d_p.tif not found!
gmtwhich [DEBUG]: GMT_End_IO: Output resource access is now disabled
gmtwhich [DEBUG]: gmtlib_unregister_io: Unregistering object no 0 [n_objects = 0]
gmt [DEBUG]: Entering GMT_Destroy_Session
```

But with GMT 6.5.0, `gmt which @earth_day_01d_p` works fine.

Also, I noticed that the internal data structure of the Blue/Black Marble images have changed from a 1-band ColorInterp/Color Table structure to a 3-band structure:

```diff
diff --git a/gdalinfo_earth_day b/gdalinfo_earth_day
index 75e7bbc91..02d818804 100644
--- a/gdalinfo_earth_day
+++ b/gdalinfo_earth_day
@@ -1,5 +1,5 @@
Driver: GTiff/GeoTIFF
-Files: earth_day_01d_p.tif
+Files: ~/.gmt/cache/earth_day_01d_p.tif
Size is 360, 180
Coordinate System is:
GEOGCRS["unknown",
@@ -26,269 +26,40 @@ Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=DEFLATE
- INTERLEAVE=BAND
- PREDICTOR=2
+ INTERLEAVE=PIXEL
Corner Coordinates:
Upper Left (-180.0000000, 90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"N)
Lower Left (-180.0000000, -90.0000000) (180d 0' 0.00"W, 90d 0' 0.00"S)
Upper Right ( 180.0000000, 90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"N)
Lower Right ( 180.0000000, -90.0000000) (180d 0' 0.00"E, 90d 0' 0.00"S)
Center ( 0.0000000, 0.0000000) ( 0d 0' 0.01"E, 0d 0' 0.01"N)
-Band 1 Block=360x22 Type=Byte, ColorInterp=Palette
- Color Table (RGB with 256 entries)
- 0: 248,224,168,255
- 1: 240,224,152,255
- 2: 104,104,96,255
- 3: 16,24,56,255
- 4: 24,28,24,255
- 5: 104,80,48,255
- 6: 220,224,228,255
- 7: 48,56,32,255
- 8: 184,192,176,255
- 9: 104,104,40,255
- 10: 16,48,8,255
- 11: 96,96,56,255
- 12: 188,184,184,255
- 13: 128,112,60,255
- 14: 48,72,24,255
- 15: 8,16,48,255
- 16: 216,200,192,255
- 17: 80,80,28,255
- 18: 56,48,44,255
- 19: 72,88,24,255
- 20: 204,168,140,255
- 21: 128,136,136,255
- 22: 72,68,40,255
- 23: 144,120,68,255
- 24: 144,144,136,255
- 25: 88,80,48,255
- 26: 168,160,152,255
- 27: 88,104,40,255
- 28: 8,16,32,255
- 29: 168,116,84,255
- 30: 120,100,48,255
- 31: 200,192,184,255
- 32: 176,168,160,255
- 33: 136,112,80,255
- 34: 204,204,200,255
- 35: 184,152,124,255
- 36: 96,76,48,255
- 37: 88,72,48,255
- 38: 184,196,192,255
- 39: 28,56,48,255
- 40: 92,100,124,255
- 41: 184,176,176,255
- 42: 160,160,152,255
- 43: 132,144,144,255
- 44: 208,192,184,255
- 45: 128,100,64,255
- 46: 72,100,24,255
- 47: 36,48,16,255
- 48: 140,140,80,255
- 49: 152,144,136,255
- 50: 56,64,16,255
- 51: 168,168,168,255
- 52: 224,200,144,255
- 53: 176,176,168,255
- 54: 80,88,28,255
- 55: 152,144,144,255
- 56: 156,148,84,255
- 57: 160,160,160,255
- 58: 16,16,48,255
- 59: 168,176,168,255
- 60: 216,200,176,255
- 61: 16,8,48,255
- 62: 72,80,24,255
- 63: 132,140,124,255
- 64: 88,80,40,255
- 65: 152,152,176,255
- 66: 16,16,56,255
- 67: 20,48,24,255
- 68: 200,200,192,255
- 69: 56,48,72,255
- 70: 40,56,24,255
- 71: 120,136,56,255
- 72: 152,160,152,255
- 73: 8,8,24,255
- 74: 192,184,176,255
- 75: 248,244,172,255
- 76: 116,124,136,255
- 77: 88,68,40,255
- 78: 64,72,32,255
- 79: 172,180,176,255
- 80: 176,152,132,255
- 81: 24,40,24,255
- 82: 60,100,24,255
- 83: 168,148,128,255
- 84: 176,132,88,255
- 85: 200,192,192,255
- 86: 76,76,48,255
- 87: 152,116,80,255
- 88: 8,16,60,255
- 89: 152,152,144,255
- 90: 240,216,152,255
- 91: 52,80,32,255
- 92: 20,24,40,255
- 93: 132,144,156,255
- 94: 88,88,48,255
- 95: 20,32,8,255
- 96: 84,68,28,255
- 97: 172,164,180,255
- 98: 152,152,152,255
- 99: 120,112,56,255
- 100: 88,88,40,255
- 101: 56,80,8,255
- 102: 80,80,40,255
- 103: 176,176,160,255
- 104: 48,56,8,255
- 105: 104,84,68,255
- 106: 48,120,76,255
- 107: 184,176,168,255
- 108: 108,96,56,255
- 109: 232,224,212,255
- 110: 16,16,40,255
- 111: 96,88,48,255
- 112: 144,152,144,255
- 113: 104,112,68,255
- 114: 220,180,148,255
- 115: 28,32,60,255
- 116: 56,72,8,255
- 117: 104,96,40,255
- 118: 56,72,24,255
- 119: 24,64,24,255
- 120: 148,164,160,255
- 121: 180,176,140,255
- 122: 216,208,196,255
- 123: 156,128,92,255
- 124: 72,84,40,255
- 125: 56,80,16,255
- 126: 72,72,24,255
- 127: 48,44,12,255
- 128: 124,108,96,255
- 129: 64,80,24,255
- 130: 132,136,148,255
- 131: 200,200,184,255
- 132: 120,100,68,255
- 133: 16,28,24,255
- 134: 144,144,144,255
- 135: 116,120,124,255
- 136: 248,224,184,255
- 137: 184,184,168,255
- 138: 72,80,32,255
- 139: 152,148,160,255
- 140: 56,76,44,255
- 141: 232,208,144,255
- 142: 176,168,168,255
- 143: 104,96,88,255
- 144: 60,56,32,255
- 145: 168,152,92,255
- 146: 184,184,176,255
- 147: 152,144,124,255
- 148: 232,180,120,255
- 149: 208,204,192,255
- 150: 152,152,128,255
- 151: 188,180,108,255
- 152: 48,56,16,255
- 153: 28,60,40,255
- 154: 56,64,24,255
- 155: 160,152,144,255
- 156: 48,72,8,255
- 157: 168,160,160,255
- 158: 228,228,184,255
- 159: 48,64,8,255
- 160: 160,152,152,255
- 161: 148,112,64,255
- 162: 32,48,8,255
- 163: 80,68,40,255
- 164: 132,140,100,255
- 165: 84,104,28,255
- 166: 36,48,28,255
- 167: 188,192,184,255
- 168: 168,168,160,255
- 169: 240,248,248,255
- 170: 60,52,24,255
- 171: 248,248,220,255
- 172: 36,36,8,255
- 173: 144,152,152,255
- 174: 32,64,8,255
- 175: 96,80,80,255
- 176: 72,140,148,255
- 177: 48,72,16,255
- 178: 200,184,176,255
- 179: 148,144,152,255
- 180: 136,140,136,255
- 181: 244,200,144,255
- 182: 180,200,212,255
- 183: 160,168,160,255
- 184: 40,64,8,255
- 185: 56,72,16,255
- 186: 48,56,24,255
- 187: 64,72,24,255
- 188: 176,184,168,255
- 189: 48,64,16,255
- 190: 104,116,56,255
- 191: 28,24,56,255
- 192: 132,124,136,255
- 193: 72,72,32,255
- 194: 80,88,40,255
- 195: 56,88,12,255
- 196: 236,204,184,255
- 197: 72,64,28,255
- 198: 64,84,32,255
- 199: 24,48,8,255
- 200: 88,80,84,255
- 201: 240,232,236,255
- 202: 148,100,60,255
- 203: 148,116,100,255
- 204: 160,144,140,255
- 205: 208,132,88,255
- 206: 36,32,24,255
- 207: 148,132,132,255
- 208: 52,84,24,255
- 209: 196,180,168,255
- 210: 204,188,140,255
- 211: 24,16,52,255
- 212: 104,120,40,255
- 213: 240,184,144,255
- 214: 72,88,32,255
- 215: 104,84,40,255
- 216: 60,52,16,255
- 217: 208,200,184,255
- 218: 32,56,8,255
- 219: 24,56,8,255
- 220: 160,168,152,255
- 221: 116,136,164,255
- 222: 204,196,172,255
- 223: 48,84,12,255
- 224: 88,128,80,255
- 225: 64,64,24,255
- 226: 248,244,192,255
- 227: 172,180,192,255
- 228: 144,144,60,255
- 229: 40,56,8,255
- 230: 76,72,88,255
- 231: 40,72,24,255
- 232: 196,180,160,255
- 233: 152,168,132,255
- 234: 64,88,24,255
- 235: 64,64,16,255
- 236: 40,48,8,255
- 237: 220,152,124,255
- 238: 92,84,28,255
- 239: 204,208,220,255
- 240: 104,128,96,255
- 241: 132,168,164,255
- 242: 40,76,8,255
- 243: 188,184,196,255
- 244: 56,76,80,255
- 245: 156,148,104,255
- 246: 244,240,156,255
- 247: 68,80,16,255
- 248: 28,64,80,255
- 249: 152,176,188,255
- 250: 132,84,56,255
- 251: 8,8,48,255
- 252: 8,8,56,255
- 253: 8,40,24,255
- 254: 8,72,96,255
- 255: 248,248,248,255
+Band 1 Block=360x7 Type=Byte, ColorInterp=Undefined
+ Min=10.000 Max=255.000
+ Minimum=10.000, Maximum=255.000, Mean=49.508, StdDev=65.299
+ NoData Value=0
+ Metadata:
+ STATISTICS_MAXIMUM=255
+ STATISTICS_MEAN=49.507654320988
+ STATISTICS_MINIMUM=10
+ STATISTICS_STDDEV=65.299010755209
+ STATISTICS_VALID_PERCENT=100
+Band 2 Block=360x7 Type=Byte, ColorInterp=Undefined
+ Min=10.000 Max=255.000
+ Minimum=10.000, Maximum=255.000, Mean=49.911, StdDev=62.129
+ NoData Value=0
+ Metadata:
+ STATISTICS_MAXIMUM=255
+ STATISTICS_MEAN=49.910941358025
+ STATISTICS_MINIMUM=10
+ STATISTICS_STDDEV=62.129078727248
+ STATISTICS_VALID_PERCENT=100
+Band 3 Block=360x7 Type=Byte, ColorInterp=Undefined
+ Min=10.000 Max=255.000
+ Minimum=10.000, Maximum=255.000, Mean=65.605, StdDev=44.820
+ NoData Value=0
+ Metadata:
+ STATISTICS_MAXIMUM=255
+ STATISTICS_MEAN=65.605154320988
+ STATISTICS_MINIMUM=10
+ STATISTICS_STDDEV=44.819618148349
+ STATISTICS_VALID_PERCENT=100
```

Looking at https://oceania.generic-mapping-tools.org/gmt_data_server.txt, the modification date was 2023-09-29, and seems to be applied from https://github.com/GenericMappingTools/gmtserver-admin/commit/5bcc3d6190cf88f9ccdbb9ed3ca80451e94e61b2.

Just checking if these changes are expected. Personally, I prefer the new 3-band structure since it will simplify things in PyGMT (https://github.com/GenericMappingTools/pygmt/issues/1442#issuecomment-1341079313), but wanted to know if breaking the downloads for GMT 6.4.0 are ok.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.