DE<>GIS: COLP Packaging

Automate the transformations that GIS applies to COLP, as outlined here

Macro Steps:

• Pull from publishing folder on Digital Ocean
• Apply transformations
• Validate Data
• Output to new bucket

Individual Transformations to call out:

• (Item 4) "XLSX: Convert csv to xlsx. IMPORTANT:This must be done carefully to avoid corrupting data that must retain leading zeros" (AR note: I assume this will happen automatically when we automate this. But maybe not)
• (Item 6) "Confirm/Review data schema is the same as previous version
  No missing geographic values (i.e. Borough, CD, etc)
  No blank values in feature class, should be “null” instead "
• (Item 8) Generate README pdf from sharepoint. Should this reside in (perhaps) markdown in our github repo?
• (Item 10) "update GDB and SHP metadata, Import metadata from previous version to GDB, Update citation publication date to last day of month (i.e. 6/30/2022)

Start by reviewing the linked GIS Open Data Update Cycle Instructions and the unchecked macro steps: pull from the publishing folder, apply transformations, and output to a new bucket. Then review Item 10 for GDB and SHP metadata requirements. Done means the listed transformations are automated while preserving the validated schema and geographic values.