NYCPlanning / NYCPlanning/data-engineering

URLs for `uscourts_courts` and `usdot_ports` are broken

Open
#1,326 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

db-facilities ingest/library
Dominant language
Python
Stars
43
Forks
3
Avg merge
23h 3m
Merged PRs (30d)
44

Description

uscourts_courts.py tries to get data from a US Courts API. but per #1320, the URL can no longer be found

this PACER API may be a good alternative: https://pacer.uscourts.gov/file-case/court-cmecf-lookup/data.json

or this Court Listener API


usdot_ports is broken the same way. Its ArcGIS layer now returns {"error":{"code":499,"message":"Token Required"}}, so it isn't public any more. Last good archive is 20250722. Needs a replacement NTAD source; geo.dot.gov didn't have an obvious one.

Current state of the uscourts block: www.uscourts.gov/ serves fine, but /federal-court-finder* returns 403 from AkamaiGHost. A browser User-Agent and full browser headers don't clear it, so it's a path rule, not a bot filter. Same shape as the USPS block we solved with session-header replay. Last good archive is uscourts_nyc_courts/20260208.

Both leave facdb on stale inputs: it currently reads uscourts_courts at 20240813.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dcpy/library/script/uscourts_courts.py and the usdot_ports script, then compare their current sources with the last good archives noted in the issue. Evaluate the PACER or Court Listener options for courts and identify a replacement NTAD source for ports. Done means both scripts retrieve current data and facdb no longer reads the stale 20240813 courts input.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.