Implement NDJSON Streaming / Export Endpoints

Standard list endpoints (GET /datasets) use simple limit/offset pagination which guarantees Out-Of-Memory (OOM) crashes on the server if a client attempts to pull the entire catalog (?limit=1000000). To fix this, we must enforce bounds on active endpoints and build dedicated streaming endpoints for scientific bulk-export workflows.

Acceptance Criteria:

• Enforce a strict max_limit (e.g., 1000) on all standard list endpoints to protect the memory pool.
• Create dedicated GET /export/{resource} endpoints.
• Implement SQLAlchemy's yield_per() cursors coupled with FastAPI's StreamingResponse to stream the data as NDJSON (JSON-Lines), maintaining a low, constant memory footprint on the server regardless of export size.

Governing ADR(s): ADR 0020

Migrated from the internal tracker, where it was #7, opened 2026-02-25.
In progress as #14.

Start with ADR 0020 and the existing standard list endpoints such as GET /datasets. Review how SQLAlchemy yield_per() can feed FastAPI StreamingResponse as NDJSON, while enforcing max_limit on list endpoints. Done means bounded standard lists and dedicated GET /export/{resource} endpoints with constant-memory streaming; note that the issue says this work is in progress as #14.