guibranco / guibranco/logstream-server
[FEATURE] Log export and archival
- Dominant language
- PHP
- Stars
- 1
- Forks
- 0
- Avg merge
- 1m
- Merged PRs (30d)
- 4
Description
## Summary
Add a `GET /api/export` endpoint that streams a filtered, time-ranged set of log entries in multiple formats, suitable for archival, compliance, and external analytics.
## Motivation
There is currently no way to extract a bulk set of log entries for offline analysis, feeding into external tools (Splunk, ELK, BigQuery), or producing evidence for compliance audits.
## Proposed endpoint
```
GET /api/export?date_from=2026-01-01&date_to=2026-01-31&app_key=billing-api&format=ndjson
```
Accepts the same filter parameters as `GET /api/logs` plus:
| Param | Options | Default | Description |
|-------|---------|---------|-------------|
| `format` | `ndjson`, `csv`, `json` | `ndjson` | Output format |
| `compress` | `true`, `false` | `false` | gzip compress the response |
## Response
- **Streaming** — flushed progressively, not buffered; large exports do not exhaust memory
- `Content-Disposition: attachment; filename="logs-2026-01-01-to-2026-01-31.ndjson"`
- `Content-Encoding: gzip` when `compress=true`
## Implementation notes
- ReactPHP streaming response with chunked transfer encoding
- File storage: read JSONL files directly (already in ndjson format)
- MariaDB storage: cursor-based pagination to avoid loading all rows at once
- Rate limit: one concurrent export per UI secret
## Acceptance criteria
- [ ] `ndjson`, `csv`, and `json` formats supported
- [ ] Response is streamed, not buffered
- [ ] `compress=true` returns gzip-encoded output
- [ ] All existing `GET /api/logs` filters work with export
- [ ] Works for both storage backends
- [ ] Auth required (`UI_SECRET`)
- [ ] OpenAPI spec updated
- [ ] Documentation in `docs/user-guide/export.md`
Contributor guide
Research direction
Start with the existing GET /api/logs endpoint to identify its filters and authentication behavior, then review the ReactPHP streaming approach and both file and MariaDB storage paths. The export endpoint should support ndjson, csv, and json, compression, progressive delivery, and the stated rate limit without buffering. Finish by updating the OpenAPI spec and docs/user-guide/export.md, then verify the acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mariadb, php
- Domain
- api, backend, databases, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100