guibranco / guibranco/logstream-server
[FEATURE] Log forwarding to external systems (Loki, Seq, webhook)
- Dominant language
- PHP
- Stars
- 1
- Forks
- 0
- Avg merge
- 1m
- Merged PRs (30d)
- 4
Description
## Summary
Forward log entries to external systems as they arrive, enabling logstream-server to act as an aggregation and fan-out layer in front of existing observability tooling.
## Motivation
Many teams already have Grafana/Loki, Seq, or custom log pipelines. Rather than replacing them, logstream-server should be able to forward entries so both can coexist, or so logstream-server provides the lightweight UDP-capable ingest layer while a heavier system handles long-term storage.
## Proposed forwarders
### Loki
```json
{
"type": "loki",
"url": "https://loki.example.com/loki/api/v1/push",
"labels": { "job": "logstream", "env": "production" },
"app_key": "billing-api"
}
```
### Seq
```json
{
"type": "seq",
"url": "https://seq.example.com",
"api_key": "your-seq-api-key"
}
```
### Generic webhook
```json
{
"type": "webhook",
"url": "https://example.com/logs",
"headers": { "X-My-Key": "secret" },
"method": "POST",
"format": "json_array"
}
```
## Configuration
Loaded from `FORWARD_CONFIG=./forwarding.json`. Forwarders can be scoped by `app_key`, `level`, or applied globally.
## Implementation notes
- Forwarding is **async and non-blocking** — entries queued and forwarded in batches on a timer
- Failed forwards retried up to 3 times with exponential backoff
- Local storage always succeeds regardless of forward outcome
- `GET /api/forwarders/status` shows queue depth and recent error rate per forwarder
## Acceptance criteria
- [ ] Loki forwarder sends entries in the Loki push API format
- [ ] Seq forwarder sends entries as CLEF JSON
- [ ] Generic webhook forwarder sends configurable JSON payload
- [ ] Forwarding is async — does not block the event loop
- [ ] Failed forwards retried with exponential backoff (max 3 attempts)
- [ ] Local storage always succeeds regardless of forward outcome
- [ ] `GET /api/forwarders/status` returns per-forwarder queue depth and error rate
- [ ] Unit tests for each forwarder's serialisation logic
- [ ] Documentation in `docs/user-guide/forwarding.md`
Contributor guide
Research direction
Start by tracing the existing ingestion and local-storage flow, then inspect how FORWARD_CONFIG is loaded; the issue names no source files or tests. Use the listed Loki, Seq, and webhook payloads as serialization targets. Done means asynchronous forwarding, retries, status reporting, unit tests, and docs/user-guide/forwarding.md satisfy every acceptance criterion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- api, backend, documentation, observability-sre
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100