Consolidate and modernize ATS logging: unify log files, timestamps, and naming
- Dominant language
- C++
- Stars
- 2k
- Forks
- 874
- Avg merge
- 6d 15h
- Merged PRs (30d)
- 46
Description
## Problem
ATS currently has **four separate log files** with inconsistent timestamp formats and overlapping purposes, while comparable proxy servers (Nginx, Envoy, HAProxy) use only one or two.
### Current log files
| Log File | Written By | Timestamp Format | Purpose |
|----------|-----------|-----------------|---------|
| `squid.log` | Logging subsystem (`LogObject`) | epoch.ms (e.g. `1741210395.123`) | Per-transaction access log |
| `error.log` | `Log::error()` via `LogObject::va_log()` | `%Y%m%d.%Hh%Mm%Ss` (e.g. `20260305.21h33m15s`) | Connection failures, DNS errors |
| `diags.log` | `Diags::print_va()` | `[DiagTimestamp]` with thread name + level | Debug messages, warnings, notes, errors |
| `traffic.out` | Same `Diags` system (stdout/stderr) | Same as `diags.log` | Duplicate of diags.log to stdout |
### Comparison with other proxies
| Proxy | Log Files |
|-------|-----------|
| Nginx | 2 — `access.log` + `error.log` |
| Envoy | 1 — access log (errors to stderr) |
| HAProxy | 2 — access log + admin log |
| **ATS** | **4** — `squid.log` + `error.log` + `diags.log` + `traffic.out` |
## Issues
1. **Three different timestamp formats** across log files make cross-log correlation difficult
2. **`diags.log` and `traffic.out`** are the same Diags system writing to two destinations — redundant
3. **`error.log` and `diags.log`** both contain error-level messages but use completely different code paths (`Log::error()` vs `Diags` Error level) with different formats
4. **No SM-aware error macros** — `SMDbg`/`TxnDbg` prefix debug messages with `[SM_ID]` but there's no `SMError`/`TxnError` equivalent for `Log::error()`, `Log::warning()`, or `Log::note()`. This makes it impossible to correlate error.log entries back to a specific transaction without manually adding the SM ID (as seen in PR #12932)
5. **"Squid log" is a legacy name** — the default format hasn't been actual Squid format in a long time, but field names (`client_req_timestamp_squid`, `cqtq`) and config still reference "squid"
## Proposed Changes
### 1. Add SM-aware error/warning/note macros
```cpp
#define SMError(fmt, ...) Log::error("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
#define SMWarning(fmt, ...) Log::warning("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
#define SMNote(fmt, ...) Log::note("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
// And TxnError/TxnWarning/TxnNote for HttpTransact context
```
### 2. Standardize timestamp format
Pick one format and use it everywhere, or at minimum make them correlatable (e.g. all ISO 8601).
### 3. Rename "squid log" to "access log"
Update config names, field names, documentation, and code references.
### 4. Consolidate log files
Long-term goal: merge `error.log`, `diags.log`, and `traffic.out` into a single diagnostic/error log, similar to how Nginx has one `error.log` for everything that isn't per-transaction access logging.
## Notes
- Items 1-3 can be done incrementally without breaking changes
- Item 4 (log consolidation) is a larger architectural change that should be designed carefully
- The SM-aware macros (item 1) are the most immediately useful and could unblock PR #12932's approach of adding `sm_id` to error log messages in a standardized way
Contributor guide
Assessment
This issue has not been assessed yet.