apache / apache/trafficserver

Consolidate and modernize ATS logging: unify log files, timestamps, and naming

Open
#12,944 0 comments 1 reaction 1 assignee Claimed by @bryancall View on GitHub
Logging New Feature
Dominant language
C++
Stars
2k
Forks
874
Avg merge
6d 15h
Merged PRs (30d)
46

Description

## Problem

ATS currently has **four separate log files** with inconsistent timestamp formats and overlapping purposes, while comparable proxy servers (Nginx, Envoy, HAProxy) use only one or two.

### Current log files

| Log File | Written By | Timestamp Format | Purpose |
|----------|-----------|-----------------|---------|
| `squid.log` | Logging subsystem (`LogObject`) | epoch.ms (e.g. `1741210395.123`) | Per-transaction access log |
| `error.log` | `Log::error()` via `LogObject::va_log()` | `%Y%m%d.%Hh%Mm%Ss` (e.g. `20260305.21h33m15s`) | Connection failures, DNS errors |
| `diags.log` | `Diags::print_va()` | `[DiagTimestamp]` with thread name + level | Debug messages, warnings, notes, errors |
| `traffic.out` | Same `Diags` system (stdout/stderr) | Same as `diags.log` | Duplicate of diags.log to stdout |

### Comparison with other proxies

| Proxy | Log Files |
|-------|-----------|
| Nginx | 2 — `access.log` + `error.log` |
| Envoy | 1 — access log (errors to stderr) |
| HAProxy | 2 — access log + admin log |
| **ATS** | **4** — `squid.log` + `error.log` + `diags.log` + `traffic.out` |

## Issues

1. **Three different timestamp formats** across log files make cross-log correlation difficult
2. **`diags.log` and `traffic.out`** are the same Diags system writing to two destinations — redundant
3. **`error.log` and `diags.log`** both contain error-level messages but use completely different code paths (`Log::error()` vs `Diags` Error level) with different formats
4. **No SM-aware error macros** — `SMDbg`/`TxnDbg` prefix debug messages with `[SM_ID]` but there's no `SMError`/`TxnError` equivalent for `Log::error()`, `Log::warning()`, or `Log::note()`. This makes it impossible to correlate error.log entries back to a specific transaction without manually adding the SM ID (as seen in PR #12932)
5. **"Squid log" is a legacy name** — the default format hasn't been actual Squid format in a long time, but field names (`client_req_timestamp_squid`, `cqtq`) and config still reference "squid"

## Proposed Changes

### 1. Add SM-aware error/warning/note macros
```cpp
#define SMError(fmt, ...) Log::error("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
#define SMWarning(fmt, ...) Log::warning("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
#define SMNote(fmt, ...) Log::note("[%" PRId64 "] " fmt, sm_id, ##__VA_ARGS__)
// And TxnError/TxnWarning/TxnNote for HttpTransact context
```

### 2. Standardize timestamp format
Pick one format and use it everywhere, or at minimum make them correlatable (e.g. all ISO 8601).

### 3. Rename "squid log" to "access log"
Update config names, field names, documentation, and code references.

### 4. Consolidate log files
Long-term goal: merge `error.log`, `diags.log`, and `traffic.out` into a single diagnostic/error log, similar to how Nginx has one `error.log` for everything that isn't per-transaction access logging.

## Notes

- Items 1-3 can be done incrementally without breaking changes
- Item 4 (log consolidation) is a larger architectural change that should be designed carefully
- The SM-aware macros (item 1) are the most immediately useful and could unblock PR #12932's approach of adding `sm_id` to error log messages in a standardized way

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.