whiteducksoftware / whiteducksoftware/flock
[1.0] Minimize and redact trace data before export
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 120
- Forks
- 14
- Avg merge
- 19h 32m
- Merged PRs (30d)
- 8
Description
Trace serialization currently carries strings and dictionary values into exports without a general capture-time limit or redaction boundary. Protecting the dashboard alone cannot protect data already written to traces.
Scope
- Define and apply a bounded diagnostic representation before trace/log export: retain useful operational fields and exclude raw protected payloads by default.
- Redact known credentials throughout nested supported values, reuse the lm_kwargs fix, and prevent diagnostic serialization errors from falling back to raw values.
- Keep explicit trusted diagnostic access separate from generally exported observability data.
Acceptance criteria
- Synthetic secrets and private payload canaries are absent from exported spans/log records, including nested containers and exception paths.
- Record-size limits are observable; useful error codes, run identifiers and timing remain available.
- Tests cover capture/export output itself rather than only a filtered UI response.
Boundaries
No blanket claim that arbitrary strings can always be recognized as secrets; unapproved raw content must be omitted rather than relying only on pattern matching.
References
- Parent topic: #280.
- Implementation dependencies: #383, #428.
- src/flock/logging/trace_and_logged.py
- src/flock/engines/dspy_engine.py
- src/flock/api/collector.py
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading src/flock/logging/trace_and_logged.py, then trace the related paths in src/flock/engines/dspy_engine.py and src/flock/api/collector.py. Done means exported spans and log records omit synthetic secrets and private payloads, including nested and exception data, while retaining useful diagnostics and exposing record-size limits; tests must verify capture/export output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- observability, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100