pipe-cd / pipe-cd/pipecd

Reduce stage log Put allocation overhead in filestore persistence

Open
#7,026 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
1.4k
Forks
364
Avg merge
1d 4h
Merged PRs (30d)
84

Description

What happened:

Summary

stageLogFileStore.Put rewrites the entire stage-log fragment on every save by JSON-encoding each LogBlock into a fresh in-memory buffer and then writing the full byte slice to the filestore. The code already carries a TODO for this hot path, and the current implementation scales allocations and latency with total block count on every write.

Affected Area

  • pkg/app/server/stagelogstore/filestore.go
  • stage log persistence in the control plane filestore path

What you expected to happen:

Large or frequently updated stage logs should be persisted with lower peak allocation overhead and an encoding path that does not materialize avoidable intermediate payloads for every block.

How to reproduce it:

  1. Open pkg/app/server/stagelogstore/filestore.go.
  2. Inspect stageLogFileStore.Put around the existing performance TODO.
  3. Note that each call loops over all lf.Blocks, marshals each block individually, appends it to a bytes.Buffer, and finally calls filestore.Put with the complete byte slice.
  4. Add a focused benchmark that grows lf.Blocks and repeatedly calls Put.
  5. Observe allocation count and write cost grow with total block count because the full fragment is rebuilt for every save.

Environment:

  • piped version: repository checkout on 2026-07-13
  • control-plane version: repository checkout on 2026-07-13
  • Others: validated by current code inspection; previous related issue #40 was closed as stale, not fixed

Technical Evidence

  • pkg/app/server/stagelogstore/filestore.go:65-82 rebuilds the payload in memory for every Put
  • The code includes TODO: Reduce the number of marshaling log blocks for improving performance
  • Upstream issue #40 tracked the same performance concern but was auto-closed as stale without an implementation change

Suspected Root Cause

The filestore path is implemented as full-object persistence, and Put currently uses per-block json.Marshal calls plus a whole-fragment buffer build before every write.

Proposed Direction

Start with a benchmark for realistic stage-log sizes, then refactor the write path to stream JSON encoding into the final buffer or otherwise reduce avoidable intermediate allocations while preserving the existing line-delimited on-disk format and EOL completion marker.

Acceptance Criteria

  • A benchmark captures baseline throughput and allocations for larger stage-log fragments
  • The write path measurably reduces allocations or latency compared with the current implementation
  • Existing read compatibility for stored stage logs is preserved
  • No unrelated stage-log behavior changes are introduced

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in pkg/app/server/stagelogstore/filestore.go, reading stageLogFileStore.Put around lines 65-82 and its performance TODO. Add the focused benchmark described in the issue for growing lf.Blocks, then compare allocations and write cost while checking existing read behavior. Done means measurable improvement, preserved line-delimited format and EOL marker, and no unrelated behavior changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.