chainguard-dev / chainguard-dev/apko
tarfs: Entry retains a full tar.Header per file; store a compact record
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 1.7k
- Forks
- 228
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 62
Description
tarfs.New (pkg/apk/expandapk/tarfs/tarfs.go:207) keeps an Entry per tar member holding a full tar.Header copy, including PAXRecords and Xattrs maps and the header's string fields, plus a fs.FileInfo. Callers that cache expanded packages keep these indexes alive for the process lifetime, so every file of every cached package carries the full header.
In a 7-day averaged heap profile of a long-running process, tarfs.New accounts for 0.9 GB in-use, of which archive/tar.parsePAX is 350 MB and readHeader another 260 MB. Those are PAX maps and header strings retained after parsing, not the index itself.
A compact entry (name, size, mode, uid/gid, mod time, typeflag, linkname, offset) with the PAX map dropped once the header is resolved would likely halve the row. FileInfo can be derived on demand from the compact record. Anything that needs a full tar.Header back can re-read it from the offset, which is what the reader path already does for content.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at tarfs.New in pkg/apk/expandapk/tarfs/tarfs.go:207 and trace how Entry, fs.FileInfo, and tar.Header are used by the reader path. Measure the retained header data and identify the existing offset-based header reread behavior. Done means entries retain only the needed compact fields, PAX data is released after resolution, and callers can still obtain file metadata and content.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100