astral-sh / astral-sh/tar-codec

Expose byte-preserving member paths and link targets when decoding

Open
#118 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
9
Forks
3
Avg merge
9m
Merged PRs (30d)
3

Description

## Summary

The high-level decoder currently converts effective member paths and link targets with `std::str::from_utf8` and returns `DecodeError::InvalidUtf8` when they are not UTF-8. This prevents consumers from decoding otherwise valid archives whose names or symlink targets are arbitrary byte sequences.

Unix filesystem names are byte strings (excluding NUL and `/`), and USTAR/GNU header fields can contain non-UTF-8 bytes. Some archive consumers need to validate and preserve those bytes without interpreting them as text.

## Suggested API direction

Please consider exposing effective paths and link targets in a byte-preserving form, either:

- by changing or extending `MemberMetadata` and link targets to use a byte-string type; or
- by providing a lower-level public member API that retains the effective raw bytes while keeping the existing UTF-8 format-neutral API.

The framing layer already computes effective paths and link targets as byte slices, so ideally the high-level projection would not make UTF-8 validation unavoidable. Extraction policy could continue to impose its own platform-appropriate name restrictions.

## Impact

Without such an API, applications that otherwise use byte-oriented filesystem models must reject non-UTF-8 archives or bypass `tar-codec`’s high-level decoder.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the high-level decoder's conversion of effective member paths and link targets through MemberMetadata and DecodeError::InvalidUtf8, then review how the framing layer exposes the underlying byte slices. The design is complete when consumers can preserve arbitrary valid bytes through a public API without making the existing UTF-8 format-neutral API or extraction policy unsafe.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.