trailofbits / trailofbits/polyfile
Settle and document the public Python API before 1.0
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 390
- Forks
- 31
- Avg merge
- 7h 52m
- Merged PRs (30d)
- 72
Description
Summary
polyfile/__init__.py exports a public Python API — Matcher, Parser, PARSERS,
register_parser, Submatch, Match, InvalidMatch — but nothing states what a caller may rely
on. There is no py.typed marker, no documented contract, and no deprecation policy. Before
declaring 1.0, decide what that surface is and write it down.
This is an entry criterion for the v1.0.0 milestone, not a defect.
Why now
The v0.6.0 cycle changed that surface repeatedly, which is the evidence that it is not yet settled:
try_all_offsetswas removed fromMatcher.__init__andAnalyzer.__init__(#3465). It had been
accepted and ignored since it was introduced.Matcher.matchchanged behavior substantially for an already-open stream (#3463): it previously
returned 1 match where a path returned 51.FileStream.__init__'slengthparameter had two contradictory meanings depending on which
branch ran (#3561). One was chosen and documented; nothing had recorded which was intended.- Both "Extending PolyFile" examples in
CLAUDE.mdwere non-functional, and theregister_parser
example documented the wrong shape entirely — it decorated a class, whereregister_parserwraps
its argument in a function wrapper (#3467). Anyone following the guide got
TypeError: MyParser() takes no arguments.
A 1.0 version number promises that breaking changes wait for 2.0. That promise is only meaningful if
the surface it covers is defined.
What this needs
- Decide what is public.
polyfile/__init__.py's exports are the current de facto answer, but
Analyzer,MagicMatcher,MagicTest,FileStreamandMatchContextare all reachable and
used by the documented extension points. Either bring them in deliberately or state that they are
internal. - Write the contract.
docs/extending_polyfile.mdexists and should carry it: what a custom
matcher and a custom parser must implement, what they may rely on, and what the library
guarantees across minor versions. The corrected examples from #3467 are the starting point. - Add a
py.typedmarker so type checkers see the annotations the package already carries, and
decide whether the annotated signatures are part of the contract. - Settle #3532 first, or state that it is out. Implementing all-offsets scanning would change
Matcher's behavior and possibly its constructor. Deciding that after 1.0 means either breaking
the promise or shipping it awkwardly. - State a deprecation policy. One minor release with a warning is the usual shape; any policy
is better than none.
Related
- #3532 — all-offsets scanning, which would change
Matcher - #3467 — the documentation defects that revealed the extension API was undocumented in practice
- #3463, #3465, #3561 — the v0.6.0 changes to this surface
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the exports in polyfile/init.py and read docs/extending_polyfile.md alongside the corrected examples from #3467. Review the public extension points and settle the relationship with #3532 before documenting the supported API, version guarantees, deprecation policy, and adding the py.typed marker. Done means the 1.0 contract and its scope are explicit in the documentation and package metadata.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100