trailofbits / trailofbits/fickling

Polyglot module improvements

Open
#93 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
670
Forks
77
PR merge metrics
No merged PRs in 30d

Description

To better account for different parser implementations and to make identification more robust, we could use PolyFile and call it in Fickling. In addition, we should ensure the module directly corresponds to the Netron consensus as the present file format descriptions can be more granular. Specifically, the version numbers in the file name are partially dynamically generated in Netron; Fickling’s file format naming convention chooses the minimum possible file format version instead (for instance, a TorchScript v1.6 file in Netron may be deemed a TorchScript v1.4 by Fickling). Any issues with the PyTorch file format versioning system should be taken into consideration.

Current list:

PyTorch v0.1.1: Tar file with sys_info, pickle, storages, and tensors
PyTorch v0.1.10: Stacked pickle files
TorchScript v1.0: ZIP file with model.json
TorchScript v1.1: ZIP file with model.json and attributes.pkl (1 pickle file)
TorchScript v1.3: ZIP file with data.pkl and constants.pkl (2 pickle files)
TorchScript v1.4: ZIP file with data.pkl, constants.pkl, and version set at 2 or higher (2 pickle files)
PyTorch v1.3: ZIP file containing data.pkl (1 pickle file)
PyTorch model archive format [ZIP]: ZIP file that includes Python code files and pickle files

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing Fickling’s current file-format list with PolyFile’s parser coverage and Netron’s format and version logic, then read the linked PyTorch versioning issue. The issue names no repository files or tests; done means a more robust identification module whose formats and version labels correspond to the agreed Netron consensus.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.