microsoft / microsoft/onnxruntime

Copyright/license data, especially for models

Open
#8,963 6 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 8h
Merged PRs (30d)
179

Description

Background

I'm working on trying to package onnxruntime for Debian, part of which involves making a fairly in-depth inventory of the copyright and license of files in the repo. In a perfect world, all open source projects would follow something like https://reuse.software , which puts unambiguous copyright and license (machine readable) in every file, or adjacent in a .license file for those files that cannot have a license header easily added.

Most of the source code I can figure out, but where I'm particularly having trouble is with the numerous .onnx model files in the repo. (and any other binary files) They aren't plain text, of course, so there's no plaintext header, there's limited license info near them (e.g. there's winml/test/collateral/models/LICENSE.md describing two licenses but it doesn't say which files those apply to), and there are files that look like they may have come from other projects - the models that are more than trivial tests according to their filename.

My current workaround

Right now, I'm just excluding everything that doesn't appear to be a trivial test or one of the two licenses in the mentioned file above, as "potentially not DFSG-free", but I'm not even confident that things with a trivial-sounding filename are in fact trivial/artificial examples constructed as part of this project and licensed along with it, vs which things are third-party models that just exhibit some important or testable behavior.

My alternative at the moment is to manually review the output of strings run on each model to make sure I only see generic-looking things, and check the git history of each file additionally, which doesn't necessarily give me a lot of data either. I'd like to be able to actually run the tests in the Debian package building process, but with so many mystery files I may just end up excluding all .onnx files from the repacked source to be sure to avoid license problems.

Describe the solution you'd like

It would be really great if someone more familiar with the history of these files, etc. could go through and add .license files next to each .onnx file (as in the REUSE standard) with copyright holder and SPDX-License-Identifier. It would make using this library elsewhere much more feasible. These can be made pretty quickly by hand or by using the reuse project's command line tool.

I'm pretty sure additional information could be added to these files for models from external sources, as long as they start with something parsable as copyright line(s) and SPDX-License-Identifier line(s).

(For an example of one I'm not equipped to figure out: it looks like the original "fast neural style" models might be "free for research and non-commercial use" which isn't DFSG free or OSI open source. I am not sure which FNS-related models are just conversions of these original non-free models, and which are independent re-implementations based on the paper and thus subject to some other license. I also see some things mentioning Bert, whose original models are Apache-2.0, etc.)

REUSE-compliance would be even better, but there's more work to that, and I can extract or infer license and copyright for most source code files just fine, it's the models I struggle with.

Happy to help or offer advice, I just don't have enough background in ML or this project to answer some of the questions around the origins of these files.

System information

  • ONNX Runtime version (you are using): master branch

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with winml/test/collateral/models/LICENSE.md and inventory the repository's .onnx and other binary model files. Use the REUSE tool, strings output, and git history to trace each model's origin and determine applicable copyright and SPDX license data. Done means each in-scope model has an adjacent parsable .license file, with unresolved provenance documented rather than guessed.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.