xiph / xiph/opus

Please provide an option to not depend on downloading model data

Open
#337 10 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
3.3k
Forks
808
PR merge metrics
No merged PRs in 30d

Description

We build opus from git source and we'd like to not switch to tarballs or download the model tarball. Given the recent xz situation downloading tarballs with no trace on git is complicated and seems like a security issue.

Having an option to stop depending on that seems like it'd be great.

With the current meson configuration:

-Dfloat-api=true 
-Dasm=disabled 
-Dhardening=true 
-Dcustom-modes=true 
-Denable-deep-plc=true 
-Denable-dred=true 
-Denable-osce=true 
-Ddocs=enabled 
-Dextra-programs=disabled 
-Dtests=disabled 
-Dintrinsics=enabled 
-Drtcd=enabled

meson seems to fail on meson.build:636:24: ERROR: File dnn/fargan_data.h does not exist. and is not even aware of the download model script living in autogen.

Whereas the download model script is passing an unknown commit https://github.com/xiph/opus/blob/ab4e83598e7fc8b2ce82dc633a0fc0c452b629aa/autogen.sh#L12C24-L12C31

What is the source of that commit? Can this model data be found on git somewhere? Can this be implemented as a git submodule?

Looking through the git history seems at some point there was a submodule, which was reverted in favour of downloading the model tarball.

Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with meson.build around line 636 and autogen.sh around line 12, then inspect the git history for the reverted model-data submodule. Reproduce the failure with the listed Meson options and trace how fargan_data.h is expected to appear. Done means a source build can use a traceable model-data source without downloading an untracked tarball.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, git
Domain
build-system, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.