API registry data is not reproducible: version the api_definition endpoint or ship meta_data.json in releases
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 17.3k
- Forks
- 1.4k
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 105
Description
Summary
scripts/fetch_meta.py fetches meta_data.json from a live endpoint on every build, and internal/registry/remote.go refreshes it at runtime. The endpoint serves continuously changing content while reporting a constant data.version of \"1.0.0\", so:
- builds of the same git tag at different dates embed different registries (official release binaries are not reproducible),
- the runtime
data_versionconditional never detects changes, - downstream packagers that pin the endpoint by hash (e.g. nixpkgs, see NixOS/nixpkgs#554000) end up with pins that die the moment the content drifts, because the previously served bytes are gone forever.
Evidence
All of the following is reproducible against the live endpoint:
-
Content drifts while
data.versionstays constant. Wayback Machine snapshots ofGET https://open.feishu.cn/api/tools/open/api_definition?protocol=meta:snapshot services resources methods normalized sha256 (of jq -S \".data\")2026-06-26 14 77 237 4d772886…2026-07-01 15 78 239 a00d5ba6…2026-08-11 15 80 245 2f45412c…today 15 81 247 244b769f…In every snapshot
data.versionis the literal\"1.0.0\". -
data_versionis an exact-match conditional against that constant, not a version selector.&data_version=1.0.0returns a 38-byte empty "not modified" body;&data_version=0.0.1,1.0.1,2.0.0,9999.0.0,2026-07-01,abc, … all return the current full document withversion: \"1.0.0\".
There is no way to request a historical version.
-
client_versionis ignored: thev1.0.58andv1.0.88URLs return byte-identical data today. -
No HTTP cache anchors: the response has no
ETag,Last-Modified, orCache-Control.
Consequences
- Runtime refresh is effectively dead for binaries with embedded data.
fetchRemoteMerged(embeddedVersion)sendsdata_version=1.0.0(the embedded registry version), the server compares it against its own constant\"1.0.0\", finds an exact match, and answers "not modified". Embedded baselines therefore never receive updates, even though the content has changed several times since. - Same-tag builds are not reproducible.
build.shrunsfetch_meta.pybeforego build, so rebuilding a tag at a different date embeds a different registry than the official binaries for that tag. - Downstream hash pins die silently. In nixpkgs the registry is a fixed-output derivation pinned by sha256. When the endpoint drifted, the bytes behind the old hash became unfetchable forever: the 1.0.58 pin can no longer be fetched from the endpoint or from
cache.nixos.org, so that nixpkgs revision cannot build the CLI from source anymore.
Workaround currently used downstream
nixpkgs now extracts the registry that upstream already embeds in each release binary (go:embed via loader_embedded.go) from the official linux-amd64 release tarball, which is immutable per version and checksummed. This was verified byte-exact against the official 1.0.88 and 1.0.58 binaries (the two releases even use different JSON key orders). It works, but it is a workaround for missing versioned distribution of this data.
Requests
Any one of these would fix reproducibility; the first two together would be ideal:
- Bump
data.versionwhenever the registry content changes, so the existingdata_versionconditional actually works (this would also revive the runtime refresh protocol). - Support fetching a specific
data_version(historical snapshot), so builds can pin the data version alongside the CLI version. - Ship
meta_data.jsonin the git tag / release assets. The file is already embedded in every release binary, so publishing it alongside the binaries is zero extra work and gives every downstream packager an immutable, versioned source.
Happy to provide any further traces or test results if useful.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/fetch_meta.py, internal/registry/remote.go, build.sh, and loader_embedded.go to trace how registry data is fetched, embedded, refreshed, and released. Reproduce the documented endpoint behavior and compare the three proposed distribution or versioning approaches. Done means release builds use an immutable, reproducible registry source and the runtime refresh behavior or release asset distribution matches the chosen approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, python
- Domain
- build-system, release
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100