getModuleFileEntryAsBytes / .bin byte-exact bundled data does not work on web (silent wrong data, not an error)
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 16.4k
- Forks
- 537
- Avg merge
- 22d 5h
- Merged PRs (30d)
- 1
Description
Summary
The byte-exact bundled-data mechanism added for getModuleFileEntryAsBytes/.bin srcs files (docs: docs/docs/advanced-images.md#byte-exact-raw-bitmap-data, example: apps/helloworld/src/valdi/hello_world/src/ByteExactImageExample.tsx) does not work on the web target, and fails silently rather than erroring.
The docs state:
getModuleFileEntryAsBytesreturns the file's exact bytes, on every platform.
This is not true for web.
Root cause
Two separate gaps, at two different layers:
-
Compiler:
.binsrcs files are never emitted into the web build output at all. I confirmed this empirically — after building a consuming app's web target, its.ayab_web_cacheoutput directory (the Valdi-compiled JS/asset tree consumed by webpack) contains zero.binfiles anywhere, even though the same module builds and bundles correctly for iOS/Android/macOS. Looking atbzl/valdi/valdi_compiled.bzl,_get_srcs_js_pathsonly maps.ts/.tsx/.jssources to.jsweb outputs — any other extension is silently skipped and never declared as a web output. The.valdimodulearchive packing that makes.binretrievable at runtime (_will_generate_valdimodule,valdi/src/valdi/runtime/Resources/Bundle.cpp) is only wired for iOS/Android/native/standalone targets; there is no web equivalent. (There is a working precedent for "verbatim-copy a raw file into a per-platform web output path" — the strings-JSON pipeline, e.g._get_web_string_resource_pathsin the same file — but that copy is performed by the Swiftlocal_valdi_compilerbinary itself as part of its declaredctx.actions.runoutputs, not by a Bazel/shell step, so extending it to.binrequires compiler-side (Swift) changes, not just Bazel/TS.) -
Runtime:
getModuleEntryinsrc/valdi_modules/src/valdi/web_renderer/src/ValdiWebRuntime.ts(~line 293) is a hardcoded stub:// Stubbed — was using jsonContext (removed). Localized strings now use // _strings_preload.js generated by collapse_web_paths instead. getModuleEntry(module: string, pathStr: string, asString: boolean) { return '{}'; }It always returns the literal string
'{}'regardless of arguments, and never throws.getModuleFileEntryAsBytescasts this toUint8Array, so callers get back the 2 bytes{/}with no error, no warning — code that then tries to decode those bytes as image data (e.g.decodeBitmap) doesn't throw either; it just silently produces a bogus/degenerate result (in our repro, a 1x1 bitmap instead of the real 60x10 source image). This is the more actionable half of the bug: even without the compiler-side fix,getModuleEntryshould fail loudly on web instead of returning stub data that downstream code can't distinguish from success.
Repro
- Add a module with a
.binsrc per theadvanced-images.mdbyte-exact pattern (srcs = glob([..., "src/**/*.bin"]),inline_assets = True). - Call
getModuleFileEntryAsBytes(module, path)for that file from a component that also renders on web (e.g. viawebpack serve/npm startin a consuming app'sweb/dir). - Native builds (iOS/Android/macOS-standalone
bazel test) return the correct bytes. The web build returnsUint8Arraybytes for the string'{}'instead, and no error is surfaced anywhere (console, exception, or otherwise).
Suggested fix
- Short term (low risk, high value): make
getModuleEntry's web stub throw instead of returning'{}', so this fails loudly instead of masquerading as success. This alone would have made the underlying bug in our app immediately obvious instead of costing significant debugging time to trace through decoded-bitmap-looks-wrong -> traced back through three layers to this stub. - Longer term: either (a) extend the compiler to emit
.binsrcs verbatim into a web output path (mirroring the strings-JSON precedent) plus a webpack-visible registry (mirroring the existing_image_registry.js/_strings_preload.jsgeneration inbzl/valdi/valdi_collapse_web_paths.bzl), giving.binreal web support; or (b) if web support isn't planned, updateadvanced-images.mdto explicitly scope the "on every platform" claim to native platforms only, so consumers don't reach for this API expecting web parity.
Environment
- Found via a downstream consumer app pinned to Valdi SHA
919f23c8ebdd3582b58821a9ff252b93ead5eb75(the commit that introduced the byte-exact.binfeature /ByteExactImageExample.tsx). - Repro path: web dev server via
webpack serve(webpack 5),bazel test(native JS engine / valdi_standalone) does not reproduce this — it only shows up in an actual browser/webpack build.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in src/valdi_modules/src/valdi/web_renderer/src/ValdiWebRuntime.ts at getModuleEntry and reproduce with the consuming app's web target via webpack serve. Then inspect bzl/valdi/valdi_compiled.bzl and the Swift local_valdi_compiler path for .bin handling. Done means the chosen web-support scope is explicit and the ByteExactImageExample.tsx flow no longer silently receives bogus data.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift, typescript, webpack
- Domain
- build-system, tooling, web-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100