influxdata / influxdata/datafusion-udf-wasm
Python guest Justfile: validate checksums for already-present case
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 20
- Forks
- 3
- Avg merge
- 6h 20m
- Merged PRs (30d)
- 24
Description
# What
The Python guest Justfile has an optimization that avoids downloading the CPython WASM build and the WASI SDK again if they are already downloaded:
https://github.com/influxdata/datafusion-udf-wasm/blob/a5b9a1d2c2a887aae99394da36e0462f13f08081/guests/python/Justfile#L33-L37
https://github.com/influxdata/datafusion-udf-wasm/blob/a5b9a1d2c2a887aae99394da36e0462f13f08081/guests/python/Justfile#L90-L94
That should not only check presence but also the checksum of the already-downloaded files and should re-download if the checksums mismatch.
# Why
Whenever we upgrade the CPython version or the WASI SDK, developers will locally still have an outdated copy and without `just clean` they will not use the new versions. This isn't an issue for our CI though.
Note that this is NOT a security issue since the checksum was validated when the data was originally downloaded, it's just that it doesn't automatically upgrade the version of a local development setup.
# How
In the if-branch, do something similar to these checks
https://github.com/influxdata/datafusion-udf-wasm/blob/a5b9a1d2c2a887aae99394da36e0462f13f08081/guests/python/Justfile#L59-L60
https://github.com/influxdata/datafusion-udf-wasm/blob/a5b9a1d2c2a887aae99394da36e0462f13f08081/guests/python/Justfile#L106
and if they fail, wipe the downloaded data and re-download.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in guests/python/Justfile, especially the already-present branches around lines 33–37 and 90–94. Compare them with the checksum checks around lines 59–60 and 106, then verify that a checksum mismatch removes the downloaded data and triggers a fresh download; the existing-download case should also retain the current optimization.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, wasm
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 55/100