binance / binance/binance-public-data

USD-M metrics: persistent checksum mismatches for STXXUSDT/HBARUSDC and blank/timestamp defects

Open
#498 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
613
PR merge metrics
No merged PRs in 30d

Description

# USD-M metrics: two checksum mismatches and reproducible blank/timestamp defects

Observations: 2026-09-10 UTC.

Please investigate and publish corrected archives/checksums, and clarify the timestamp convention and blank-value semantics. Originals and HTTP receipts are preserved.

Across our frozen 190,399-key listing: 48,221 numeric/period-defective targets, 395 structurally defective targets, and two checksum inconsistencies remained after exact-source refresh. This is an observed collection scope, not a claim about all Binance datasets.

## Checksum inconsistencies

https://data.binance.vision/data/futures/um/daily/metrics/STXXUSDT/STXXUSDT-metrics-2026-07-09.zip

- Downloaded SHA-256: `09fd4c2b9498e7e1c2d470050a7040e5b6d51e33dfbcb969ff9d3b564a60b0ea`
- Published SHA-256: `801944e243b0ebcd47ec1f2837413afedc96441feb325ce71e14b338c260d4c1`
- Last observed: 2026-09-10T19:40:25.796460+00:00
- Response headers: `{"Date": "Thu, 10 Sep 2026 19:40:26 GMT", "Last-Modified": "Wed, 15 Jul 2026 13:17:10 GMT", "ETag": "\"183d1f838496ab89cd18ab6ce2b70dce\"", "X-Cache": "Miss from cloudfront", "Content-Length": "9463"}`

https://data.binance.vision/data/futures/um/daily/metrics/HBARUSDC/HBARUSDC-metrics-2026-07-09.zip

- Downloaded SHA-256: `8d6e3d3efff6e615be11e43c22df3ecda579aeeb45b3da41c88a65662b5e2cc5`
- Published SHA-256: `060025bb8887f2c0456d3333fb3a70001f3dfa5662132b0f895a7f3d3247bd52`
- Last observed: 2026-09-10T19:40:27.535877+00:00
- Response headers: `{"Date": "Thu, 10 Sep 2026 19:40:28 GMT", "Last-Modified": "Wed, 15 Jul 2026 13:17:05 GMT", "ETag": "\"d7f563900c0c2c99b7fd066e02d404c4\"", "X-Cache": "Miss from cloudfront", "Content-Length": "9810"}`

Reproduction: download each URL and its `.CHECKSUM` sibling, then run `shasum -a 256 -c filename.zip.CHECKSUM` in the download directory. Three observations including the initial recovery spanned at least one hour; both pairs remained inconsistent.

## Additional reproducible archive defects

- https://data.binance.vision/data/futures/um/daily/metrics/ZSUSDT/ZSUSDT-metrics-2026-09-08.zip
- Archive SHA-256: `5a163183fdd3ddf63b206d607fc253984911941bde36384f0b3090e183dd5298`; invalid values: {"sum_taker_long_short_vol_ratio|empty": 45, "count_toptrader_long_short_ratio|empty": 7, "sum_toptrader_long_short_ratio|empty": 6, "count_long_short_ratio|empty": 1}; timestamps: []
- https://data.binance.vision/data/futures/um/daily/metrics/ZMUSDT/ZMUSDT-metrics-2026-09-08.zip
- Archive SHA-256: `38b9f8eadca308c7797bf12a32cc6e57adfe77c127b2290fb703a555850f2bb9`; invalid values: {"sum_taker_long_short_vol_ratio|empty": 49, "count_toptrader_long_short_ratio|empty": 1, "sum_toptrader_long_short_ratio|empty": 1}; timestamps: []
- https://data.binance.vision/data/futures/um/daily/metrics/ZKJUSDT/ZKJUSDT-metrics-2026-09-08.zip
- Archive SHA-256: `ca28851560da561a46ddd5c801cf8cf90826b0b60f0afea746146802c76ce756`; invalid values: {"count_toptrader_long_short_ratio|empty": 1, "sum_toptrader_long_short_ratio|empty": 1, "count_long_short_ratio|empty": 1, "sum_taker_long_short_vol_ratio|empty": 1}; timestamps: []
- https://data.binance.vision/data/futures/um/daily/metrics/TURBOUSDT/TURBOUSDT-metrics-2026-07-09.zip
- Archive SHA-256: `ec8dd4c9bf733ee3018d6c54546f676ab69090db33a65a69d8cb7a37692488dd`; invalid values: {}; timestamps: [{"raw": "2026-07-10 00:00:00", "reason": "later_day"}]
- https://data.binance.vision/data/futures/um/daily/metrics/HOMEUSDT/HOMEUSDT-metrics-2026-07-09.zip
- Archive SHA-256: `4c05691403dc6d1a7df6323894295226c42c711721cef26a14e24546f2fb8fe6`; invalid values: {}; timestamps: [{"raw": "2026-07-10 00:00:00", "reason": "later_day"}]
- https://data.binance.vision/data/futures/um/daily/metrics/AAOIUSDT/AAOIUSDT-metrics-2026-07-09.zip
- Archive SHA-256: `172dd7265f1e609eb6cd50bde1da931bc36bbdd95c74a1031ffcbc15289048f9`; invalid values: {}; timestamps: [{"raw": "2026-07-10 00:00:00", "reason": "later_day"}]

A local comparison of all 395 timestamp-defective archives with adjacent-day archives found zero exact same-timestamp duplicates, 14 conflicting comparisons, 789 same-timestamp absences and five unavailable/unverified neighboring comparisons. No timestamps were shifted and no blanks were imputed. Reordering is not a repair of absent values.

Please confirm whether blanks mean unavailable observations or a defined zero-denominator condition, and provide historical corrections where possible.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by downloading the listed metrics archives and their .CHECKSUM siblings, then run the provided shasum command and inspect the reported blank values and timestamps. Done means the affected archives and checksums are corrected where possible, with the timestamp convention and blank-value semantics documented.

Written by the indexing model from the issue text.

Assessment

Domain
data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.