binance / binance/binance-public-data
Monthly SPOT kline archives disagree with daily archives and API for 2020-12-21 and 2021-09-29
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 613
- PR merge metrics
- No merged PRs in 30d
Description
```markdown
## Summary
I found Binance SPOT monthly kline archive rows that disagree with Binance SPOT daily kline archive rows for the same symbol, interval, and open time.
For the checked rows, the daily archive matches:
- `GET /api/v3/klines`
- `GET /api/v3/uiKlines`
- `https://www.binance.com/api/v3/uiKlines`
The monthly archive differs. This suggests that some monthly SPOT kline archive files may contain stale or incorrect historical rows.
## Expected behavior
For the same `symbol`, `interval`, and `open_time`, the SPOT monthly kline archive and SPOT daily kline archive should contain the same OHLCV/trade-count data.
## Actual behavior
For several historical candles, the monthly archive disagrees with the daily archive. The daily archive matches current Binance API and chart-side `uiKlines`, while the monthly archive does not.
## Scope observed
Affected symbols observed:
- ADAUSDT
- BNBUSDT
- BTCUSDT
- DOGEUSDT
- ETHUSDT
- LINKUSDT
- SOLUSDT
- TRXUSDT
- USDCUSDT
- XRPUSDT
Affected intervals observed:
- 1m
- 5m
- 15m
- 1h
Affected dates/times observed:
- 2020-12-21, mostly around 13:00-13:49 UTC
- 2021-09-29, 2 observed rows
I checked 398 conflicting rows. All 398 matched daily archive + API/uiKlines and did not match the monthly archive values.
## Reproducible example
### BTCUSDT 1m, open time 2020-12-21 13:19:00 UTC
Monthly archive file:
```text
https://data.binance.vision/data/spot/monthly/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12.zip
```
Daily archive file:
```text
https://data.binance.vision/data/spot/daily/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12-21.zip
```
Observed monthly checksum:
```text
a53a9352fd808d10981f68191138b39ef142e15ca88e03a061309f1699419749 BTCUSDT-1m-2020-12.zip
```
Observed daily checksum:
```text
59543b0862faf5953ce8c4e4efe3892d681711de345ced3ec5cf4dd9185a7154 BTCUSDT-1m-2020-12-21.zip
```
Monthly archive row:
```text
1608556740000,22647.85000000,22661.57000000,22633.30000000,22646.53000000,47.74209100,1608556799999,1081400.91111170,760,16.52699800,374330.37124588,0
```
Daily archive row:
```text
1608556740000,22647.85000000,22666.66000000,22628.62000000,22637.92000000,109.87666500,1608556799999,2488573.22763245,2138,49.06835600,1111469.47191801,0
```
API checks:
```text
https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1
https://api.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1
https://www.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1
```
All three API endpoints return the daily archive value:
```text
[1608556740000,"22647.85000000","22666.66000000","22628.62000000","22637.92000000","109.87666500",1608556799999,"2488573.22763245",2138,"49.06835600","1111469.47191801","0"]
```
## Shell reproduction
```bash
mkdir -p /tmp/binance-kline-check
cd /tmp/binance-kline-check
curl -LO https://data.binance.vision/data/spot/monthly/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12.zip
curl -LO https://data.binance.vision/data/spot/daily/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12-21.zip
sha256sum BTCUSDT-1m-2020-12.zip BTCUSDT-1m-2020-12-21.zip
echo "Monthly row:"
unzip -p BTCUSDT-1m-2020-12.zip | awk -F, '$1==1608556740000 {print; exit}'
echo "Daily row:"
unzip -p BTCUSDT-1m-2020-12-21.zip | awk -F, '$1==1608556740000 {print; exit}'
echo "API kline row:"
curl -s 'https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1'
echo "API uiKlines row:"
curl -s 'https://api.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1'
echo "Website uiKlines row:"
curl -s 'https://www.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1'
```
## Additional examples
### BTCUSDT 5m, open time 2020-12-21 13:15:00 UTC
Monthly-derived row:
```text
close=22646.53000000
volume=496.20080600
num_trades=7440
```
Daily/API/uiKlines row:
```text
close=22637.92000000
volume=558.33538000
num_trades=8818
```
### ADAUSDT 5m, open time 2021-09-29 00:00:00 UTC
Monthly-derived row:
```text
close=2.04000000
volume=1099557.90000000
num_trades=1790
```
Daily/API/uiKlines row:
```text
close=2.04000000
volume=1103591.40000000
num_trades=1791
```
### USDCUSDT 1m, open time 2020-12-21 13:19:00 UTC
Monthly-derived row:
```text
close=1.00000000
volume=3150.53000000
num_trades=8
```
Daily/API/uiKlines row:
```text
close=1.00010000
volume=231669.02000000
num_trades=62
```
## Impact
This appears to affect a small number of historical candles, but the differences are material for backtesting and historical research because OHLC, volume, quote volume, and trade-count fields can change.
Observed in my audit:
- 398 conflicting rows checked.
- 398/398 daily archive values matched API/uiKlines.
- 0/398 monthly archive values matched API/uiKlines when daily differed.
- Max observed OHLC error: about 297.82 bps.
- Max observed close error: about 205.20 bps.
- Volume and trade-count fields were often materially understated in the monthly rows.
This can affect intraday backtests that use:
- high/low stop-loss or take-profit simulation;
- breakout signals;
- volume or quote-volume filters;
- trade-count filters;
- volatility filters;
- scalping logic on 1m, 5m, 15m, or 1h candles.
## Request
Could Binance please confirm whether the affected monthly SPOT kline archive files should be regenerated from the daily/API source of truth?
If the disagreement is expected, could Binance document which source should be treated as canonical when monthly archives disagree with daily archives and API/uiKlines?
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the shell reproduction against the monthly and daily archive URLs, then compare the sample rows with the listed klines and uiKlines API endpoints. Done means confirming the canonical source and either regenerating the affected monthly archives or documenting why the sources differ.
Written by the indexing model from the issue text.
Assessment
- Domain
- api, data
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100