binance / binance/binance-public-data

Monthly SPOT kline archives disagree with daily archives and API for 2020-12-21 and 2021-09-29

Open
#475 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
613
PR merge metrics
No merged PRs in 30d

Description

```markdown
## Summary

I found Binance SPOT monthly kline archive rows that disagree with Binance SPOT daily kline archive rows for the same symbol, interval, and open time.

For the checked rows, the daily archive matches:

- `GET /api/v3/klines`
- `GET /api/v3/uiKlines`
- `https://www.binance.com/api/v3/uiKlines`

The monthly archive differs. This suggests that some monthly SPOT kline archive files may contain stale or incorrect historical rows.

## Expected behavior

For the same `symbol`, `interval`, and `open_time`, the SPOT monthly kline archive and SPOT daily kline archive should contain the same OHLCV/trade-count data.

## Actual behavior

For several historical candles, the monthly archive disagrees with the daily archive. The daily archive matches current Binance API and chart-side `uiKlines`, while the monthly archive does not.

## Scope observed

Affected symbols observed:

- ADAUSDT
- BNBUSDT
- BTCUSDT
- DOGEUSDT
- ETHUSDT
- LINKUSDT
- SOLUSDT
- TRXUSDT
- USDCUSDT
- XRPUSDT

Affected intervals observed:

- 1m
- 5m
- 15m
- 1h

Affected dates/times observed:

- 2020-12-21, mostly around 13:00-13:49 UTC
- 2021-09-29, 2 observed rows

I checked 398 conflicting rows. All 398 matched daily archive + API/uiKlines and did not match the monthly archive values.

## Reproducible example

### BTCUSDT 1m, open time 2020-12-21 13:19:00 UTC

Monthly archive file:

```text
https://data.binance.vision/data/spot/monthly/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12.zip
```

Daily archive file:

```text
https://data.binance.vision/data/spot/daily/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12-21.zip
```

Observed monthly checksum:

```text
a53a9352fd808d10981f68191138b39ef142e15ca88e03a061309f1699419749 BTCUSDT-1m-2020-12.zip
```

Observed daily checksum:

```text
59543b0862faf5953ce8c4e4efe3892d681711de345ced3ec5cf4dd9185a7154 BTCUSDT-1m-2020-12-21.zip
```

Monthly archive row:

```text
1608556740000,22647.85000000,22661.57000000,22633.30000000,22646.53000000,47.74209100,1608556799999,1081400.91111170,760,16.52699800,374330.37124588,0
```

Daily archive row:

```text
1608556740000,22647.85000000,22666.66000000,22628.62000000,22637.92000000,109.87666500,1608556799999,2488573.22763245,2138,49.06835600,1111469.47191801,0
```

API checks:

```text
https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1
https://api.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1
https://www.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1
```

All three API endpoints return the daily archive value:

```text
[1608556740000,"22647.85000000","22666.66000000","22628.62000000","22637.92000000","109.87666500",1608556799999,"2488573.22763245",2138,"49.06835600","1111469.47191801","0"]
```

## Shell reproduction

```bash
mkdir -p /tmp/binance-kline-check
cd /tmp/binance-kline-check

curl -LO https://data.binance.vision/data/spot/monthly/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12.zip
curl -LO https://data.binance.vision/data/spot/daily/klines/BTCUSDT/1m/BTCUSDT-1m-2020-12-21.zip

sha256sum BTCUSDT-1m-2020-12.zip BTCUSDT-1m-2020-12-21.zip

echo "Monthly row:"
unzip -p BTCUSDT-1m-2020-12.zip | awk -F, '$1==1608556740000 {print; exit}'

echo "Daily row:"
unzip -p BTCUSDT-1m-2020-12-21.zip | awk -F, '$1==1608556740000 {print; exit}'

echo "API kline row:"
curl -s 'https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1'

echo "API uiKlines row:"
curl -s 'https://api.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1'

echo "Website uiKlines row:"
curl -s 'https://www.binance.com/api/v3/uiKlines?symbol=BTCUSDT&interval=1m&startTime=1608556740000&endTime=1608556799999&limit=1'
```

## Additional examples

### BTCUSDT 5m, open time 2020-12-21 13:15:00 UTC

Monthly-derived row:

```text
close=22646.53000000
volume=496.20080600
num_trades=7440
```

Daily/API/uiKlines row:

```text
close=22637.92000000
volume=558.33538000
num_trades=8818
```

### ADAUSDT 5m, open time 2021-09-29 00:00:00 UTC

Monthly-derived row:

```text
close=2.04000000
volume=1099557.90000000
num_trades=1790
```

Daily/API/uiKlines row:

```text
close=2.04000000
volume=1103591.40000000
num_trades=1791
```

### USDCUSDT 1m, open time 2020-12-21 13:19:00 UTC

Monthly-derived row:

```text
close=1.00000000
volume=3150.53000000
num_trades=8
```

Daily/API/uiKlines row:

```text
close=1.00010000
volume=231669.02000000
num_trades=62
```

## Impact

This appears to affect a small number of historical candles, but the differences are material for backtesting and historical research because OHLC, volume, quote volume, and trade-count fields can change.

Observed in my audit:

- 398 conflicting rows checked.
- 398/398 daily archive values matched API/uiKlines.
- 0/398 monthly archive values matched API/uiKlines when daily differed.
- Max observed OHLC error: about 297.82 bps.
- Max observed close error: about 205.20 bps.
- Volume and trade-count fields were often materially understated in the monthly rows.

This can affect intraday backtests that use:

- high/low stop-loss or take-profit simulation;
- breakout signals;
- volume or quote-volume filters;
- trade-count filters;
- volatility filters;
- scalping logic on 1m, 5m, 15m, or 1h candles.

## Request

Could Binance please confirm whether the affected monthly SPOT kline archive files should be regenerated from the daily/API source of truth?

If the disagreement is expected, could Binance document which source should be treated as canonical when monthly archives disagree with daily archives and API/uiKlines?
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by running the shell reproduction against the monthly and daily archive URLs, then compare the sample rows with the listed klines and uiKlines API endpoints. Done means confirming the canonical source and either regenerating the affected monthly archives or documenting why the sources differ.

Written by the indexing model from the issue text.

Assessment

Domain
api, data
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.