binance / binance/binance-public-data

Fix PERIOD_START_DATE restricts downloads before 2020-01-01

Open
#398 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
613
PR merge metrics
No merged PRs in 30d

Description

---

### **Description:** 📝
When attempting to download **daily `klines` data with 1-minute intervals** ⏱️ for dates earlier than **2020-01-01** 📅, the script fails to fetch the data correctly. This issue occurs due to the variable `PERIOD_START_DATE` in `enums.py`:

```
PERIOD_START_DATE = "2020-01-01"
```

⚠️ **Even when providing a `-startDate` argument earlier than `2020-01-01`, the download starts from `2020-01-01`.**
Example command:
```
python3 download-kline.py -t spot -skip-monthly 1 -i 1m -s BTCUSDT -startDate 2017-08-17 -endDate 2025-02-07
```

---

### **Cause of the Problem:** 🔍
The issue lies in the following section of `download-kline.py`, specifically on 📌 [line 108](https://github.com/binance/binance-public-data/blob/master/python/download-kline.py#L108):

```
period = convert_to_date_object(datetime.today().strftime('%Y-%m-%d')) - convert_to_date_object(PERIOD_START_DATE)
dates = pd.date_range(end=datetime.today(), periods=period.days + 1).to_pydatetime().tolist()
dates = [date.strftime("%Y-%m-%d") for date in dates]
```

❗ The constant `PERIOD_START_DATE` is used to generate the date range, overriding the `-startDate` argument.

---

### **Proposed Fix (Specific to `download-kline.py`):** 💡
I modified the logic to dynamically use the earlier date between `PERIOD_START_DATE` and the provided `-startDate` argument. This ensures that the script respects user-defined start dates:

```
period_start_date = PERIOD_START_DATE
if args.startDate and args.endDate and START_DATE <= convert_to_date_object(args.startDate):
period_start_date = args.startDate

period = convert_to_date_object(datetime.today().strftime('%Y-%m-%d')) - convert_to_date_object(period_start_date)
dates = pd.date_range(end=datetime.today(), periods=period.days + 1).to_pydatetime().tolist()
dates = [date.strftime("%Y-%m-%d") for date in dates]
```

✅ **This fix only affects** `download-kline.py` **and does not modify** `PERIOD_START_DATE` **globally, avoiding potential issues in other parts of the project.**

---

### **Expected Behavior:** ✔️
The download should correctly start from the user-defined `-startDate 2017-08-17`, even if it is earlier than `2020-01-01`.

---

### **Files Affected:** 📂
- **download-kline.py**
- **enums.py**

---

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in download-kline.py around line 108 and trace how PERIOD_START_DATE from enums.py is used to build the date list when -startDate and -endDate are supplied. Run the provided BTCUSDT 1-minute command with a pre-2020 start date and verify that the download begins at the requested date without changing the global constant.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
cli, data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.