binance / binance/binance-public-data
Fix PERIOD_START_DATE restricts downloads before 2020-01-01
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 613
- PR merge metrics
- No merged PRs in 30d
Description
---
### **Description:** 📝
When attempting to download **daily `klines` data with 1-minute intervals** ⏱️ for dates earlier than **2020-01-01** 📅, the script fails to fetch the data correctly. This issue occurs due to the variable `PERIOD_START_DATE` in `enums.py`:
```
PERIOD_START_DATE = "2020-01-01"
```
⚠️ **Even when providing a `-startDate` argument earlier than `2020-01-01`, the download starts from `2020-01-01`.**
Example command:
```
python3 download-kline.py -t spot -skip-monthly 1 -i 1m -s BTCUSDT -startDate 2017-08-17 -endDate 2025-02-07
```
---
### **Cause of the Problem:** 🔍
The issue lies in the following section of `download-kline.py`, specifically on 📌 [line 108](https://github.com/binance/binance-public-data/blob/master/python/download-kline.py#L108):
```
period = convert_to_date_object(datetime.today().strftime('%Y-%m-%d')) - convert_to_date_object(PERIOD_START_DATE)
dates = pd.date_range(end=datetime.today(), periods=period.days + 1).to_pydatetime().tolist()
dates = [date.strftime("%Y-%m-%d") for date in dates]
```
❗ The constant `PERIOD_START_DATE` is used to generate the date range, overriding the `-startDate` argument.
---
### **Proposed Fix (Specific to `download-kline.py`):** 💡
I modified the logic to dynamically use the earlier date between `PERIOD_START_DATE` and the provided `-startDate` argument. This ensures that the script respects user-defined start dates:
```
period_start_date = PERIOD_START_DATE
if args.startDate and args.endDate and START_DATE <= convert_to_date_object(args.startDate):
period_start_date = args.startDate
period = convert_to_date_object(datetime.today().strftime('%Y-%m-%d')) - convert_to_date_object(period_start_date)
dates = pd.date_range(end=datetime.today(), periods=period.days + 1).to_pydatetime().tolist()
dates = [date.strftime("%Y-%m-%d") for date in dates]
```
✅ **This fix only affects** `download-kline.py` **and does not modify** `PERIOD_START_DATE` **globally, avoiding potential issues in other parts of the project.**
---
### **Expected Behavior:** ✔️
The download should correctly start from the user-defined `-startDate 2017-08-17`, even if it is earlier than `2020-01-01`.
---
### **Files Affected:** 📂
- **download-kline.py**
- **enums.py**
---
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in download-kline.py around line 108 and trace how PERIOD_START_DATE from enums.py is used to build the date list when -startDate and -endDate are supplied. Run the provided BTCUSDT 1-minute command with a pre-2020 start date and verify that the download begins at the requested date without changing the global constant.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- cli, data-engineering
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100