执行yahoo normalize data 发生 ValueError: unconverted data remains when parsing with format "%Y-%m-%d" 错误
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 48.7k
- Forks
- 7.7k
- PR merge metrics
- No merged PRs in 30d
Description
🐛 Bug Description
To Reproduce
Steps to reproduce the behavior:
1.python scripts/data_collector/yahoo/collector.py normalize_data --source_dir ~/.qlib/stock_data/source/cn_data --normalize_dir ~/.qlib/stock_data/normalize/cn_data --region CN --interval 1d
2.我已经下载了cn_data数据,normalize 过程中,在第341 sh600489.csv 发生错误
Expected Behavior
Screenshot
File "E:\Data\PROGRAM\Python\source\qlib\scripts\data_collector\yahoo\collector.py", line 395, in normalize_yahoo
df.index = pd.to_datetime(df.index)
...
ValueError: unconverted data remains when parsing with format "%Y-%m-%d": " 09:30:00+08:00", at position 1343. You might want to try:
- passing format if your strings have a consistent format;
- passing format='ISO8601' if your strings are all ISO8601 but not necessarily in exactly the same format;
- passing format='mixed', and the format will be inferred for each element individually. You might want to use dayfirst alongside this.
Environment
Note: User could run cd scripts && python collect_info.py all under project directory to get system information
and paste them here directly.
- Qlib version:0.9.7
- Python version: 3.12.9
- OS (
Windows,Linux,MacOS): windows
下载的原始数据
sh600468.csv
sh600469.csv
归一后的数据
- Commit number (optional, please provide it if you are using the dev version):
Additional Notes
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with scripts/data_collector/yahoo/collector.py, especially normalize_yahoo around line 395, and reproduce the issue with the normalize_data command from the report. Inspect the attached CSV data around the failing row and how its timestamp format reaches pandas. Done means the supplied dataset completes normalization without the reported ValueError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100