alibaba / alibaba/FederatedScope

Flex-lora里面数据集加载异常,报错如下

Open
#793 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.5k
Forks
261
PR merge metrics
No merged PRs in 30d

Description

Downloading and preparing dataset json/default to file:///home/xidiannss/test/cache/huggingface/datasets/json/default-d8c646ba37a080c7/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51...
Downloading data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 6921.29it/s]
Extracting data files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 1585.75it/s]
Dataset json downloaded and prepared to file:///home/xidiannss/test/cache/huggingface/datasets/json/default-d8c646ba37a080c7/0.0.0/0f7e3662623656454fcd2b650f34e886a7db4b9104504885bd462096cc7a9f51. Subsequent calls will reuse this data.
Traceback (most recent call last):
File "/home/xidiannss/test/datasets_test.py", line 2, in
load_dataset(path="json", data_files="global_test.json",cache_dir="cache/huggingface/datasets")
File "/home/xidiannss/anaconda3/envs/zc-lora/lib/python3.10/site-packages/datasets/load.py", line 1794, in load_dataset
ds = builder_instance.as_dataset(split=split, verification_mode=verification_mode, in_memory=keep_in_memory)
File "/home/xidiannss/anaconda3/envs/zc-lora/lib/python3.10/site-packages/datasets/builder.py", line 1089, in as_dataset
raise NotImplementedError(f"Loading a dataset cached in a {type(self._fs).__name__} is not supported.")
NotImplementedError: Loading a dataset cached in a LocalFileSystem is not supported.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with datasets_test.py and the load_dataset(path="json", data_files="global_test.json", cache_dir="cache/huggingface/datasets") call shown in the report. Reproduce the failure and inspect the dataset-loading path around the LocalFileSystem NotImplementedError. Done means the JSON dataset loads successfully from the configured cache without this exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.