Is this a bug, or user error: NotImplementedError: Dataset is not picklable
Open
Nobody has claimed this yet.
bug
dask
- Dominant language
- Python
- Stars
- 150
- Forks
- 23
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 2
Description
python
Python 3.11.8 | packaged by conda-forge | (main, Feb 16 2024, 20:53:32) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cf
>>> cf.__version__
'3.16.1'
Attempt to use cf-python to read pp and write some netcdf. Code is:
import cf
import dask
dask.config.set(scheduler='processes',num_workers=12)
def convert(glob):
ff = cf.read(glob)
cf.write(ff,'all_year.nc',mode='w')
if __name__=="__main__":
convert('*.pp')
Platform is jasmin sci6, data is N1280 pp output.
Error log here
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the minimal Python script in the issue and inspect the linked error log, focusing on the dask process scheduler and the “Dataset is not picklable” failure. Determine whether the failure is caused by cf.read/cf.write usage or cf-python, then document a reproducible conclusion and the expected fix or workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100