carpentries-incubator / carpentries-incubator/lesson-parallel-python
new chapter for IO-boundness, data formats and xarray
Open
new content
- Dominant language
- Dockerfile
- Stars
- 11
- Forks
- 15
- Avg merge
- 3m
- Merged PRs (30d)
- 3
Description
Learning objectives:
- Assess the applicability of different data formats: csv, parquet, netcdf, npy, pickle.
- Discuss the importance of data standards with regards to FAIR principles.
- Assess IO performance for different formats.
- Use XArray (or equivalent machinery) to read and write massive amounts of data in parallel.
Contributor guide
Research direction
No files or tests are named. Review the existing lesson chapter structure and conventions first, then develop a chapter covering CSV, Parquet, NetCDF, NPY, and pickle formats, FAIR data standards, I/O performance, and parallel reading and writing with XArray or equivalent machinery.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data, documentation
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100