Parallelism across entire preprocessing pipeline
Open
enhancement
good first issue
- Dominant language
- Python
- Stars
- 3
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Write now we use multiprocessing for semantic mapping, but we could also just do it over the entire pipeline. Maybe sklearn has a built-in function for this?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the preprocessing pipeline from its current multiprocessing implementation for semantic mapping, then review how sklearn could support parallel execution across the remaining stages. The issue names no files or tests, so first identify the relevant entry points and establish how correctness and performance should be measured before deciding what “entire pipeline” parallelism means.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, scikit-learn
- Domain
- machine-learning
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100