Imageomics / Imageomics/TreeOfLife-toolbox
Reformulate as TreeOfLife-toolbox
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 7
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Specifically presented as taking the output of `distributed-downloader` and then does all these things to _create_ the TreeOfLife dataset.
- [x] Rename repo: TreeOfLife-toolbox
- [x] Rename under `src`: `TOL-toolbox`
- [ ] Reorganize under `TOL-toolbox` to have submodules based on function (e.g., `filtering`, `data-transfer`) with sub-submodules (e.g., `research-filtering`)
- [ ] Add from TreeOfLife-dev as submodules in `src/TOL-toolbox`:
- [ ] webdataset creation
- [ ] metadata transfer
- [ ] lookup table creator
- [ ] Add from TreeOfLife-dev at root-level `scripts/` directory all lookup table exclude creators (many of these are [here](https://github.com/Imageomics/TreeOfLife-dev/tree/main/src/processing)):
- [ ] PDQ hash process
- [ ] Image type separator (museum, camera trap, and citizen science categories)
- [ ] Museum Specimen processing: label vs specimen filter
- [ ] camera trap processing: megadetector & reducer
- [ ] citizen science image processing
- [ ] face detection
- [ ] Add from TreeOfLife-dev at root-level `notebooks/` directory all lookup table exclude creators:
- [ ] clustering and other EDA determinations
- [ ] Add root-level `data/` folder with embeddings for museum support sets (relates to #22).
This will be done following the merge of PRs #1 through #15.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing PRs #1 through #15, since the issue says this work follows their merge. Then inspect the existing src/TOL-toolbox layout and the referenced TreeOfLife-dev processing directory before mapping work into src/TOL-toolbox, scripts/, notebooks/, and data/. Done means the unchecked reorganization, additions, and embedding folder in the checklist are completed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook
- Domain
- computer-vision, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100