dmarx / dmarx/Multi-Modal-Comparators
Feature brainstorming
- Dominant language
- Jupyter Notebook
- Stars
- 39
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
- [ ] loss scoring
- [x] multi-perceptor
- [x] weighted multi-perceptor
- [ ] cutout methods? + augs? make that an independent library maybe?
- [ ] perceptor weight interpolations/schedules - https://discord.com/channels/729741769192767510/730484623028519072/956979309686423602
- [x] API should be agnostic wrt media type, i.e. contrasting modalities could both be text, or one be audio and other video, etc.
- [ ] optionally augment w positional information/embeddings?
- [ ] Maybe some minimal translation API to facilitate use by non-english users and conversely support for non-english models
- see aphantasia's SBERT utilization: https://github.com/eps696/aphantasia
- [x] Check for installed/available CLIP, use vendored if not available
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.