hiyouga / hiyouga/Dual-Contrastive-Learning
Some questions with baselines
- Dominant language
- Python
- Stars
- 168
- Forks
- 30
- PR merge metrics
- No merged PRs in 30d
Description
Your work is very good and effective. But I have some questions about the baseline approach. I tried different hyperparameters to adjust supervised contrastivelearning or unsupervised contrastive learning to fine-tune BERT, and then to classify. But I've never been able to do anything better than just Cross-Entropy. I wonder what I didn't take into account? I've seen a lot of papers that contrastive learning can help improve classification results, but here I always get the opposite. Maybe I want to know the hyperparameters you set when you ran the comparison.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the repository's baseline training and comparison configuration, then compare it with the hyperparameters described in the issue's BERT fine-tuning experiments. Done means the baseline settings and evaluation procedure are documented well enough to explain the reported Cross-Entropy result.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100