hiyouga / hiyouga/Dual-Contrastive-Learning

Some questions with baselines

Open
#7 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
168
Forks
30
PR merge metrics
No merged PRs in 30d

Description

Your work is very good and effective. But I have some questions about the baseline approach. I tried different hyperparameters to adjust supervised contrastivelearning or unsupervised contrastive learning to fine-tune BERT, and then to classify. But I've never been able to do anything better than just Cross-Entropy. I wonder what I didn't take into account? I've seen a lot of papers that contrastive learning can help improve classification results, but here I always get the opposite. Maybe I want to know the hyperparameters you set when you ran the comparison.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the repository's baseline training and comparison configuration, then compare it with the hyperparameters described in the issue's BERT fine-tuning experiments. Done means the baseline settings and evaluation procedure are documented well enough to explain the reported Cross-Entropy result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.