google / google/example_extrapolation

Did you use clinc150 data_imbalanced vs data_full?

Open
#1 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
9
Forks
4
PR merge metrics
No merged PRs in 30d

Description

Hi,

I experimented with verifying your approach however I didn't receive such low F1 numbers for few shot. In looking at data_full, banking has the same amount of training samples as all other domains/intents. Did you in fact use data imbalanced?

Contributor guide

Open the contributing guide

Research direction

Compare the reported CLINC150 experiment with the repository's data_full and data_imbalanced datasets, paying particular attention to the banking domain and training-sample counts. First identify which dataset the experiments used, then verify the few-shot F1 results. Done means documenting whether the discrepancy comes from dataset selection or another reproducible setup difference.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.