LAION-AI / LAION-AI/CLAP

Is there any possible script/instruction to look at for fine-tuning?

Open
#129 9 comments 5 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.3k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

Hi! First of all thanks a lot for such amazing project.

I wonder if there is a valid way to fine-tune the model for specific tasks using customized datasets? I am trying to adapt the model to improve the performance and the source data structure I have is also for classification. It has below content:

  • Source audio
  • Text prompt
  • The event ID

I found that this script looks promising, but I do not know how to configure my data in order to replicate the process on my datasets since I found in the options, the sccipt has many setting very specific to ESC50. Do we have any gist on this problem? Thanks in advance!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with experiment_scripts/finetune-esc50.sh and compare its ESC50-specific options with the requested audio, text prompt, and event ID fields. Done means documenting the dataset format and configuration needed to reproduce fine-tuning on a custom classification dataset.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, shell
Domain
documentation, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.