LAION-AI / LAION-AI/Open-Assistant

Per language tree manager settings

Open
#1,257 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

backend
Dominant language
Python
Stars
37.4k
Forks
3.3k
PR merge metrics
No merged PRs in 30d

Description

Recently additional languages were added to the system. The current tree manager settings keeps 100 trees active which is a reasonable number for languages like English. For languages with few contributors this number is probably too high with the consequence that it will take a long time until trees reach their goal size, go through ranking and get the ready_for_export state.

Here is a snapshot of the language distribution of (non-synthetic) messages in the system ~2023-02-06 09:00 UTC:

 lang  | count 
-------+-------
 vi    |     9
 hu    |    10
 de    |  1410
 ja    |   105
 ko    |     4
 fr    |   699
 zh    |    97
 en    | 15118
 es    |   741
 pt-BR |   213
 ru    |  1056

Current goal_tree_size is set to 12 messages, i.e. to complete 100 trees 1200 messages are necessary. A simple solution would be to specify certain settings like the number of active trees per language. We could for example use 25 as default and specify higher values for the most active languages. The configuration parameters of the tree manager can be found in config.py#L8-L142.

(If you have ideas for an algorithm to dynamically adjusts the max_active_trees value please let us know. Please also consider side-effects like less diverse tasks/less fun for contributors when the tree count is getting too low.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with backend/oasst_backend/config.py, especially lines 8-142, and trace where the tree manager reads max_active_trees and goal_tree_size. Define how language-specific settings and a default should be represented, then verify that active-tree limits respond appropriately to the message distribution without removing task diversity. No test file or additional entry point is named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.