lablup / lablup/backend.ai-webui

Decide whether the Astryx catalog's bare generic keywords are worth their cost

Open
#9,145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133
Forks
81
Avg merge
1d 12h
Merged PRs (30d)
355

Description

Follow-up to FR-3709. **Rewritten after measuring against the CLI's own scorer** — the recipe this ticket originally prescribed is disproven, see "The trade-off" below.

## How the ranker actually scores (measured, not read)

Measured by calling `scoreQuery` / `tokenizeQuery` from the installed `@astryxdesign/cli@0.5.0` directly, so these are the running numbers rather than a reading of the source.

Single signal, best wins: exact name 100, exact keyword 90, keyword substring (min 4 chars, >=50% overlap) 70, description whole-word 50. For a multi-word query, each token is scored and the total is `avg(matched) + min(matched-1,3)*12 + coverage*15`. Ties break by domain, then **alphabetically by name**.

For a component doc, only `name`, `keywords` and `usage.description` are indexed. `bestPractices` and `examples` are not.

## What this means for BAI\* vs core

**One-word query that is a core component's exact name: 90 is our hard ceiling and it cannot be beaten.** Measured — a `BAICard` doc loaded with every keyword trick still scores 90 against `Card`'s 100. The only route to 100 is for our doc's `name` to BE the query, i.e. shipping a doc named `Card` from BUI, and that is worse than the problem: two packages owning one name makes `astryx component Card` fail with `Component "Card" is provided by multiple packages. Re-run with --package `. Not worth doing.

_Two-word queries: BAI_ already wins, with no change needed.\* Measured:

|query|BAICard|core Card|
|---|---|---|
|`card`|90|**100**|
|`card container`|**117**|102|
|`card with tabs`|**117**|108|
|`section card`|**117**|108|

The coverage bonus is what does it, and our docs carry richer keyword sets than core's. Since an agent's real queries are phrases rather than bare nouns, the common case is already right.

## The remaining problem, and the trade-off

Where BAI\* loses is not to core but to **its own siblings**: `BAITable`, `BAISelect` and `BAITag` tie at 90 with the ~60 specialised components that also claim the bare noun, and the alphabetical tie-break then buries the generic one (`BAITable` is rank 20 for `"table"`).

The original prescription here — "strip the bare generic keyword from specialised components" — **backfires**, measured on `BAITable` vs `BAIAgentTable`:

|sibling's keywords|`table`|`agent table`|
|---|---|---|
|today, includes bare `table`|BAIAgentTable 90 > BAITable 90|BAIAgentTable **117** > BAITable 98|
|qualified `agent table`, description still says "table"|BAITable 90 > BAIAgentTable 50|BAITable 98 > BAIAgentTable **97**|
|qualified, description avoids "table"|BAITable 90 > BAIAgentTable 43|tie 98 = 98, decided **alphabetically**|

So qualifying fixes the bare word and costs the sibling its own query — either losing it outright, or reducing it to a spelling coin-flip (`BAIAgentTable` happens to sort before `BAITable`; a `BAIUserTable` would lose the same tie).

## The decision this ticket is really asking for

There is no keyword arrangement that gives both. Pick one:

1. **Leave it.** Bare `"table"` keeps returning a specialised sibling first. Phrase queries — the common case — are already correct, and `astryx component BAITable` is exact and unaffected.
1. **Qualify the siblings**, accepting that each specialised component's own two-word query drops from ~117 to ~98 and, where the names sort the wrong way, may fall behind the generic one.

Recommendation: **option 1**, unless someone observes an agent actually reaching for a bare noun and picking the wrong component. The measurement above is the evidence to revisit with, not a reason to churn 60 doc files now.

## Non-goals

Do not chase rank 1 against a core component whose exact name is the query — that ranking is correct, and the only mechanism that would beat it breaks `astryx component`.

JIRA Issue: FR-3719

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing FR-3709 and the measured scoreQuery/tokenizeQuery results from @astryxdesign/cli@0.5.0. Compare the two keyword strategies against the documented BAIAgentTable and BAITable cases, then record whether to leave the catalog unchanged or qualify sibling keywords; done means a decision is documented without unnecessary changes to the component docs.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
cli, search
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.