facebook / facebook/astryx

Vibe Test — API Concerns (rolling)

Open
#3,164 27 comments 0 reactions 0 assignees View on GitHub
vibe-test
Dominant language
TypeScript
Stars
13k
Forks
1.1k
Avg merge
1d 15h
Merged PRs (30d)
690

Description

# Vibe Test — API Concerns (rolling)

This is the **single rolling tracker** for API concerns surfaced by the nightly vibe test. Instead of filing a new issue per night, the Vibe Test Debugger appends a comment here whenever a nightly suggests a component API fights developer/agent expectations.

**Numeric scores live on the wiki:** [Vibe Test Scores](https://github.com/facebookexperimental/xds/wiki/Vibe-Test-Scores). This issue is for **actionable API discussion only**.

## How this works

- The nightly vibe test records scores → appended to the wiki scores page.
- When the Debugger classifies a failure as an **API concern** (not a CLI-doc or setup issue), it adds a comment below with: the component, the friction, reproduction count, and which nightly(ies) it appeared in.
- An API concern is only actionable after it **reproduces across 2+ nightlies** (single-night differences are often model variance). Comments track that recurrence here.
- Resolved/validated concerns get checked off; abandoned ones (not reproducible) are noted and closed out in-thread.

## Why a single issue

Previously every nightly filed its own issue and closed the prior one, scattering API findings (Button `label`, Icon `name`, variant naming, ToggleButtonGroup `onChange`) across dozens of closed issues — making the "has this recurred?" check require archaeology. Consolidating here makes recurrence obvious.

## Currently tracked

| Concern | Component | First seen | Status |
|---------|-----------|------------|--------|
| `label` required even with `children` | Button | 2026-06-19 | tracking |
| `name` vs `icon` prop | Icon | 2026-06-14 | tracking |
| Variant naming (`filled`/`outlined` vs `primary`/`secondary`) | Button | 2026-06-13 | tracking |
| `onChange` type friction (single-select) | ToggleButtonGroup | 2026-06-18 | tracking |

_New concerns are added as comments and rolled into this table._

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.