lm-sys / lm-sys/FastChat

[Model Submission] OpenAI-Compatible API — Xiaothink-T17-RWKV5-MLA-0.6B-Frontend

Open
#3,922 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
39.5k
Forks
4.8k
PR merge metrics
No merged PRs in 30d

Description

**Organization:** Xiaothink Framework
**Submitter:** Individual developer / open-source contributor
**Model type:** Open weights + public API (OpenAI-compatible)

We would like to submit **Xiaothink-T17-RWKV5-MLA-0.6B-Frontend** to the Arena for evaluation. It is a 0.6B-parameter model trained from scratch, specialized in generating beautiful static frontend pages (single-file HTML/CSS/JS) from ultra-short prompts (10–20 tokens).

---

### Why This Model Fits Arena

- **Vertical Specialization:** Trained specifically for frontend static-page generation — a narrow but high-traffic category on Code Arena WebDev.
- **Primary Language Support (Chinese):** The model is specifically optimized for Chinese prompts and generating frontend pages with Chinese content, filling a gap for Chinese-language frontend tasks in the Arena.
- **Ultra-short Prompt Competence:** Produces complete, production-quality single-file HTML from ≤20-token descriptions, aligning with real user prompt patterns (Landing Page, Portfolio, Dashboard, etc.).
- **Competitive Blind-test Performance:** In an independent blind evaluation judged by DeepSeek-V4, the model **outperforms Llama-3-8B** and **approaches GPT-OSS-20B** quality on static frontend page generation tasks.
- **Open Weights:** Full model weights publicly available on ModelScope for transparency and reproducibility.
- **Novel Architecture:** RWKV-5 with Multi-head Latent Attention (MLA) — an RNN-Transformer hybrid offering constant memory usage and stable inference cost, relevant to Arena's diverse model pool.

---

### API Specification

| Field | Value |
|---|---|
| **Base URL** | `http://cn-hk-bgp-4.ofalias.net:27076/llmapi/v1` |
| **Protocol** | OpenAI API v1 compatible |
| **Endpoints** | `GET /models`, `POST /chat/completions` |
| **Auth** | `Authorization: Bearer ` |
| **Model name (for requests)** | `t17-rwkv5-mla-0.6B-frontend` |
| **Streaming** | Supported (`stream: true` → SSE; `stream: false` → JSON) |
| **Output format** | WanlyFrontend Code + Single-file HTML code |

---

### Example Requests

**Non-streaming:**
```bash
curl -X POST http://cn-hk-bgp-4.ofalias.net:27076/llmapi/v1/chat/completions \
-H "Authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"model": "t17-rwkv5-mla-0.6B-frontend",
"messages": [{"role": "user", "content": "极简的个人作品集网站"}],
"stream": false
}'
```

**Streaming (SSE):**
```bash
curl -N -X POST http://cn-hk-bgp-4.ofalias.net:27076/llmapi/v1/chat/completions \
-H "Authorization: Bearer " \
-H "Content-Type: application/json" \
-d '{
"model": "t17-rwkv5-mla-0.6B-frontend",
"messages": [{"role": "user", "content": "加密货币数据仪表盘"}],
"stream": true
}'
```

**Model list:**
```bash
curl -X GET http://cn-hk-bgp-4.ofalias.net:27076/llmapi/v1/models \
-H "Authorization: Bearer "
```

---

### Obtaining an Evaluation API Key

The API is not open-access (rate-limited to protect inference compute). The Arena team can obtain a dedicated evaluation key with elevated quota by contacting:

- **Email:** `xiaothink@foxmail.com`
- **Subject:** `[Arena Evaluation Key Request] Xiaothink-T17 Frontend`
- We will provision a long-lived key with sufficient rate limits for Arena's voting volume and commit to keeping it active for the full 30-day evaluation window per Arena's Leaderboard Policy.

---

### Open Weights & Technical Details

- **ModelScope:** https://www.modelscope.cn/models/ericsjq/Xiaothink-T17-RWKV5-MLA-0.6B-Frontend
- **Official site:** http://xiaothink.top/
- **Architecture:** RWKV-5 with Multi-head Latent Attention (MLA)
- **Parameters:** 0.6B
- **Primary Language:** Chinese (optimized for Chinese prompts and content generation)
- **License:** see ModelScope model card
- **Training:** from-scratch (not a fine-tune of an existing base model)
- **Specialty:** static frontend page generation (single-file HTML/CSS/JS) from ultra-short prompts

---

### Blind-Test Evidence (DeepSeek-V4 as Judge)

An independent pairwise blind evaluation was conducted using DeepSeek-V4 as the LLM judge, comparing generated static frontend pages on a held-out prompt set. Results:

| Model | Relative Quality |
|---|---|
| Llama-3-8B (general) | Baseline — Xiaothink-T17 outperforms |
| GPT-OSS-20B (general) | Xiaothink-T17 approaches parity |
| **Xiaothink-T17-0.6B-Frontend** | **Exceeds Llama3-8B, ~matches GPT-OSS-20B in this narrow vertical** |

Full evaluation logs and prompt sets are available on request. We understand Arena will re-evaluate using its own native mechanism (human blind voting); the DeepSeek-V4 score is provided only as a capability reference.

Image

---

### Availability Commitment

- API endpoint will remain publicly accessible for at least 30 days post-launch per Arena Leaderboard Policy.
- We will not pause or revoke access once testing begins.
- Compute is self-hosted on dedicated GPU infrastructure — not a shared free tier.
- We are prepared for the ~200-vote stabilization period and will maintain capacity throughout.

---

### Preferred Arena Category

- **Primary:** Code Arena → WebDev → HTML / Website
- **Secondary:** Code Arena → WebDev → UI Component

---

### Contact

- **Email:** xiaothink@foxmail.com
- **Organization:** Xiaothink Framework
- **Website:** http://xiaothink.top/

We are happy to make any API adjustments (additional endpoints, response format tweaks, etc.) the Arena team requires. Looking forward to your feedback.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the Arena model-submission requirements and the proposed OpenAI-compatible GET /models and POST /chat/completions endpoints. Verify the ModelScope model details, API availability, evaluation category, and 30-day availability commitment; done means the submission is accepted for evaluation or has a documented reason for rejection.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.