JesseRWeigel / JesseRWeigel/get-math-done

Evaluate Kimi K2.6 as research router (BrowseComp 83.2)

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Context

ThursdAI 2026-04-23 covered **Kimi K2.6** (Moonshot AI) with numbers directly relevant to research-copilot agents:

- 1T-param MoE, 32B active
- SWE-Bench Pro: 58.6 (beats GPT-5.4 and Opus 4.6)
- **BrowseComp: 83.2** — this is the one we care about
- 256K context
- $0.95/M input tokens on Cloudflare Workers AI
- Modified MIT license
- Artificial Analysis Intelligence Index: #4 (score 54)

BrowseComp measures agentic research performance (multi-step web exploration, tool use, cross-document synthesis). The get-X-done ecosystem is exactly that workload.

## Proposal

Evaluate Kimi K2.6 as a router/researcher model against whatever this copilot currently uses (likely Claude Sonnet 4.6 or similar).

1. Wire Kimi K2.6 (via Cloudflare Workers AI or direct Moonshot API) as an alternate model.
2. Run the copilot's existing benchmark tasks (or pick 10 representative real queries).
3. Compare: answer quality, correctness, cost per query, latency.
4. Decide: adopt, hybrid (route by query difficulty), or reject.

Companion issue should exist in each of the get-X-done repos. The tier-1 repos (math, review, legal) evaluate first; tier-2/3 follow if results are favorable.

## Links

- ThursdAI 2026-04-23: https://sub.thursdai.news/p/apr-23-openais-week-gpt-55-gpt-image
- Kimi K2.6 on HF: https://huggingface.co/moonshotai/Kimi-K2.6

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.