a2ui-project / a2ui-project/a2ui

[FEATURE]: Make the express format perform well with Gemma 2B models

未关闭
#2,569 1 条评论 0 个 reaction 已指派 1 人 已被 @jacobsimionato 认领 在 GitHub 查看
P2
主要语言
TypeScript
星标
16.4k
派生
1.3k
平均合并
2 天 13 小时
30 天内合并 PR
134

描述

https://github.com/a2ui-project/a2ui/pull/2551 proposes a new "Vertical" inference format which is faster and more accurate, especially on Gemma models.

Vertical performs way better than Express. But, potentially we could update the Express system prompt to be more descriptive, or make the compiler more permissive, or change the format in some way to avoid the types of errors that we see.

Steps
- Use the new eval cases added in https://github.com/a2ui-project/a2ui/pull/2551 and evaluate the same Gemma 2B model with Express format
- Brainstorm different changes we could make to the express prompt or parser, and see if they fix the issues. We can just do small runs, e.g. 5-10 data points to verify this at first. Let's prefer simple tweaks e.g. to the prompt first, before making more radical changes the compiler or format itself.
- Prepare a report on the different approaches that are possible, and how effective they are. They could be implemented as "options" on the express format perhaps. At this point, consider doing larger eval runs with 2-5 runs per data point to build confidence that the fixes work well.

贡献指南

打开贡献指南

调研方向

Review the new eval cases from PR #2551. Run the Gemma 2B model with the existing Express format to benchmark performance. Experiment with tweaks to the Express system prompt, then test on 5-10 data points. Document the effectiveness of each approach and consider making them configurable options.

由索引模型根据 Issue 内容生成。

评估

技术栈
machine-learning, typescript
领域
ai, tooling
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
活跃
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。