ByteDance-Seed / ByteDance-Seed/In-Place-TTT

关于In-Place TTT在instruction-tuned模型上的使用

Open
#2 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
286
Forks
26
PR merge metrics
No merged PRs in 30d

Description

@GuhFeng 作者您好,我最近正在学习模型测试时学习相关内容,非常感谢您开源这项优秀的工作。

留意到您在论文中主要使用base模型去continual training,我在想将in-place ttt应用到instruct模型(比如Qwen3-1.7B)上,考虑到base模型和instruct模型上差异,想请教几个问题:

1. in-place ttt模块是否可以直接应用到instruction-tuned模型上?这么做会不会有性能损失或者其他难以预见的结果?

2. 如果可以,是否推荐冻结模型其它参数,只训练新增的ttt模块?

再次感谢您的开源工作,期待得到您的回答🙏🏻

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no source files, tests, or entry points. First clarify whether support for instruction-tuned models such as Qwen3-1.7B is intended, then review the existing in-place TTT integration and determine how freezing the other model parameters should be evaluated; done means an agreed implementation path and documented experimental results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.