workload_generate_aiob前向反向compute time计算问题
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 184
- Avg merge
- 13h 4m
- Merged PRs (30d)
- 1
Description
文件：aicb/workload_generator/AIOB_simAI_workload_generator.py
类：class SIMAI_workload
函数：workload_generate_aiob
现象：
- 不使用专家并行，不使用序列并行，开启重计算(recompute_activations)的时候，
“forward_compute_time *= 2” 并没有赋值到后面backward计算中。
2.backward_compute_time没有包括反向grad compute time。
问题： - 这里forward_compute_time *= 2应该是要计入backward计算时间吧？
- 为什么代码中只有"self.args.recompute_activations and 'attention' in name"的时候才执行重计算？
- 反向grad compute time计入到哪个部分了？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with aicb/workload_generator/AIOB_simAI_workload_generator.py, then inspect class SIMAI_workload and workload_generate_aiob. Trace how forward_compute_time, backward_compute_time, recompute_activations, attention, and backward grad compute time are handled. Done means the intended accounting is resolved and the three reported questions have a clear, validated answer.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100