Brainstorms about code refactor

Here is a simple list of brainstorms and nits of code refactors to further improve code clarity

• Refactor the Buffer and rollout buffer (@fzyzcjy several PRs, partial)
• Use hydra or something similar for structured configs
• Refactor (extract) weight update logic into separate classes (need to also consider FSDP) #91 (partial)
• Pass in variables to replace globals in megatron_utils/data.py - #84
• Refactor sglang_example.py (@fzyzcjy some PRs, partial)
• Maybe refactor eval vs train to make it more symmetric/dense
• (tiny) Scaleable update_weight_utils #89, make args not mutated, refactor RayActor, etc (not listed here since long)
• TODO: carefully read code and propose more

Details are discussed offline with @zhuzilin

(seems I cannot do comments for code on a branch - can only do it for code diff on a PR...)

naive brainstorm for buffer & sglang_example refactor: roughly remove rollout_buffer folder, refactor Buffer into pieces, refactor sglang_example.py and Buffer, change data flow. pseudocode: https://gist.github.com/fzyzcjy/c0f59f947af38a23c588f79bffc3e830

oudated fig:

Start by reviewing the brainstormed areas: the rollout_buffer folder, Buffer, sglang_example.py, and the linked pseudocode gist. The issue describes broad data-flow and configuration refactors, with several items already partial or handled elsewhere. Done would require narrowing the scope and agreeing on a concrete refactor plan.