Implement a basic solution for continue learning with LLMs and make it compatible with current design

```[tasklist]
### Tasks
- [x] Identifying specific scenarios satisfy (1) Continuing learning with LLMs is meaningful. (2) The identified scenarios are not well-solved by existing works. Performing experiments to verify both (1) and (2).
- [ ] Perform experiments to verify some basic ideas. In this stage, compatibility is not mandatory.
- [ ] Implement the method formally. Make it compatible with current design.
```