8-bit inference
Open
bug
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🐛 Describe the bug
When using 8-bit inference, there is a hook function that is created by the `accelerate` library and that replaces the `forward` method. When using `inspect.getfullargspec` to set `self.forward_kwargs` with the arguments of `forward`, we actually inspect the arguments of the hook instead of the arguments of the real `forward` function called by the hook.
### Which trlX version are you using?
0.7.0
### Additional system and package information
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.