RosettaCommons / RosettaCommons/foundry
Is it desirable to utilize tensor cores by modifying matmul precision in bin files?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 966
- Forks
- 181
- Avg merge
- 4d 4h
- Merged PRs (30d)
- 2
Description
When running rfd3 there is a line that says:
You are using a CUDA device ('NVIDIA GeForce RTX 4060 Laptop GPU') that has Tensor Cores. To properly utilize them, you should set `torch.set_float32_matmul_precision('medium' | 'high')` which will trade-off precision for performance. For more details, read https://pytorch.org/docs/stable/generated/torch.set_float32_matmul_precision.html#torch.set_float32_matmul_precision
If I modify the rfd3 in the bin to this:
#!/opt/.miniconda/envs/protein_foundry/bin/python3.12
try:
import torch
# TO-DO: make if-statement for tensor cores?
torch.set_float32_matmul_precision("high")
except Exception:
print("Failed to Set matmul precision.")
import sys
from rfd3.cli import app
if __name__ == '__main__':
if sys.argv[0].endswith('.exe'):
sys.argv[0] = sys.argv[0][:-4]
sys.exit(app())
The comment goes away. That said, I have no idea if this is a good idea or if the precision loss will be harmful in some way.
I tried using seed=42 in the command line interface; however, I got subtle differences even without this modification that make me think I might need an additional setting to make it deterministic. Weirdly, the "high" precision setting gave me something more different than the "medium" precision setting. My best guess is time-derived pseudo-randomness somewhere is messing with the outputs.
Since the medium gave me something more similar, I'm thinking I will leave this in my bin file, but since I really don't know what I'm doing I thought it wise to ask the experts. Thanks!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the rfd3 bin entry point and its import of rfd3.cli.app, then review the linked PyTorch set_float32_matmul_precision documentation. Compare rfd3 runs with seed=42 under the reported precision settings and record the observed output differences. Done means the project has a clear, tested decision about whether the setting belongs in the launcher and how determinism should be handled.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- cli, machine-learning, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100