hadriansecurity / hadriansecurity/subwiz
Implement batch inference for multi-apex
Open
- Dominant language
- Python
- Stars
- 391
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
Use batch inference for efficiency
Edit:
> when we perform inference for multi-apex we essentially do a for loop
> ```python
> gpt_model, tokenizer = get_model_and_tokenizer(force_download, device=device)
> found_domains = set()
>
> for apex in sorted(domain_groups):
> found_domains |= _get_domains_for_group(...)
>
> return sorted(found_domains)
> ```
>
> Here we are not making use of batch inference, as in we don't batch different apex related subdomains simultaneously.
Contributor guide
Assessment
This issue has not been assessed yet.