hadriansecurity / hadriansecurity/subwiz

Adding fake subdomains to a low-resource domain.

Open
#29 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
391
Forks
25
PR merge metrics
No merged PRs in 30d

Description

I sometimes encounter a situation where there are very few examples of subdomains for a certain domain, for example, there is only `www.some_domain.com` or `subdomain.com` and there is no more data.

Two simple ideas arise:
- Before tokenization, add fake subdomains that will provide additional information for the model and possibly improve search results. For example `[git, gitlab, ...]`, or `[ftp, mail, ...]`. It seems like a difficult question, which subdomains to choose. Perhaps it is worth considering the distribution of the training data.
- Enable the option to automatically restart subdomain scanning using information from newly found subdomains. Sure, you could get a new list of domains, paste it into `example_input.txt`, and restart, but that doesn't seem very convenient. You can continue searching in a loop until no new domains appear.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.