Improve sorting and validation script

@Alex-302 commented on [Wed Jun 21 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420)

**Feature requests**
- rules validating:
For example:
`||example.com###advert` - the rule is invalid. `||` are unnecessary for element hiding rules;
`||example.com^$$domain=domain.org` - two option separators.
`||example.com^$domain=domain.org,domain2.com` - invalid domain separator. Must be `|`.
If it possible - add validation of CSS rules.
- black list for the files, which must be processed(the list of the files that should be sorted);
- committing changes to the repository;
- show request before/after sorting ;
- print all processed directories;
---

Currently our script has a bug with sorting order. Sometimes the order of sorting is violated - sorting of strings occurs alphabetically. Expected sorting process:
- sorting of element hiding rules in descending order;
- sorting by CSS selector in descending order;
- merging the rules with the same CSS selector;
- removing duplicated domains;
- sorting of URL rules
- sorting the rules in descending order;
- sorting of the rule modifiers(`domain=`, `app=`, `protobuf=`, 'replace=' must be at the and of modifiers list)
- sorting and merging domains list if the rules contain only `$domain=` modifier
- removing duplicates;
- types of rules that must be ignored(allow sorting as string, but don't change them):
JS(`#%#`), CSS(`#$#`), content replacing rules(`$replace`), protobuf(`$protobuf=`), Content Security Policy (`$csp`), application modifier for Android and Windows('$app=')

Comments `!` and hints `!+` are separators. Rules that are between the comments should be sorted separately from the rest.

Addition about the hints: the rule with hint should not be sorted - `!+`+`rule` .
For example:
```
!+ HINT
some.rule_zzz
some.rule_bbb
some.rule_aaa
```
result of sorting is
```
!+ HINT
some.rule_zzz
some.rule_aaa
some.rule_bbb
```

---

@vozersky commented on [Mon Aug 14 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-320921252)

We might also need to add a duplicate check to the script:
https://github.com/AdguardTeam/AdguardForWindows/issues/1850#issuecomment-322088426

---

@Alex-302 commented on [Wed Aug 09 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-321229847)

@vozersky


---

@ameshkov commented on [Mon Aug 14 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-322159826)

@Alex-302 need a better example.

For instance, give us a list with 50 rules and show how it should be sorted.

---

@Alex-302 commented on [Mon Aug 14 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-322183849)

#### Before

```
!
||graph.facebook.com^$app=jp.gocro.smartnews.android
||graph.facebook.com^$app=com.zynga.crosswordswithfriends
||graph.facebook.com^$app=com.urbandroid.sleep
! razlozhi.ru|yandex.by|yandex.com.tr|yandex.kz|yandex.ru|yandex.ua|syl.ru
/:\/\/(otzovik.com)\/[a-zA-Z0-9-_]*==\//$script,domain=otzovik.com
/:\/\/(otzovik.com)\/[a-zA-Z0-9-_]*=\//$script,domain=otzovik.com
!
!+ PLATFORM(iOS, ext_safari)
||oops.rustorka.com^
!
://*.rumedia.ws^$empty,domain=~rumedia.ws
rustorka.com##a[href^="http://www.gearbest.com/"]
rustorka.com##div[align="center"] > a > img
/(rustorka.com\/forum\/misc\/js\/(?!ifix)(?!ajax)(?!main)(?!fancybox)(?!scrolltopcontrol)(?!jquery)(?!ct)(?!bbcode))/$domain=rustorka.com
@@||rustorka.com/forum/shoutbox_view.php
!+ PLATFORM(ext_ff, ext_opera, ext_ublock)
rustorka.com##.post_body > div[align="center"] > a > img
!
myfin1.by##.menu_level_1 > li.left_bt
myfin2.by##.menu_level_1 > li.left_bt
myfin3.by##.menu_level_1 > li.left_bt
myfin4.by##.menu_level_1 > li.left_bt
bgoperator.ru##.l-index__side_a > canvas[width="188"][height="312"]
razlozhi.ru##._23 > div._68._36
tourister.ru##body > div[onclick="TourWindowCClose();"]
torrent-games.net###ubm_div
inosmi.ru##.banner
||r.mradx.net/img/D2/C79060.mp4
||r.mradx.net/img/20/9D02B4.ogg
esports.mail.ru##.rb-video-widget
esports.mail.ru##a.b-bg[target="_blank"]
seedoff.cc##div[id*="Composite"]
||r.mradx.net/img/D2/C79060.mp4
||r.mradx.net/img/20/9D02B4.ogg
esports.mail.ru##.rb-video-widget
esports.mail.ru##a.b-bg[target="_blank"]
seedoff.cc##div[id*="Composite"]
||animespirit.ru/banners/
fotokto.ru###pageContainer > div[class="m2"]
||simpsonsua.com.ua/photos/baner-reklama/
||cy-pr.com/images/stat/tr.png
miytvideo.ru##body > noindex
goldenshara.net##center > a[href="/goldenshara.net.php?url"] > img
zaycev.online##div[data-cookie="mainBanner"]
||bazr.ru/vast?$xmlhttprequest
||image.winudf.com/*/upload/promopure/$domain=apkpure.com
24auto.ru##.brrr_place
mp3cc.com###headerBanner
mp3cc.com##.playlist-ad
mp3cc.com###footerBanner
anistar.me##body > div[class^="heade-"]
chatovod.ru##.chatlist > tbody > tr[class="bold"]:not([class^="chatitem"])
||chatovod.ru/i/promo2/workle.png
```

#### After

```
!
||graph.facebook.com^$app=jp.gocro.smartnews.android|com.zynga.crosswordswithfriends|com.urbandroid.sleep
! razlozhi.ru|yandex.by|yandex.com.tr|yandex.kz|yandex.ru|yandex.ua|syl.ru
/:\/\/(otzovik.com)\/[a-zA-Z0-9-_]*==\//$script,domain=otzovik.com
/:\/\/(otzovik.com)\/[a-zA-Z0-9-_]*=\//$script,domain=otzovik.com
!
!+ PLATFORM(iOS, ext_safari)
||oops.rustorka.com^
!
/(rustorka.com\/forum\/misc\/js\/(?!ifix)(?!ajax)(?!main)(?!fancybox)(?!scrolltopcontrol)(?!jquery)(?!ct)(?!bbcode))/$domain=rustorka.com
://*.rumedia.ws^$empty,domain=~rumedia.ws
@@||rustorka.com/forum/shoutbox_view.php
rustorka.com##a[href^="http://www.gearbest.com/"]
rustorka.com##div[align="center"] > a > img
!+ PLATFORM(ext_ff, ext_opera, ext_ublock)
rustorka.com##.post_body > div[align="center"] > a > img
!
mp3cc.com###footerBanner
mp3cc.com###headerBanner
fotokto.ru###pageContainer > div[class="m2"]
torrent-games.net###ubm_div
razlozhi.ru##._23 > div._68._36
inosmi.ru##.banner
24auto.ru##.brrr_place
chatovod.ru##.chatlist > tbody > tr[class="bold"]:not([class^="chatitem"])
bgoperator.ru##.l-index__side_a > canvas[width="188"][height="312"]
myfin1.by,myfin2.by,myfin3.by,myfin4.by##.menu_level_1 > li.left_bt
mp3cc.com##.playlist-ad
esports.mail.ru##.rb-video-widget
esports.mail.ru##a.b-bg[target="_blank"]
anistar.me##body > div[class^="heade-"]
tourister.ru##body > div[onclick="TourWindowCClose();"]
miytvideo.ru##body > noindex
goldenshara.net##center > a[href="/goldenshara.net.php?url"] > img
zaycev.online##div[data-cookie="mainBanner"]
seedoff.cc##div[id*="Composite"]
||animespirit.ru/banners/
||bazr.ru/vast?$xmlhttprequest
||chatovod.ru/i/promo2/workle.png
||cy-pr.com/images/stat/tr.png
||image.winudf.com/*/upload/promopure/$domain=apkpure.com
||r.mradx.net/img/20/9D02B4.ogg
||r.mradx.net/img/D2/C79060.mp4
||simpsonsua.com.ua/photos/baner-reklama/
```

---

@Alex-302 commented on [Tue Aug 22 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-323991154)

### Feature request:
add support of blacklist file with dead domains list, which must be removed from the rules and the rules which contains only one domain.

---

@Mizzick commented on [Mon Oct 30 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-340542959)

@Alex-302 pls give us more details about content rules?
`example.org$$script[data-src="banner"]`

---

@Alex-302 commented on [Mon Oct 30 2017](https://github.com/AdguardTeam/AdguardFilters/issues/5420#issuecomment-340545386)

@Mizzick
What exactly?
Please take a look
https://kb.adguard.com/en/general/how-to-create-your-own-ad-filters#html-filtering-rules-1