HelloZeroNet / HelloZeroNet/ZeroNet
[Scalability] The proposal of the new policy of processing data/user/*/* files
- Dominant language
- JavaScript
- Stars
- 18.8k
- Forks
- 2.3k
- PR merge metrics
- No merged PRs in 30d
Description
ZeroNet has blacklists now, which allows us not seeing some unwanted persons, but doesn't help protecting agains massive automated attacks.
On the other hand, we can implement whitelists. If ZeroNet refuses to recieve and save the user data from users not in whitelist, that policy is totally resistant to spam. But it has a significant drawback: a new person has no chance to be whitelisted, since no one in the whole world recieves and sees her messages by default.
So we need some mix of the both possibilities.
My proposal is that we can have a 3-way White/Gray/Black classification, and the Gray area is dinamically adjusted depending on the disk space available and other conditions (such as an automatical spam detection).
Probably we should also use 2 size limits per a site, not just one: the soft limit and the hard limit.
How it works:
* **Blacklisted user IDs.** ZeroNet never saves data that is blacklisted on the local host.
* **Whitelisted user IDs.** ZeroNet processes data from whilested users in the usual way.
* **Any other (gray) user IDs.** ZeroNet receives the data and dispatches it to other hosts in the usual way, but if the site grows above the soft limit, some data can be deleted from the local host.
We should have some rule to decide, which data should be deleted and when it should happen. I have the following considerations:
* All gray users are equal for us (until the spam detection is implemented), so we can choose them randomly.
* But the recently modified data can potentially be more important, so we should keep it for some time. That means the pure random selection is not an option.
* The algorithm must not give the same results on different hosts, or else Zeronet drops the same users from all the hosts at once.
So I propose the following algorithm:
* ZeroNet generates a random number (X) for the local host and saves it localy.
* The random number X is used in the combination with the user ID to evaluate the affinity factor. It can be `affinity = X xor user_id` or `affinity = sha(X xor user_id)` or so. The affinity factor is pretty random per user, but it remains unchanged over time, so we can rely on it.
* For every gray user, the function is evaluated: `current_affinity = f(time_from_the_last_modification, affinity_factor)`. The details of f() is not important here, but the result of f() should be inversely proportional fo time_from_the_last_modification and directly proportional to affinity_factor.
* When the soft size limit is exceeded, ZeroNet tries to delete the data with the lowest current_affinity first and proceeds until the site size gets back to the limit. Data with the high affinity are probably never will be deleted, which guards us from accidental erasing data of inactive users from whole the network.
With that mechanics, the new posts, comments, messages etc are always displayed, so we can read them and react (answer the post, like it, add the author to the white or black list and so on). If the site runs out of the disk limit, the older content gradually fades out. It looks quite natural, since if we didn't react to the content, we were probably not interested. And if we want to see the whole site, we can increase the limit at any time and Zeronet will redownload the data.
The sites should have an API to whitelist users automatically. For example, when you *follow a user* at ZeroMe, *like a post*, *press the Reply button*, the site code knows you are interested, and the user is whitelisted.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.