HelloZeroNet / HelloZeroNet/ZeroNet

Feature Request: Download size and Recursion Debth Limited Cluster Download Option

Offen
#1,290 0 Kommentare 2 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
JavaScript
Sterne
18.8k
Forks
2.3k
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

The idea is that while a person is reading one zite,
the server(Python code) checks, whether the zites that
are linked from that, open, zite, have been already downloaded
and if not, then downloads them in the background.

Only those zites can be downloaded in the background that
do not require the user to interact with any ZeroNet dialogues.

Maximum allowed recursion depth should be 3,

because otherwise inexperienced users can easily freeze their system.

The default value of for the recursion depth should be 0,

which means that the feature is SWITCHED OFF.
Recursion depth of 1 means that only the
zites that are directly linked from the open zite are
download candidates. The content.json already contains
file size fields, so it is possible to avoid downloading zites that are
"too big".

A counter-measure to an attack, where some zite links to
multiple TiB of zites, is that the overall size of the recursive
background download of a single opened zite has a limit,
which might be about 3MiB by default. Which zites to choose
to download is the classical Knapsack problem:

http://www.es.ele.tue.nl/education/5MC10/Solutions/knapsack.pdf
The same document at ZeroNet:
http://127.0.0.1:43110/19r55ZDt5UJtyTwRZkQCScyQT5Rt8smZ8/website_bonnet/various_files/2018_02_16_copy_of_http_www_es_ele_tue_nl_slash_education_slash_5MC10_slash_Solutions_slash_knapsack.pdf

The "value" that needs to be packed into the limited data-size backpack/knapsack
is the number of files. The more individual files, the better, because
the situation with their download is very similar to that of the classical FTP,
where x MiB worth of files takes much longer to download individually than
it takes to download them in a single, UN-compressed tar-file, because
with individual downloads there is the file download handshake overhead
and network delays with every file, but with a single tar-file the handshake
related overhead and network delays are borne only once.

I do not know, how the ZeroNet server's download system has been implemented, but
one optimization option might be to store very small zites that are not
owned by the server owner in a form of a pair of 2 files:

(content.json ; the_whole_zite.tar)

and use a temporarily un-packed copy of the zite at some
temporary local cache for local browsing. The reason, why

tar files MUST NOT BE COMPRESSED

is that compressing them introduces an attack option, where
someone compresses multiple TiB of zeros into a very tiny compressed
file and the moment the tiny compressed file is decompressed....

Thank You for reading this comment :-)

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.