Transfer file blocks instead of the whole file

[!CAUTION]
Do not work on this issue yet.
This is intended only for the first Beta Release and must not be started before then.

Currently, all files are handled as a single unit. When a change occurs in a file, the entire file is transferred to other peers, even if only a small part has changed. This is inefficient for large files or for frequent small edits.

Goal

Implement a mechanism to split each file into chunks/blocks, and when a change is detected in a specific block, transfer only that block to other peers instead of the whole file.

High-Level Requirements

• Introduce a concept of file blocks/chunks:
  • Define how files are split (fixed-size blocks, content-defined chunks, etc.).
  • Ensure the chunking is deterministic so that all peers can agree on block boundaries.
• Detect which blocks have changed when a file is modified.
• Transfer only the changed blocks (plus any necessary metadata) to other peers.
• Ensure data integrity:
  • Consider using checksums/hashes per block.
  • Validate blocks upon receipt.
• Maintain compatibility with the existing protocol:
  • Define a migration or fallback path for peers that only support whole-file transfers.
  • Ensure that older peers can still synchronize correctly (even if less efficiently).

Considerations / Open Questions

• Chunk size strategy: fixed size vs. variable (e.g., content-defined chunking).
• Indexing: how to efficiently map file offsets → blocks and detect dirty/changed blocks.
• Performance trade‑offs:
  • CPU cost for computing hashes/checksums.
  • Memory overhead for block metadata.
• Failure handling:
  • How to retry or request missing/corrupted blocks.
  • How to reconcile when block metadata and file contents diverge.

No files, tests, or entry points are named. First map the existing whole-file transfer and peer protocol paths, then document how chunking, change detection, integrity checks, and fallback would fit them. Done means changed blocks synchronize correctly while older peers remain compatible, but the design questions need resolution before implementation.