JacobLinCool / JacobLinCool/particeps

Implement a durable replay-safe Android upload outbox

Open
#9 0 comments 0 reactions 0 assignees View on GitHub
area: android P0
Dominant language
Kotlin
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Summary

Replace the retry-time export generation path with a durable, replay-safe Android upload outbox.

The app must construct and persist one complete encrypted Protocol v1 bundle before starting HTTP. Every retry of that staged item—after response loss, process death, or reboot—must transmit exactly the same bytes and identity. Collection remains append-only and may continue while an upload is terminally failed.

## Requirements

### Durable staging

- Generate the complete ciphertext bundle in Android no-backup storage before opening the HTTP request.
- Atomically persist the bundle and outbox metadata: `bundle_id`, exact first and last sequence, actual event count, byte count, SHA-256, configuration digest, creation time, and retry state.
- Permit at most one staged bundle per active study. New collection events remain outside that immutable staged range until it is acknowledged.
- Recover an incomplete create/metadata commit deterministically; never send an uncommitted or partially written file.
- Reboot, process death, WorkManager retry, and response loss must reuse the same file and metadata without re-encryption.
- Define safe cleanup and crash recovery for every transition from staged to acknowledged to reclaimed.

### HTTP contract

- Send a fixed `Content-Length`, standards-compliant `Content-Digest` using SHA-256, `bundle_id`, exact claimed range, event count, and configuration digest.
- Enforce a hard 32 MiB request-body limit. Target approximately 16 MiB per normal bundle so metadata and operational variation do not approach the hard bound.
- Disable redirects, including same-origin redirects.
- Accept only:
- `201 Created` with a receipt for a newly persisted object; or
- `200 OK` with the identical receipt for an exact replay.
- Reject `202 Accepted` and any acknowledgement whose bundle ID, digest, byte count, range, event count, or configuration digest does not exactly match the staged item.
- Retry only I/O failures and HTTP 408, 425, 429, and 5xx responses, respecting bounded backoff and `Retry-After` where valid.
- Treat every other 4xx as an explicit terminal upload failure for that staged item. Surface the failure locally without stopping collection or silently discarding data.

### Commit and reclaim

- Advance the upload watermark only after a complete matching receipt is durably committed locally.
- Never infer acknowledgement from a TCP/HTTP success alone.
- Permit encrypted event reclaim only for the acknowledged prefix and only after the watermark/receipt commit is recoverable.
- Preserve enough state to reconcile a crash between receipt receipt, local commit, file removal, and store reclaim.
- Emit bounded local lifecycle/audit events using fixed reason codes; never log ciphertext, identifiers, headers, or configuration material.

## Acceptance criteria

- Packet capture or test-server fixtures prove that every retry of one `bundle_id` is byte-for-byte identical.
- Tests cover process death and reboot before staging commit, during upload, after server persistence but before response, during local receipt commit, and during cleanup/reclaim.
- Exact replay returns/accepts `200`; a mismatched receipt, `202`, redirect, or conflicting response never advances the watermark.
- I/O, 408, 425, 429, and 5xx retry; other 4xx enter a visible terminal upload state while collection continues.
- Files over 32 MiB are never sent, and target chunking respects event boundaries without splitting one encoded event.
- Concurrent worker execution cannot stage or upload two bundles for the same study.
- The watermark never advances past an event not covered by an exact acknowledged receipt.
- Failpoint tests establish that cleanup cannot orphan an unacknowledged range or cause it to be regenerated under the same identity.

## Documentation

Update the upload contract, system design, researcher guide, data dictionary, threat model, operational troubleshooting, and Android storage/recovery design.

## Dependencies

Blocked by #8, which defines the final Protocol v1 bundle, digest, identifier, and receipt fields.

Contributor guide

Open the contributing guide

Research direction

No files or existing test paths are named; first read the Protocol v1 definitions in blocked issue #8, then locate the Android upload, persistence, and worker entry points. Use the listed process-death, reboot, replay, receipt, redirect, retry, size, concurrency, and cleanup scenarios as tests, and consider the work done only when the exact acceptance criteria hold.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, kotlin
Domain
api, data, mobile, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.