arkavo-org / arkavo-org/OpenTDFKit
StreamingBenchmark: testMemoryUsageWith2MBChunk peaks ~213MB vs 70MB assertion (flaky in test job)
- Dominant language
- Swift
- Stars
- 1
- Forks
- 1
- Avg merge
- 9d 4h
- Merged PRs (30d)
- 1
Description
## Summary
`StreamingBenchmarkTests.testMemoryUsageWith2MBChunk` asserts peak memory **< 70 MB** when streaming-encrypting a 50 MB file in 2 MB chunks, but actual peak is **~213 MB** — over 3× the threshold. The test fails deterministically locally and is environment-flaky in CI's `test` job.
## Evidence
`OpenTDFKitTests/StreamingBenchmarkTests.swift:156`
```
XCTAssertLessThan(peakMemory, Int64(70 * 1024 * 1024), "Peak memory should be less than 70MB for 50MB file")
// XCTAssertLessThan failed: ("213483520") is not less than ("73400320")
```
- Fails in **both debug and release** (`swift test` and `swift test -c release`) → not a debug-only artifact.
- **Pre-existing**: reproduces identically on `77920b3` (predates the ConnectRPC work in #39).
- **Flaky in CI**: the `test` job runs all tests in debug, including this benchmark; it has passed some CI runs and failed others depending on memory-measurement timing.
## Why it matters
1. **A perpetually-/intermittently-red `test` job hides real regressions.** This (together with a now-fixed stale unwrap test) kept the `test` check red on `main` from 2026-03-20, which masked other failures.
2. **It may indicate a real memory inefficiency** — a streaming encryptor processing 2 MB chunks of a 50 MB file should not need to hold ~213 MB resident. Worth confirming whether `TDFEncryptor.encryptFile` is buffering more than it should (e.g. holding the whole input/output in memory) rather than truly streaming.
## Proposed remediation (pick per investigation)
- [ ] Investigate `TDFEncryptor.encryptFile` peak memory for chunked input. If it's genuinely buffering the whole file, fix the streaming path (the real win). If ~213 MB is expected/acceptable, recalibrate the assertion to a realistic, stable bound.
- [ ] Move the benchmark suites (`*BenchmarkTests`) out of the default `test` job so `test` is deterministic — they already have a dedicated `benchmark` job (release). Memory/timing assertions don't belong in the correctness gate.
## Repro
```bash
swift test --filter "StreamingBenchmarkTests/testMemoryUsageWith2MBChunk"
swift test -c release --filter "StreamingBenchmarkTests/testMemoryUsageWith2MBChunk"
```
_Surfaced while getting CI green for #39._
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with OpenTDFKitTests/StreamingBenchmarkTests.swift:156 and reproduce the failure with the two swift test commands in the issue. Inspect TDFEncryptor.encryptFile to determine whether the peak reflects unintended buffering; done means either correcting the streaming path, recalibrating the bound if the usage is expected, or moving benchmark suites out of the default test job.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- swift
- Domain
- performance, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100