Benchmark Subqueries

# Subquery Performance Benchmarks

Create system-level benchmarks for subqueries to assess performance, scalability, and memory usage.

## Context

Subqueries are implemented using in-memory Materializer processes that maintain indexes of subquery results and trigger move-in/move-out events when data changes. We need benchmarks that run the full system (sync service + Postgres) and measure real-world performance characteristics.

## Memory Locations to Monitor

### Consumer Process State (`Consumer.State`)

| Field | What It Stores | Memory Impact |
|-------|----------------|---------------|
| `buffer` | Transactions queued while buffering (reverse order) | O(buffered_txns × txn_size) - can grow unbounded during initial snapshot or move-in waits |
| `txn_offset_mapping` | List of `{shape_offset, txn_boundary}` tuples for flush alignment | O(unflushed_txns × 32 bytes) |
| `writer` | Storage writer state (ETS refs, file handles, buffers) | Varies by storage backend |
| `shape` | Full Shape struct including `shape_dependencies` | O(query_complexity + nested_shapes) |
| `transaction_builder` | Partial transaction fragments being assembled | O(fragment_size) - transient |

### MoveIns State (`Consumer.MoveIns`)

| Field | What It Stores | Memory Impact |
|-------|----------------|---------------|
| `waiting_move_ins` | Map of `name → {pg_snapshot, {ref_key, MapSet[moved_values]}}` | O(concurrent_move_ins × values_per_move_in) |
| `filtering_move_ins` | List of `{pg_snapshot, MapSet[keys]}` for completed but filtering move-ins | O(filtering_move_ins × keys_per_move_in) - can be large if many rows moved in |
| `touch_tracker` | Map of `key → xid` tracking which keys were touched | O(touched_keys) - grows with change volume, GC'd periodically |
| `in_flight_values` | Precalculated map of all moved-in values to skip in WHERE evaluation | O(total_in_flight_values) |
| `moved_out_tags` | Map of `move_in_name → MapSet[tags]` for move-outs during move-in | O(concurrent_move_ins × moved_out_tags) |
| `move_in_buffering_snapshot` | Union snapshot `{xmin, xmax, xip_list}` | ~100 bytes (but `xip_list` can grow) |
| `maximum_resolved_snapshot` | Snapshot for visibility boundary | ~100 bytes |
| `minimum_unresolved_snapshot` | Snapshot for visibility boundary | ~100 bytes |

### InitialSnapshot State (`Consumer.InitialSnapshot`)

| Field | What It Stores | Memory Impact |
|-------|----------------|---------------|
| `pg_snapshot` | Tuple `{xmin, xmax, xip_list}` | O(in_progress_txns) - `xip_list` can be large under high concurrency |
| `awaiting_snapshot_start` | List of `GenServer.from()` references | O(waiting_clients) |

### Materializer Process State (`Consumer.Materializer`)

| Field | What It Stores | Memory Impact |
|-------|----------------|---------------|
| `index` | All rows in subquery result: `key → value` | O(rows × row_size) - **primary memory consumer** |
| `tag_indices` | Reverse index: `tag_hash → MapSet[keys]` | O(unique_tags × keys_per_tag) |
| `value_counts` | Reference counting: `value → count` | O(unique_values × 16 bytes) |
| `subscribers` | MapSet of subscribed PIDs | O(subscriber_count) |

---

## Proposed Benchmarks

### 1. Concurrent Shape Creation

**Measure:**
- Time to create all shapes
- Memory usage

**Vary:**
- Number of shapes
- Number of subqueries per shape
- Size of subquery result set
- Subquery nesting depth
- DB latency (higher latency increases buffer size)
- Composite vs single-column key

### 2. Replication Throughput

**Measure:**
- Latency from insert to client receipt
- Memory usage

**Vary:**
- Number of shapes
- Number of transactions
- Number of rows per transaction
- Number of subqueries per shape
- Size of subquery result set
- Composite vs single-column key

### 3. Move-In/Move-Out

**Measure:**
- Latency from insert/delete to client receipt
- Memory usage (peak and steady-state)
- Connection pool utilisation
- GC pause times
- Message queue lengths

**Vary:**
- Move-in vs move-out
- Number of subqueries per shape
- Number of shapes
- Batch size (rows affected per move-in/out)
- DB latency (higher latency increases buffer size)
- Subquery nesting depth
- Composite vs single-column key