Optimize middleware for large datasets

**Description**
Currently core lacks proper compression support.

**Proposed solution**
Add configurable compression with sensible defaults.

**Priority**: Low/Medium

The issue names no files, tests, entry points, compression format, or acceptance criteria. First locate the core middleware and determine where large datasets are handled; before coding, clarify supported compression options and sensible defaults, then verify the chosen behavior with project tests.

Written by the indexing model from the issue text.