Support for storage quota for realtime tables

Storage quota is currently available for offline tables, done during segment upload. For realtime tables we'd like to enable tail deletion of old segments once a table uses more than a set quota. For our use case this is preferable to setting ingestion rate limits as fresh data has more value than historic data. This helps when multiple tables share the same tenant and ingestion rate can be highly unpredictable.

We'd prefer to add this as a non-blocking/asynchronous process. Strict adherence to the quota is not necessarily required and a segment commit should not be blocked if the new segment would go over the configured quota. Instead, we'd prefer a lazy tail deletion of old segments when the quota is breached in order to not impact segment commit duration/p99 freshness.

I did not find similar issues open and would be open to adding this.

No files or tests are named. Start by locating the existing offline-table quota handling during segment upload and the realtime segment lifecycle; done means realtime tables asynchronously delete old tail segments after the configured quota is exceeded without blocking segment commits.

Written by the indexing model from the issue text.