storage sink: clarify flush interval contract vs per-entry max residency

Problem

The current cloud storage sink implementation does not treat flush-interval as a per-entry maximum residency time.

Today, the sink uses a periodic ticker in the buffering path. On each tick, if there is any pending batch in the shard, it flushes the current pending batch set. This means:

• flush by interval currently means periodic flush scan
• it does not mean: once a specific entry has stayed in the sink for flush-interval, that entry must be flushed immediately

This is easy to misunderstand from the configuration name and from a user perspective.

Current Behavior

For a pending batch in one writer shard:

• flush by size: a single dispatcher + table + version batch reaches file-size, then that table batch is flushed
• flush by interval: a ticker fires every flush-interval, and the shard flushes whatever pending batches exist at that time
• flush by barrier: only the target dispatcher is flushed before the marker continues

Because interval flush is ticker-driven, the oldest pending entry may stay less than one full interval, or slightly more, depending on when it arrived relative to the ticker phase and scheduling.

Why This Matters

If users interpret flush-interval as a maximum per-entry waiting time, the current implementation does not provide that contract.

Also, the current ticker-based model may miss an opportunity to build larger batches. If interval flush were driven by the oldest pending entry deadline instead of a fixed ticker phase, the sink could both:

• provide a clearer residency-time contract
• often accumulate larger batches before flushing

Suggested Follow-up

Decide and document the intended contract for flush-interval:

1. Keep the current contract and document it explicitly as a periodic shard-level flush scan.
2. Change the implementation so flush-interval means: once the oldest pending entry reaches its deadline, the sink must flush.

If option 2 is chosen, the follow-up design should clarify:

• flush granularity at deadline: shard-wide vs dispatcher-wide vs table-wide
• interaction with file-size flush
• interaction with barrier flush
• how to schedule the next wake-up from the oldest pending entry instead of a fixed ticker

Scope

This issue is only about clarifying and possibly redefining the interval-flush contract in cloud storage sink buffering. It does not propose behavior changes in this PR.

Inspect the cloud storage sink buffering path and its periodic ticker behavior. Decide whether flush-interval remains a periodic shard-level scan or becomes an oldest-entry deadline, then define its interaction with file-size and barrier flushes; the work is done when the contract and scheduling behavior are documented or agreed for implementation.