Active, background codecs shouldn't be reclaimed.

While I have no problem with the user agent reclaiming inactive codecs, I have some significant misgivings about active video decoders being reclaimed from background tabs.

The idea of reclaiming video decoders from inactive tabs seems to follow the current behaviour for <video>. It makes sense to reclaim decoders used by <video> because, by their nature, these render to the screen - if the page isn't visible, why keep decoding? However, that's not necessarily true in the WebCodec case. One of the nice things about this API is that it unlocks a variety of new use cases, many of which we haven't thought of, that no longer rely on the traditional fetch-decode-render pipeline. It seems a shame, therefore, to enforce the same resource-management rules that don't necessarily fit these new workflows.

The argument is that the user can still hear the audio output from an inactive tab, but can't see the video output. This assumes
that the only purpose of decoding video is to render a picture to the page immediately. I can think of a number of uses of a video decoder that don't do this. For example, extracting burned-in timecode, or detecting faces. It's likely a user would switch tabs while waiting for the video to be processed and then wonder why their processing has failed or stalled. A more contrived example might be one in which the video is being used to generate sound or avant-garde music. In that case, the audio would stop when the user switches tabs.

I would also argue that a bad actor can overcome the decoder reclamation anyway, by creating a video encoder and continually feeding it with trivial, inconsequential data.

Is there any reason why a background tab can't just have a shorter idle timeout instead?

Start with issue #363 and the described WebCodecs behavior for active video decoders in background tabs. Review the reclamation concern alongside the examples of timecode extraction, face detection, and audio generation. Done means resolving whether active background codecs should use a shorter idle timeout instead of being reclaimed.