Preserving relationship of ancillary data with video frames

Hey, I'm looking at making use of the ancillary data support, but I am a bit perplexed by the fact that ancillary data is an entirely different payload type, to be sent over a separate endpoint.

Many ancillary mapping mechanisms, such as SMPTE 2010 (Mapping of ANSI/SCTE 104 Messages), require a strict relationship between a given video frame and its ancillary data (eg a single SCTE 104 message should be associated to one frame with that scheme).

As far as I understand, requiring ancillary data to be sent and received through a separate endpoint severs that relationship, am I missing something? How do you recommend a receiver re-associates each video frame with its respective ancillary data?

Start with the ancillary data support and separate endpoint behavior described in the AWS CDI SDK documentation linked from the repository. Determine whether the documentation explains how receivers preserve the relationship between video frames and ancillary data; done means the association mechanism or its limitation is clearly documented.

Written by the indexing model from the issue text.