MV duplicate (within a row) handling for forwardIndexDisabled columns

Support for disabling the forward index was added (details can be found in this issue: https://github.com/apache/pinot/issues/6473). As part of our analysis, we found that for MV columns with duplicate entries within a row, regenerating the forward index to include the duplicated entries is not possible today. More details about this issue can be found in [this document](https://docs.google.com/document/d/1MNLLhYCg5e-UFBQ6wTBODd41sDsbjevwRfwoGuNowWw/edit?usp=sharing). To correctly regenerate the forward index for a MV column with duplicates within a row the information about the frequency of duplicated keys per row need to be tracked in an on-disk file. Opening this issue to track adding support for this.

Until this is fixed, MV columns with duplicates will need to be backfilled if the forward index is to be enabled at a later point in time. Or customers need to assess that they do not need the duplicates per row, in which case reload code path will create the forward index without duplicates per row.

cc @siddharthteotia

Start by reading issue 6473 and the linked design document for the forward-index-disabled column behavior. Trace the MV reload and forward-index regeneration paths, then define how duplicate-key frequency per row is persisted on disk and verify that regeneration preserves those duplicates.

Written by the indexing model from the issue text.