Allow arbitrary "dedup unit" sizes in RGBGFX

This would be useful for "8×16 OAM mode", mainly, but also metatile setups that require four unique tile IDs (usually those that don't go through an extra indirection layer).

Start by reading RGBGFX's current deduplication-unit handling and how it relates to 8×16 OAM mode. Clarify the expected arbitrary-size behavior for metatile setups requiring four unique tile IDs, then verify that both stated use cases work without breaking existing behavior.

Written by the indexing model from the issue text.