Backport V4 publishing changes to release/10.0 to enable 10.0 repos to onboard

To let `release/10.0` repos onboard to Arcade V4 publishing, the V4 publishing changes currently in `main` need to be backported to the `release/10.0` branch.

### Scope
- Identify the set of V4 publishing changes in `main` that need to flow to `release/10.0`.
- Backport them to `release/10.0`.
- Validate that a 10.0 repo can successfully onboard to and publish via V4 after the backport.

### Context
Migrated from personal work-tracking item W135. Related to the V4 publishing rollout tracking issue https://github.com/dotnet/arcade/issues/16697 and the V3 deprecation follow-up https://github.com/dotnet/arcade/issues/17111.

Start by reviewing the V4 publishing changes on main and the release/10.0 branch, using rollout issue #16697 for context. Identify the changes needed for 10.0, backport them, and validate that a 10.0 repository can onboard and publish through V4 successfully.

Written by the indexing model from the issue text.