Iceberg Spark 4.2 feature tracking

### Feature Request / Improvement

Spark 4.2 adds a number of features and optimizations that may benefit the Iceberg Spark connector. This issue tracks the corresponding Iceberg work. If you are interested in an item, please open a dedicated sub-issue and link it back here.

- Partition pruning ([SPARK-55596](https://issues.apache.org/jira/browse/SPARK-55596)) @szehon-ho
- DSv2 Transaction API ([SPARK-55855](https://issues.apache.org/jira/browse/SPARK-55855), [API foundations](https://github.com/apache/spark/pull/54642), [transaction management](https://github.com/apache/spark/pull/55278))
- DSv2 table pinning and refresh semantics ([SPARK-58389](https://issues.apache.org/jira/browse/SPARK-58389), [spark#57799](https://github.com/apache/spark/pull/57799), [spark#53109](https://github.com/apache/spark/pull/53109))
- INSERT WITH SCHEMA EVOLUTION ([syntax](https://github.com/apache/spark/pull/53732), [DSv2 implementation](https://github.com/apache/spark/pull/54488))
- Auto CDC / native CDC integration ([DSv2 CDC API](https://github.com/apache/spark/pull/54738)) @huan233usc
- CREATE TABLE LIKE for V2 ([SPARK-33902](https://issues.apache.org/jira/browse/SPARK-33902), [spark#54809](https://github.com/apache/spark/pull/54809)) @rahulsmahadev
- SupportsSchemaEvolution capability ([SPARK-55689](https://issues.apache.org/jira/browse/SPARK-55689), [spark#54658](https://github.com/apache/spark/pull/54658)) @rahulsmahadev
- Support row group skipping for shredded variant columns (https://github.com/apache/iceberg/pull/16133) @nssalian

Also add tests for existing behavior:

- DSv2 cache invalidation and refresh ([spark#55416](https://github.com/apache/spark/pull/55416), [spark#55536](https://github.com/apache/spark/pull/55536))
- Validate Spark 4.2 behavior to official Iceberg release : @aokolnychyi

### Query engine

Spark

### Willingness to contribute

- [x] I can contribute this improvement/feature independently
- [x] I would be willing to contribute this improvement/feature with guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time

Choose one Spark 4.2 item from the tracking list and read its linked Spark JIRA or pull request first; the issue asks contributors to open a dedicated sub-issue for that work. Done means the selected Iceberg Spark connector change is implemented, its behavior is tested, and the dedicated sub-issue links back here.

Written by the indexing model from the issue text.