In a Query-Only Spark Session, the Latest Visible Commit Is Not Updated

In the current version, HoodieFileIndex is a member variable of HoodieBaseRelation. PR #7871 has made Hudi's acquisition of Relation behave more like Spark's. However, in Spark, the relation is cached as follows:
catalog.getCachedPlan(qualifiedTableName, () => \{
  val dataSource = DataSource(
    sparkSession,
    userSpecifiedSchema = if (table.schema.isEmpty) None else Some(table.schema),
    partitionColumns = table.partitionColumnNames,
    bucketSpec = table.bucketSpec,
    className = table.provider.get,
    options = dsOptions,
    catalogTable = Some(table)
  )
  LogicalRelation(dataSource.resolveRelation(checkFilesExist = false), table)
})
This results in the continuous use of the same HoodieFileIndex instance.

However, HoodieFileIndex contains cached items like cachedAllPartitionPaths and cachedAllInputFileSlices, which only reset upon creating a new HoodieFileIndex. A sparkSession will only execute refreshTable when actions like 'insert' are performed. If HoodieFileIndex is never refreshed, then a SparkSession that only executes queries will always see the version that was cached during the initial query. This is not the expected behavior. In practice, Delta Lake seems to attempt updating the snapshot with each listFiles operation. After PR #7871, Hudi would recreate the relation with each query, obtaining the latest snapshot. Therefore, I believe there should be an assessment at the start of listFiles to determine whether the cache (snapshot) needs to be updated.

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-7264
- Type: Improvement

Start by tracing HoodieFileIndex.listFiles and the cachedAllPartitionPaths and cachedAllInputFileSlices state, then compare how HoodieBaseRelation is reused with Spark's cached relation behavior described in the issue. Reproduce a query-only SparkSession with a newly visible commit and verify that listFiles observes the latest visible version without an insert or refreshTable action.

Written by the indexing model from the issue text.