Clustering failed are caused by incompatible types in the not null column when written by spark and flink

### Bug Description

**What happened:**
when I use spark create a hudi table, the table some column（eg recordkey col） with not null property, then, I use spark sink data to this table for history initial, latestly, I use flink streaming sink realtime data to this table with insert operation and inline clustering，then，I encountered an exception when clustering.
**What you expected:**

after create table，Spark and Flink maintain a consistent schema when writing to the same table.

**Steps to reproduce:**
1.use sparksql create a hudi table with hudi_test_id not null

```sql
CREATE TABLE hudi_table (
ts BIGINT,
hudi_test_id BIGINT not null,
rider STRING,
driver STRING,
fare DOUBLE,
city STRING
) USING HUDI
PARTITIONED BY (city);
```

2.use sparksql insert into data to above table
3.use flinksql insert into data to above table with insert write operation and inline async clustering

### Environment

Hudi version: hudi-0.1.41
Sink engine: spark-3.4.2,flink-1.16.2
Relevant configs: hive sync strategy RT

### Logs and Stack Trace

```java
2025-12-30 14:24:29,788 INFO org.apache.parquet.hadoop.InternalParquetReader [] - block read in memory in 1 ms .row count = 107
2025-12-30 14:24:29,788 ERROR org.apache.hudi.sink.clustering.ClusteringOperator [] - Executor action [Execute clustering for instant 20251230140224328 from task 0] error
org.apache.hudi.exception.HoodieEception: unable to read next record from parquet file
at org.apache.hudi.common.util.ParquetIterator.hasNext(ParquetIterator.java:54)~[hudi-flink1.16-bundle-0.14.1.jar]
...
at org.apache.hudi.client.utils.ConcateingOperator.hasNext(ConcateingOperator.java:251)~[hudi-flink1.16-bundle-0.14.1.jar]~[hudi-flink1.16-bundle-0.14.1.jar]
at org.apache.hudi.sink.clustering.ClusteringOperator.doClustering(ClusteringOperator.java:251)~[hudi-flink1.16-bundle-0.14.1.jar]~[hudi-flink1.16-bundle-0.14.1.jar]
Caused by: org.apache.parquet.io.ParquetDecodingException:Can not read value at 0 in block -1 in file hdfs://nsfed//hudi/rtcp_001/qrt_app_rtcp_001_test/hudi_table/dt=20251230/00000003-3f63-4106-9b82-952240e7f8cc-0_3-10-13_20251230111845024.parquet
at org.apache.parquet.hadoop.InternalParquetReader.nextKeyValue(InternalParquetReader.java:254)~[hudi-flink1.16-bundle-0.14.1.jar]~[hudi-flink1.16-bundle-0.14.1.jar]
...
Caused by: org.apache.parquet.io.ParquetDecodingException: The requested schema is not compatible with the file schema. incompatible types: required int64 hudi_test_id != optional int64 hudi_test_id
at org.apache.parquet.io.ColumnIOFactory$ColumnIOCreatorVistor.incompatibleSchema(ColumnIOFactory.java:101)~[hudi-flink1.16-bundle-0.14.1.jar]~[hudi-flink1.16-bundle-0.14.1.jar]
...
```

Reproduce the Spark SQL create and insert followed by Flink SQL insert with inline async clustering, then inspect the stack trace through ParquetIterator.java and ClusteringOperator.java. Compare the required-versus-optional hudi_test_id schemas shown in the Parquet error; done means Spark and Flink write a consistent schema and clustering completes successfully.

Written by the indexing model from the issue text.