[Bug] Hidden dangers in lookup function

### Search before asking

- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.

### Paimon version

Paimon: 0.8
Flink: 1.16

### Compute Engine

Flink

### Minimal reproduce step

SecondaryIndexLookupTable ----- RocksDBSetState.java ----- get(K key) ------ List valueBytes = cache.getIfPresent(keyBytes);
'lookup.cache' = 'full', 'lookup.cache-rows'='20'
table.exec.state.ttl = 5 min
with massive data

### What doesn't meet your expectations?

In my case, joinKey <> PrimaryKey.
Looking at the source code in **RocksDBSetState.java ----- get(K key)**, it will query the **db** (with the condition valueBytes==null) when the cache cannot read the data .
There are some hidden dangers in the conditions for querying db. After logging, I found that there may be a situation where **valueBytes is not null but the list is empty**, which will result in no longer querying db. In some extreme cases, it seems that there is data in the db, but due to this query condition, the indexState returns an empty mapping, indirectly leading to lookup failure.
Can u further test and change the condition for querying db to "if (value Bytes!=null&&value Bytes. isEmpty())"?

----------------------------------------------------------------------

I have another question. When two identical tasks are running at the same time, one task(task 1) can successfully read the lookup table data, but the other cannot(task 2). In task2, through logging, I force IndexState to read the cache and db, but they return empty data.

Then I found that the data read by the two tasks during the last lookup was different.
Task1
2024-04-23 01:27:47.347 INFO Reading ORC rows from data-ed87638b-834b-46a2-a192-37130bd74f8c-0.orc with {include: null, offset: 3, length: 1159, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 01:27:47.555 INFO Reading ORC rows from data-ed87638b-834b-46a2-a192-37130bd74f8c-1.orc with {include: null, offset: 3, length: 889, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 01:27:47.778 INFO Reading ORC rows from data-ed87638b-834b-46a2-a192-37130bd74f8c-2.orc with {include: null, offset: 3, length: 1052, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}

**2024-04-23 02:02:28.815 INFO Reading ORC rows from data-75ec10a0-651d-42a1-87f2-8a7a0df66652-0.orc with {include: null, offset: 3, length: 1015, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}**
2024-04-23 02:02:28.965 INFO Reading ORC rows from data-75ec10a0-651d-42a1-87f2-8a7a0df66652-1.orc with {include: null, offset: 3, length: 887, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 02:02:29.586 INFO Reading ORC rows from data-75ec10a0-651d-42a1-87f2-8a7a0df66652-2.orc with {include: null, offset: 3, length: 1181, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}

Task2
2024-04-23 02:02:33.165 INFO Reading ORC rows from data-ed87638b-834b-46a2-a192-37130bd74f8c-0.orc with {include: null, offset: 3, length: 1159, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 02:02:33.326 INFO Reading ORC rows from data-ed87638b-834b-46a2-a192-37130bd74f8c-1.orc with {include: null, offset: 3, length: 889, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 02:02:34.201 INFO Reading ORC rows from data-ed87638b-834b-46a2-a192-37130bd74f8c-2.orc with {include: null, offset: 3, length: 1052, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}

**Read records after the last lookup**
2024-04-23 02:03:56.862 INFO Reading ORC rows from data-75ec10a0-651d-42a1-87f2-8a7a0df66652-0.orc with {**include: null, offset: 3, length: 1015,** schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 02:03:56.978 INFO Reading ORC rows from data-75ec10a0-651d-42a1-87f2-8a7a0df66652-1.orc with {**include: null, offset: 3, length: 887**, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}
2024-04-23 02:03:57.973 INFO Reading ORC rows from data-75ec10a0-651d-42a1-87f2-8a7a0df66652-2.orc with {**include: null, offset: 3, length: 1181**, schema: struct<_SEQUENCE_NUMBER:bigint,_VALUE_KIND:tinyint,xxx>, includeAcidColumns: true, allowSARGToFilter: false, useSelected: false}

It can be visually seen that in task2 where lookup failed, a portion of the data was read after the last lookup. That is to say, the data read by task1 will eventually be read by task2, but the time nodes they read are different. What is the reason for this differentiation? Can u optimize it? I personally believe that the issue occurred during the refresh function.

### Anything else?

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Start at SecondaryIndexLookupTable and RocksDBSetState.java, especially get(K key), using the reported lookup.cache settings and table.exec.state.ttl configuration. Reproduce the empty-list cache case with massive data, then inspect the refresh function alongside the ORC read timing from both tasks. Done means the lookup does not miss existing index data and the differing refresh behavior is explained or corrected with regression coverage.

Written by the indexing model from the issue text.