[Bug] Failed to execute delete with an append table by a timestamp condition statement, error "failed to parse timestamp datatype value"
- Dominant language
- Java
- Stars
- 3.4k
- Forks
- 1.4k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 396
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar.
### Paimon version
0.8.2
### Compute Engine
spark 3.4.1
### Minimal reproduce step
1. create a append table in spark-sql
CREATE TABLE paimon.events.event_test (
id varchar(100),
time timestamp,
created_at timestamp,
new_session varchar(600),
source_tmp varchar(6000),
time_dt date
) partitioned by (time);
2. insert some data to the table in spark-sql
insert into paimon.events.event_test values('111', cast('2024-10-01 01:00:00' as timestamp), cast('2024-10-01 01:00:00' as timestamp), '111', '111', cast('2024-10-01' as date));
insert into paimon.events.event_test values('111', cast('2024-10-01 06:00:00' as timestamp), cast('2024-10-02 06:00:00' as timestamp), '111', '111', cast('2024-10-01' as date));
insert into paimon.events.event_test values('111', cast('2024-10-01 12:00:00' as timestamp), cast('2024-10-01 12:00:00' as timestamp), '111', '111', cast('2024-10-01' as date));
insert into paimon.events.event_test values('111', cast('2024-10-01 18:00:00' as timestamp), cast('2024-10-01 18:00:00' as timestamp), '111', '111', cast('2024-10-01' as date));
insert into paimon.events.event_test values('111', cast('2024-10-02 01:00:00' as timestamp), cast('2024-10-02 01:00:00' as timestamp), '111', '111', cast('2024-10-02' as date));
insert into paimon.events.event_test values('111', cast('2024-10-02 06:00:00' as timestamp), cast('2024-10-02 06:00:00' as timestamp), '111', '111', cast('2024-10-02' as date));
insert into paimon.events.event_test values('111', cast('2024-10-02 12:00:00' as timestamp), cast('2024-10-02 12:00:00' as timestamp), '111', '111', cast('2024-10-02' as date));
insert into paimon.events.event_test values('111', cast('2024-10-02 18:00:00' as timestamp), cast('2024-10-02 18:00:00' as timestamp), '111', '111', cast('2024-10-02' as date));
insert into paimon.events.event_test values('111', cast('2024-10-03 01:00:00' as timestamp), cast('2024-10-03 01:00:00' as timestamp), '111', '111', cast('2024-10-03' as date));
insert into paimon.events.event_test values('111', cast('2024-10-03 01:00:01' as timestamp), cast('2024-10-03 01:00:01' as timestamp), '111', '111', cast('2024-10-03' as date));
3. execute select in spark-sql, it would execute successfully
select * from paimon.events.event_test where time = '2024-10-03 01:00:01';
4. execute delete in spark-sql, it would raise an error
delete from paimon.events.event_test where time = '2024-10-03 01:00:01';

### What doesn't meet your expectations?
it could execute select successfully, and it failed to execute delete with the same condition statement.
### Anything else?
I made a test with a primary table, both select and delete with timestamp datatype condition could be executed successfully.
I also made a test with a append table, both select and delete with date datatype condition could be executed successfully.
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure in Spark SQL using the append-table schema and timestamp inserts from the issue, then compare the successful SELECT with the failing DELETE path. Done means DELETE accepts the same timestamp condition on an append table without the parse-timestamp error, while the existing primary-table and date-condition cases continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100