BigQuery: Copy SELECT / latest-partition WHERE uses PARSE_DATE('%Y%m%d', '__NULL__') and the query fails
- Dominant language
- Python
- Stars
- 74.8k
- Forks
- 18.3k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 685
Description
### Bug description
When using a BigQuery time-partitioned table in SQL Lab, Copy SELECT statement to the clipboard (and table preview selectStar) automatically appends a latest-partition predicate. If the table has a __NULL__ partition (rows where the partition column is NULL), the generated SQL is invalid:
WHERE 'date' = PARSE_DATE('%Y%m%d', '__NULL__')
PARSE_DATE('%Y%m%d', '__NULL__') fails because __NULL__ is not a YYYYMMDD date. The same can happen with __UNPARTITIONED__.
**How to reproduce**
1. Connect a BigQuery database.
2. Open SQL Lab and select a table that is column-partitioned by a DATE field (e.g. date).
3. Ensure the table has at least one row where the partition column is NULL (BigQuery then has a __NULL__ partition in INFORMATION_SCHEMA.PARTITIONS).
4. In the table tree, click Copy SELECT statement to the clipboard.
5. Paste and run the SQL.
**Expected behavior**
The generated SELECT should filter on the latest real time partition (numeric YYYY / YYYYMM / YYYYMMDD / YYYYMMDDHH), not on BigQuery’s special partition IDs (__NULL__, __UNPARTITIONED__).
If there is no usable time partition, the extra WHERE should be omitted rather than emitting an invalid PARSE_DATE.
### Screenshots/recordings
### Superset version
6.1.0
### Python version
3.12
### Node version
18 or greater
### Browser
Chrome
### Additional context
Additional context
Database: Google BigQuery.
Table is time-unit column-partitioned (not ingestion-time). Partition field is a DATE column named date. The table contains NULL values in that column, so INFORMATION_SCHEMA.PARTITIONS includes partition_id = '__NULL__'.
This affects SQL Lab Copy SELECT statement and any other path that uses select_star(..., latest_partition=True) for BigQuery (table preview / sample).
### Checklist
- [x] I have searched Superset docs and Slack and didn't find a solution to my problem.
- [x] I have searched the GitHub issue tracker and didn't find a similar bug report.
- [ ] I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.
Contributor guide
Research direction
Start at the BigQuery select_star(..., latest_partition=True) entry point used by SQL Lab Copy SELECT and table preview. Reproduce the case with __NULL__ and __UNPARTITIONED__ partition IDs, then verify that only a usable latest real partition produces a predicate and that no usable partition omits the extra WHERE.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100