polars collect does not print all values of `cat` type columns
Open
Nobody has claimed this yet.
A:plugin-polars
status:needs-triage
- Dominant language
- Rust
- Stars
- 40.5k
- Forks
- 2.3k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 79
Description
Describe the bug
When displaying a Parquet file using polars collect, certain columns display blank values. NOTE, I believe the issue is related to the fact the column types are cat or category:
| e2e_instance_id | series_id | f_string | f_int | f_uint | f_float | f_bool | time |
|---|---|---|---|---|---|---|---|
| cat | cat | str | i64 | u64 | f64 | bool | datetime |
How to reproduce
- Display the contents of the Parquet file using
polars open 0000000002.parquet | polars collect
Note the output has blank values:
╭────┬──────────────────────────────────────┬────────────────────┬──────────┬───────┬────────┬─────────┬────────┬────────────────╮
│ # │ e2e_instance_id │ series_id │ f_string │ f_int │ f_uint │ f_float │ f_bool │ time │
├────┼──────────────────────────────────────┼────────────────────┼──────────┼───────┼────────┼─────────┼────────┼────────────────┤
│ 0 │ 44c37767-f5cf-490f-87ce-56a27913b2b9 │ series-number-0001 │ off │ 5 │ 5 │ 0.30 │ true │ 14 minutes ago │
│ 1 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-0010 │ on │ 4 │ 4 │ 0.20 │ true │ 14 minutes ago │
│ 2 │ │ series-number-0100 │ off │ 8 │ 8 │ 0.90 │ false │ 14 minutes ago │
│ 3 │ │ series-number-1000 │ on │ 3 │ 3 │ 2.90 │ true │ 14 minutes ago │
│ 4 │ │ │ off │ 3 │ 3 │ 1.90 │ true │ 14 minutes ago │
│ 5 │ │ │ off │ 5 │ 5 │ 0.30 │ true │ 14 minutes ago │
│ 6 │ │ │ on │ 3 │ 3 │ 0.30 │ true │ 14 minutes ago │
│ 7 │ │ │ on │ 4 │ 4 │ 0.20 │ true │ 14 minutes ago │
│ 8 │ │ │ off │ 1 │ 1 │ 2.20 │ false │ 14 minutes ago │
│ 9 │ │ │ off │ 8 │ 8 │ 0.90 │ false │ 14 minutes ago │
│ 10 │ │ │ off │ 3 │ 3 │ 1.90 │ true │ 14 minutes ago │
│ 11 │ │ │ off │ 2 │ 2 │ 2.90 │ true │ 14 minutes ago │
╰────┴──────────────────────────────────────┴────────────────────┴──────────┴───────┴────────┴─────────┴────────┴────────────────╯
If I use duckdb to read the content, we see the values in those columns:
duckdb -csv -c "select * from '/Volumes/RAMDisk/influxdb3/ingest/dbs/dev-2/alt-0/2025-05-22/00-25/0000000002.parquet'" | from csv
Output
╭────┬──────────────────────────────────────┬────────────────────┬──────────┬───────┬────────┬─────────┬────────┬─────────────────────╮
│ # │ e2e_instance_id │ series_id │ f_string │ f_int │ f_uint │ f_float │ f_bool │ time │
├────┼──────────────────────────────────────┼────────────────────┼──────────┼───────┼────────┼─────────┼────────┼─────────────────────┤
│ 0 │ 44c37767-f5cf-490f-87ce-56a27913b2b9 │ series-number-0001 │ off │ 5 │ 5 │ 0.30 │ true │ 2025-05-22 00:25:09 │
│ 1 │ 44c37767-f5cf-490f-87ce-56a27913b2b9 │ series-number-0010 │ on │ 4 │ 4 │ 0.20 │ true │ 2025-05-22 00:25:09 │
│ 2 │ 44c37767-f5cf-490f-87ce-56a27913b2b9 │ series-number-0100 │ off │ 8 │ 8 │ 0.90 │ false │ 2025-05-22 00:25:09 │
│ 3 │ 44c37767-f5cf-490f-87ce-56a27913b2b9 │ series-number-0100 │ on │ 3 │ 3 │ 2.90 │ true │ 2025-05-22 00:25:11 │
│ 4 │ 44c37767-f5cf-490f-87ce-56a27913b2b9 │ series-number-1000 │ off │ 3 │ 3 │ 1.90 │ true │ 2025-05-22 00:25:09 │
│ 5 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-0001 │ off │ 5 │ 5 │ 0.30 │ true │ 2025-05-22 00:25:09 │
│ 6 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-0001 │ on │ 3 │ 3 │ 0.30 │ true │ 2025-05-22 00:25:10 │
│ 7 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-0010 │ on │ 4 │ 4 │ 0.20 │ true │ 2025-05-22 00:25:09 │
│ 8 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-0010 │ off │ 1 │ 1 │ 2.20 │ false │ 2025-05-22 00:25:10 │
│ 9 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-0100 │ off │ 8 │ 8 │ 0.90 │ false │ 2025-05-22 00:25:09 │
│ 10 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-1000 │ off │ 3 │ 3 │ 1.90 │ true │ 2025-05-22 00:25:09 │
│ 11 │ dd7221f9-0b30-4e28-8672-412c5a0c883d │ series-number-1000 │ off │ 2 │ 2 │ 2.90 │ true │ 2025-05-22 00:25:11 │
╰────┴──────────────────────────────────────┴────────────────────┴──────────┴───────┴────────┴─────────┴────────┴─────────────────────╯
Expected behavior
I expected nu to display all the values of the e2e_instance_id and series_id columns.
Configuration
| key | value |
|---|---|
| version | 0.104.0 |
| major | 0 |
| minor | 104 |
| patch | 0 |
| branch | |
| commit_hash | |
| build_os | macos-aarch64 |
| build_target | aarch64-apple-darwin |
| rust_version | rustc 1.86.0 (05f9846f8 2025-03-31) (Homebrew) |
| cargo_version | cargo 1.86.0 |
| build_time | 2025-04-29 23:31:45 +00:00 |
| build_rust_channel | release |
| allocator | standard |
| features | default, sqlite, trash |
| installed_plugins | formats 0.104.0, polars 0.104.0 |
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with polars open 0000000002.parquet | polars collect, using the attached Parquet file, and compare the categorical columns with the shown DuckDB output. Trace the polars collect display path; done means all e2e_instance_id and series_id values are shown rather than blank.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust, shell
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100