apache / apache/arrow

[R] printing data in Table/RecordBatch print method

Open
#32,110 1 comment 0 reactions 0 assignees View on GitHub
Component: Python Component: R Status: needs champion Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Related to ARROW-16776 but after a brief discussion with Neal Richardson, he requested that I split the improvement request into separate issues.

When working with Arrow datasets/tables, I often find myself wanting to interactively print or "see" the results of a query or the first few rows of the data without having to fully collect into memory.

It would be ideal to lazily print some data with Table/RecordBatch print methods, however, currently, the print methods return schema without data.

IE:

``` r
library(dplyr)
library(arrow)

mtcars %>% arrow::write_parquet("mtcars.parquet")
car_ds <- arrow::open_dataset("mtcars.parquet")

car_ds
#> FileSystemDataset with 1 Parquet file
#> mpg: double
#> cyl: double
#> disp: double
#> hp: double
#> drat: double
#> wt: double
#> qsec: double
#> vs: double
#> am: double
#> gear: double
#> carb: double
#>
#> See $metadata for additional Schema metadata

car_ds %>%
compute()
#> Table
#> 32 rows x 11 columns
#> $mpg
#> $cyl
#> $disp
#> $hp
#> $drat
#> $wt
#> $qsec
#> $vs
#> $am
#> $gear
#> $carb
#>
#> See $metadata for additional Schema metadata
```

**Reporter**: [Thomas Mock](https://issues.apache.org/jira/browse/ARROW-16777)
#### Related issues:
- [[R] dplyr::glimpse method for arrow table and datasets](https://github.com/apache/arrow/issues/32109) (Blocked)
- [[R] Improved print methods for Arrow tabulars](https://github.com/apache/arrow/issues/30237) (is duplicated by)

**Note**: *This issue was originally created as [ARROW-16777](https://issues.apache.org/jira/browse/ARROW-16777). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with the R Table/RecordBatch print methods and reproduce the mtcars example using open_dataset() and compute(). Review the related print-method and glimpse issues for existing scope, then define done as printing a bounded preview of data lazily without fully collecting it into memory.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.