apache / apache/arrow-rs

Logging Support

Open
#1,495 0 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 16h
Merged PRs (30d)
168

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

Currently `arrow` and `parquet` do not have any log instrumentation, and I have frequently found myself adding logging to track down a problem, only to then have to remove this logging for the final PR.

**Describe the solution you'd like**

This [log](https://docs.rs/log/latest/log/) crate is well-supported within the Rust ecosystem and allows filtering out callsites at both runtime and compile time. It is also used downstream by [DataFusion](https://github.com/apache/arrow-datafusion/blob/master/datafusion/Cargo.toml#L70). This would allow adding instrumentation to the code that can be enabled for debugging a problem, and then disabled when no longer desired.

**Describe alternatives you've considered**

There is also the [tracing](https://docs.rs/tracing/latest/tracing/) ecosystem, but this is likely significant overkill given neither crate makes significant use of tokio. Fortunately [tracing-log](https://docs.rs/tracing-log/latest/tracing_log/) provides an adapter should a downstream wish to use tokio-tracing, whilst allowing `arrow`, etc... to use the significantly simpler `log` ecosystem.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the arrow and parquet crates and the log crate documentation to understand the proposed integration scope. Determine which components need instrumentation and how logging should be enabled or filtered, then confirm the approach with project maintainers before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.