apache / apache/datafusion

Move `tempfile` crate to dev dependencies in datafusion core crate

Open
#17,522 1 comment 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

`tempfile` crate is currently included as a normal dependency in the `datafusion` crate. It would be possible to move the dependency to the `dev-dependencies` since the `tempfile` capabilities are only used by integration and unit tests.

Specifically, the following function:

```
/// Generate CSV partitions within the supplied directory
pub fn populate_csv_partitions(
tmp_dir: &TempDir,
partition_count: usize,
file_extension: &str,
) -> Result {
...
```
It is the only one that uses `tempfile` capabilities.

### Describe the solution you'd like

It would be possible to move the `tempfile` crate from `dependencies` to `dev-dependencies` by decorating the `populate_csv_partitions` with the `#[cfg(test)]` attribute and replicating the function where needed in the integration tests (i.e.: `datafusion/core/tests/dataframe/mod.rs`)

### Describe alternatives you've considered

_No response_

### Additional context

I'm opening this as a feature request since it's not an actual bug.

Contributor guide

Open the contributing guide

Research direction

Locate the datafusion core crate's dependency declarations and all uses of populate_csv_partitions, then inspect datafusion/core/tests/dataframe/mod.rs for the integration-test helper. Confirm that tempfile is only needed by tests, move it to the test dependency set, and ensure the unit and integration tests still compile and pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
build-system, testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.