dotnet / dotnet/spark

[FEATURE REQUEST]: Support for unit testing using `Show()` in Dataframe

Open
#1,094 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

**Is your feature request related to a problem? Please describe.**
When we write unit test and set breakpoints, one cannot debug the DataFrame in Quick Watch window because the `Console.WriteLine` will not work. Hence, having trouble doing Test Driven Development and quick prototyping.

**Describe the solution you'd like**
Could you add/replace/support `Console.WriteLine` in `Microsoft.Spark.Sql.DataFrame.Show()` to `System.Diagnostics.Debug.WriteLine` so it works in the immediate window. Then I can write `df.Show()` in the code or in the Quick Watch and it will work.

**Describe alternatives you've considered**
I have to use `Collect()` which is difficult to interpret and hangs in the Quick Watch

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.