dotnet / dotnet/spark

Q: How to handle exceptions?

Open
#1,134 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
2.1k
Forks
332
Avg merge
1d 20h
Merged PRs (30d)
9

Description

I'm facing a scenario where reading files fails, because certain folders within my source (data lake) are empty. The following code (snippet) will then fail:

```
foreach(var repoFolder in repoFolders)
{
var inputDf = spark
.Read()
.Option("multiline", "true")
.Json($"{cdsInputRootUri}/{snapshotId}/{repoFolder}/*/*.json");
...
```
In Python I could `try` but using a C# `try-catch` around my code won't work. What's the recommended way of converting such code to DotNet Spark?

```
for add in addrs:
try:
spark.read.format("parquet").load(add)
except:
print(add)
addrs.remove(add)
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.