[BUG]: "Exists" does not work on Synapse
- Dominant language
- C#
- Stars
- 2.1k
- Forks
- 332
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 9
Description
**Describe the bug**
We are trying to use the new Exists feature in .NET for Apache Spark 2.0 on Azure Synapse.
Unfortunately, the corresponding function returns "False" for every path, regardless of whether it points to a valid file or not.
**To Reproduce**
In our tests, we have the file directly in Synapse primary data lake.
The file can be read without any problems using:
spark.Read().Schema(schema).Parquet(sourcepath)
We use the following code to check if the file exists or not:
FileSystem fs = FileSystem.Get(spark.SparkContext.HadoopConfiguration());
bool fileExists = fs.Exists(sourcepath);
We are using Apache Spark Version 3.1 with .NET for Apache Spark 2.0.0.
**Expected behavior**
If the path points to an existing file, the function should return "True".
Contributor guide
Assessment
This issue has not been assessed yet.