Azure / Azure/Azure-DataFactory

File reference is created when converting from CSV to Parquet

Open
#237 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
529
Forks
623
PR merge metrics
No merged PRs in 30d

Description

I have a dataflow that converts a csv file to parquet.

Here is my flow:
![image](https://user-images.githubusercontent.com/10215464/101823433-f65bcd80-3ae7-11eb-97bc-9e371c80fff6.png)

The issue appears to be in the sink. I want the file to be added in a folder that is in a container so I have this in my dataset:
![image](https://user-images.githubusercontent.com/10215464/101823502-17242300-3ae8-11eb-972d-7e412f6a6289.png)

When I run this, the folder is being created and the parquet file is being added to the folder but then outside of the folder is a random file that is being created with 0 bytes and has the same name as the folder path shown in the dataset. This file has no content type and is essentially blank. When I download it it's actually the folder referenced in my dataset.

This also happens when I select to move my source file to a new folder:
![image](https://user-images.githubusercontent.com/10215464/101832018-0083c900-3af4-11eb-9c5e-9e3ecb078dec.png)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.