Azure / Azure/Azure-DataFactory

storing integers as a string, then concatenating in file name causes azure blob write error "... failed because container does not exist"

Open
#378 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
529
Forks
623
PR merge metrics
No merged PRs in 30d

Description

I believe there is something wrong in some situations like concat() & pipeline parameters when you use string parameters that only contains integers, ex '202012'.

The work around is store integer parameters as integers and use a
@toString(yourIntParam) when you want to convert it to a string, but there should be some sort of warning or error handling that lets the user know its a problem

Error Message:
"Job failed due to reason: Data preview, debug, and pipeline data flow execution failed because container does not exist"

1. I create a pipeline parameter of type string with key name reportYearMonthStr
2. I store only integers in the reportYearMonthStr parameter value: '202012'
3. I concatenate the reportYearMonthStr parameter into _another_ string which i pass to a data flow
Data Flow Parameter: FileName type: String
Value: (Pipeline expression) @concat('FileName',**pipeline().parameters.reportYearMonthStr**,'.csv')

Note, at this point i'm not even referencing the FileName data flow parameter in the actual data flow. It exists but the data flow isn't using it. The pipeline execution still fails and I get error:
- Error Message:
- "Job failed due to reason: Data preview, debug, and pipeline data flow execution failed because container does not exist"

IF, in the FileName parameter, I replace the reportYearMonthStr reference with its actual value, **'202012'**, the job completes successfully.
@concat('FileName',**'202012'**,'.csv')

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.