microsoft / microsoft/finops-toolkit
config_runbackfilljob fails to run using datafactory studio to run the job
Nobody has claimed this yet.
- Dominant language
- PowerShell
- Stars
- 603
- Forks
- 249
- Avg merge
- 7d 11h
- Merged PRs (30d)
- 11
Description
🐛 Problem
running pipeline config_runbackfilljob in data factory studio fails to run. Set the scope config in settings.json however once you try run the job you get a myriad of errors
One was export time cannot exceed 84 months
One was startdate and enddate must be in the same month and another was
Either both startDate and endDate needs to be present or both should not be present."
Date format used for the above tests was DD/MM/YYYY
On Second round of testing using a single month with date format MM/DD/YYYY it worked for one month only
Using the same MM/DD/YYYY and spanning multiple months it fails, this was a backfill pipeline and has a variable start and end date, you would assume you could put in start date 13 months ago and end date current date to get 13 months of backfill but that is not working.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing config_runbackfilljob from Azure Data Factory Studio using the settings.json scope configuration and the reported DD/MM/YYYY and MM/DD/YYYY date ranges. Compare single-month and multi-month backfills, then define done as allowing a valid multi-month startDate and endDate without the conflicting validation errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, powershell
- Domain
- cloud, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100