microsoft / microsoft/Dynamics-365-FastTrack-Implementation-Assets
Facing issue while reading manifest file from cdmutil_console_app.exe
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 494
- Forks
- 296
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 5
Description
Hi all,
I am trying to create external tables in azure synapse serverless sql pool from manifest files in adls g2 by using CDMUTIL console app. But it always throwing strange error. The implementation looks straight forward but its not working for me. I have followed following docs to implement:
https://en.dynagile.com/blogandnews/pratique-6-bi-architecture-from-datalake-to-data-warehouse
https://github.com/microsoft/Dynamics-365-FastTrack-Implementation-Assets/tree/master/Analytics/CDMUtilSolution
error:
10:35:39 info: CDMUtil.ConsoleApp[0] Key: TenantId Value: 75668f36-65d3-4d90-a990-4bb8ec16728f
10:35:39 info: CDMUtil.ConsoleApp[0] Key: AccessKey Value:***
10:35:39 info: CDMUtil.ConsoleApp[0] Key: ManifestURL Value: https://s.net/d365/Bank.manifest.cdm.json
10:35:39 info: CDMUtil.ConsoleApp[0] Key: TargetDbConnectionString Value:
10:35:39 info: CDMUtil.ConsoleApp[0] Key: DefaultStringLength Value: 4000
10:35:39 info: CDMUtil.ConsoleApp[0] Key: DDLType Value: SynapseExternalTable
10:35:39 info: CDMUtil.ConsoleApp[0] Key: Schema Value: dbo
10:35:39 info: CDMUtil.ConsoleApp[0] Key: FileFormat Value: CSV
10:35:39 info: CDMUtil.ConsoleApp[0] Key: ParserVersion Value: 1.0
10:35:39 info: CDMUtil.ConsoleApp[0] Key: TranslateEnum Value: false
10:35:39 info: CDMUtil.ConsoleApp[0] Key: ProcessEntities Value: false
10:35:39 info: CDMUtil.ConsoleApp[0] Key: CreateStats Value: true
10:35:39 info: CDMUtil.ConsoleApp[0] Reading Manifest metadata https://*********.net/d365/Bank.manifest.cdm.json
Unhandled exception. System.AggregateException: One or more errors occurred. (Index and length must refer to a location within the string. (Parameter 'length'))
---> System.ArgumentOutOfRangeException: Index and length must refer to a location within the string. (Parameter 'length')
at System.String.Substring(Int32 startIndex, Int32 length)
at CDMUtil.Manifest.ManifestBase.mountStorage(AdlsContext adlsContext, String localFolder) in D:\project\team viewer\data\Dynamics-365-FastTrack-Implementation-Assets-master\Analytics\CDMUtilSolution\CDMUtil\Manifest\ManifestHandler.cs:line 1212
at CDMUtil.Manifest.ManifestBase..ctor(AdlsContext adlsContext, String currentFolder, ILogger _logger) in D:\project\team viewer\data\Dynamics-365-FastTrack-Implementation-Assets-master\Analytics\CDMUtilSolution\CDMUtil\Manifest\ManifestHandler.cs:line 1196
at CDMUtil.Manifest.ManifestReader..ctor(AdlsContext adlsContext, String currentFolder, ILogger logger) in D:\project\team viewer\data\Dynamics-365-FastTrack-Implementation-Assets-master\Analytics\CDMUtilSolution\CDMUtil\Manifest\ManifestHandler.cs:line 20
at CDMUtil.Manifest.ManifestReader.manifestToSQLMetadata(AppConfigurations c, List`1 metadataList, ILogger logger, String parentFolder) in D:\project\team viewer\data\Dynamics-365-FastTrack-Implementation-Assets-master\Analytics\CDMUtilSolution\CDMUtil\Manifest\ManifestHandler.cs:line 292
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at CDMUtil.ConsoleApp.Main(String[] args) in D:\project\team viewer\data\Dynamics-365-FastTrack-Implementation-Assets-master\Analytics\CDMUtilSolution\CDMUtil_ConsoleApp\Program.cs:line 40
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at CDMUtil/Manifest/ManifestHandler.cs around mountStorage line 1212, following the ManifestBase and ManifestReader call sites shown in the stack trace. Reproduce the failure with the supplied manifest and configuration, then determine which input causes the substring bounds error; done means manifest metadata can be read without the exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, csharp, sql
- Domain
- cloud, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100