microsoft / microsoft/BCApps

[Bug]: BC 28.5 on-prem — Invoke-NAVApplicationDatabaseConversion fails with TypeInitializationException in OpenTelemetry (regression vs 28.3)

Open
#11,216 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

AL: Tools Ownership: Manual Ownership: Needs Review
Dominant language
AL
Stars
683
Forks
459
Avg merge
3d 26m
Merged PRs (30d)
633

Description

Describe the issue

On Business Central 28.5 on-premises (build 28.0.54016.0), every Management cmdlet that constructs a NavEnvironment fails immediately when executed from powershell.exe.

Invoke-NAVApplicationDatabaseConversion: The type initializer for
'OpenTelemetry.Instrumentation.Http.Implementation.HttpHandlerDiagnosticListener' threw an exception.

Event Viewer:

TypeLoadException: Could not load type 'System.Diagnostics.ActivitySourceOptions'
from assembly 'System.Diagnostics.DiagnosticSource, Version=8.0.0.0, ...'

Stack:

OpenTelemetry.Trace.ActivitySourceFactory.Create
 -> OpenTelemetry.Instrumentation.Http.Implementation.HttpHandlerDiagnosticListener..cctor()
 -> OpenTelemetry.Trace.HttpClientInstrumentationTracerProviderBuilderExtensions.AddHttpClientInstrumentation
 -> Microsoft.BusinessCentral.Telemetry.OpenTelemetry.OpenTelemetrySpanLogger..ctor
 -> Microsoft.Dynamics.Nav.Diagnostic.NavOpenTelemetryLogger..ctor
 -> Microsoft.Dynamics.Nav.Runtime.NavEnvironment..ctor
 -> InvokeNAVApplicationDatabaseConversion.InternalProcessRecord()

Root cause analysis

This is a regression introduced in 28.5. Comparing two servers with otherwise identical configuration (Windows Server 2022, .NET Framework 4.8 release 528449, Windows PowerShell 5.1, same side-by-side BC installs):

28.3 — 28.0.52201.0 (works) 28.5 — 28.0.54016.0 (fails)
System.Diagnostics.DiagnosticSource 10.0.0.0 10.0.0.0
OpenTelemetry.Instrumentation.Http 1.9.0.41 1.16.0.1135

OpenTelemetry.Instrumentation.Http was upgraded from 1.9.0.41 to 1.16.0.1135. Version 1.16 references System.Diagnostics.DiagnosticSource 8.0.0.0 and requires the ActivitySource API surface from 9.x/10.x in the static constructor of HttpHandlerDiagnosticListener. Version 1.9 did not touch that code path, so the mismatch was harmless.

Correct binding redirects do exist:

  • Service\Microsoft.Dynamics.Nav.Server.exe.configSystem.Diagnostics.DiagnosticSource 0.0.0.0-10.0.0.0 -> 10.0.0.0
  • Service\Management\Microsoft.Dynamics.Nav.Management.dll.config — same redirects

The NST service works because it is an .exe and reads its own .exe.config. However, .NET Framework never reads *.dll.config — configuration is resolved only from <process>.exe.config. When Management cmdlets are loaded into powershell.exe, no redirect applies, the CLR resolves DiagnosticSource 8.0.0.0, and the type initializer throws.

Impact: all on-premises upgrade automation on 28.5 is blocked — database conversion, and any other cmdlet that builds a NavEnvironment.

Expected behavior

Invoke-NAVApplicationDatabaseConversion (and other Management cmdlets creating a NavEnvironment) should run successfully from powershell.exe after importing NavAdminTool.ps1, exactly as they do on BC 28.3 and earlier.

Steps to reproduce
  1. Install Business Central 28.5 on-premises (build 28.0.54016.0) on Windows Server 2022, .NET Framework 4.8.
  2. Open a clean powershell.exe -NoProfile session (Windows PowerShell 5.1, Desktop edition).
  3. Import the admin tool:
    . 'C:\Program Files\Microsoft Dynamics 365 Business Central\280\Service\NavAdminTool.ps1'
    
  4. Run a database conversion against a database from an earlier version (e.g. 27.3):
    Invoke-NAVApplicationDatabaseConversion `
        -DatabaseServer 'SQLSERVER' `
        -DatabaseName   'BC Test' `
        -Force
    
  5. The cmdlet fails with TypeInitializationException before touching SQL.

Repeating the same steps on BC 28.3 (28.0.52201.0) succeeds.

Additional context

Environment

  • Windows Server 2022 Standard / Datacenter
  • .NET Framework 4.8, release 528449
  • Windows PowerShell 5.1 (Desktop) — reproduced on builds 5.1.20348.4294 and confirmed working on 5.1.20348.5499 with 28.3
  • Side-by-side BC installs present on both servers; ruled out as a factor (both machines have 5 versions installed, PATH/PSModulePath contain no BC entries, GAC contains no System.Diagnostics.DiagnosticSource)

Things that were ruled out

  • Licence — the cmdlet fails before licence validation
  • GAC pollution — no DiagnosticSource in GAC on either machine
  • PATH / PSModulePath — no BC paths present
  • Az / AzureRM modules — not installed
  • Preloading the correct assembly via Assembly.LoadFrom plus an AppDomain.AssemblyResolve hook does not help, because the OpenTelemetry reference is resolved in the default load context, which does not see the LoadFrom context

Workarounds

  1. Copy the <assemblyBinding> section from Microsoft.Dynamics.Nav.Management.dll.config into %WINDIR%\System32\WindowsPowerShell\v1.0\powershell.exe.config, then run the cmdlet with the working directory set to ...\280\Service. This is global and affects every PowerShell session on the machine, so it must be reverted afterwards.
  2. Downgrade the platform to 28.3.
  3. Perform the upgrade inside a Docker container built from a 28.3 artifact.

Suggested fix

Any of the following would resolve it:

  • Ship a powershell.exe.config fragment (or documented merge step) together with the Administration Tool component
  • Apply the binding redirects programmatically from NavAdminTool.ps1, e.g. via a dedicated AppDomain with the correct AppDomainSetup.ConfigurationFile
  • Pin OpenTelemetry.Instrumentation.Http to a version compatible with the shipped System.Diagnostics.DiagnosticSource 10.0.0.0, or ship a matching DiagnosticSource assembly version
  • Make NavEnvironment tolerate OpenTelemetry initialisation failures instead of propagating them as a fatal TypeInitializationException
I will provide a fix for a bug
  • I will provide a fix for a bug

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing Invoke-NAVApplicationDatabaseConversion from a clean Windows PowerShell session after importing Service\NavAdminTool.ps1. Read Service\Management\Microsoft.Dynamics.Nav.Management.dll.config and Service\Microsoft.Dynamics.Nav.Server.exe.config, then compare how powershell.exe loads their binding redirects. Done means the conversion and other cmdlets that construct NavEnvironment run on 28.5 without a global PowerShell configuration workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
backend, cli
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.