microsoft / microsoft/psi

Serialization Exception when opening templated objects from another framework.

Open
#75 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
574
Forks
104
Avg merge
19m
Merged PRs (30d)
1

Description

This might be related to #74. I have a PsiStore recorded on Windows/.NET Framework and wanted to reply it on a Linux/.NET Core. Currently it only have one stream of type: List<MathNet.Spatial.CoordinateSystem>.

When I try to reopen the stream on the Linux machine, the following inner exception is thrown:

An unhandled exception of type 'System.AggregateException' occurred in Microsoft.Psi.dll: 'Pipeline 'default' was terminated because of one or more unexpected errors'
 Inner exceptions found, see $exception in variables window for more details.
 Innermost exception 	 System.Runtime.Serialization.SerializationException : Failed to create a deserializer for type urn:MathNet/Numerics/LinearAlgebra:DenseColumnMajorMatrixStorageOfdouble because no type was registered for this name and the source type MathNet.Numerics.LinearAlgebra.Storage.DenseColumnMajorMatrixStorage`1[[System.Double, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]], MathNet.Numerics, Version=4.9.1.0, Culture=neutral, PublicKeyToken=cd8b63ad3d691a37 could not be found. Add a reference to the assembly containing this type, or register an alternate type for this name.
   at Microsoft.Psi.Serialization.KnownSerializers.GetUntypedHandler(Int32 handlerId, Type baseType) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/KnownSerializers.cs:line 392
   at Microsoft.Psi.Serialization.RefHandler`1.Deserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 120
   at Microsoft.Psi.Serialization.ClassSerializer`1.Deserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/ClassSerializer.cs:line 82
   at Microsoft.Psi.Serialization.RefHandler`1.InnerDeserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 267
   at Microsoft.Psi.Serialization.RefHandler`1.Deserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 139
   at Microsoft.Psi.Serialization.ArraySerializer`1.Deserialize(BufferReader reader, T[]& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/ArraySerializer.cs:line 48
   at Microsoft.Psi.Serialization.RefHandler`1.InnerDeserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 267
   at Microsoft.Psi.Serialization.RefHandler`1.Deserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 139
   at Microsoft.Psi.Serialization.ClassSerializer`1.Deserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/ClassSerializer.cs:line 82
   at Microsoft.Psi.Serialization.RefHandler`1.InnerDeserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 267
   at Microsoft.Psi.Serialization.RefHandler`1.Deserialize(BufferReader reader, T& target, SerializationContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Serialization/RefHandler.cs:line 139
   at Microsoft.Psi.Components.DeserializerComponent`1.Receive(Message`1 msg, Envelope envelope) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Components/DeserializerComponent.cs:line 43
   at Microsoft.Psi.Pipeline.<>c__DisplayClass104_0`1.<CreateReceiver>b__0(Message`1 m) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Executive/Pipeline.cs:line 367
   at Microsoft.Psi.Executive.PipelineElement.<>c__DisplayClass45_1`1.<TrackStateObjectOnContext>b__1() in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Executive/PipelineElement.cs:line 184
   at Microsoft.Psi.Executive.PipelineElement.<>c__DisplayClass44_0.<TrackStateObjectOnContext>b__0() in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Executive/PipelineElement.cs:line 162
   at Microsoft.Psi.Executive.PipelineElement.<>c__DisplayClass45_0`1.<TrackStateObjectOnContext>b__0(T m) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Executive/PipelineElement.cs:line 184
   at Microsoft.Psi.Receiver`1.<>c__DisplayClass13_0.<.ctor>b__0(Message`1 m) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Streams/Receiver{T}.cs:line 52
   at Microsoft.Psi.Receiver`1.DeliverNext() in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Streams/Receiver{T}.cs:line 297
   at Microsoft.Psi.Scheduling.Scheduler.ExecuteAndRelease(SynchronizationLock synchronizationObject, Action action, SchedulerContext context) in /home/xiangzht/Dev/Psi/psi/Sources/Runtime/Microsoft.Psi/Scheduling/Scheduler.cs:line 542

I did some digging into the problem and it's because the typefinder (/Sources/Runtime/Microsoft.Psi/Common/TypeResolutionHelper.cs) cannot find the templated type System.Double, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089. When the typefinder is looking for the assembly, it is missing in .NET Core.

This is because .NET Framework and .NET Core use different base assemblies to store these basis data types. .NET Framework uses mscorlib whereas .NET Core uses System.Private.CoreLib.

I was able to fix this with this really hacky fix in TypeResolutionHelper.cs:

        private static Assembly AssemblyResolver(AssemblyName assemblyName)
        {
            // Get the list of currently loaded assemblies
            Assembly[] loadedAssemblies = AppDomain.CurrentDomain.GetAssemblies();

           // Fix the name difference between .NET Core and .NET Framework
            if (assemblyName.FullName.StartsWith("mscorlib"))
            {
                return loadedAssemblies.FirstOrDefault(a => a.GetName().FullName.StartsWith("System.Private.CoreLib"));
            }

            // Attempt to match by full name first
            var assembly = loadedAssemblies.FirstOrDefault(a => a.GetName().FullName == assemblyName.FullName);
            if (assembly != null)
            {
                return assembly;
            }

            // Otherwise try to match by simple name without version, culture or key
            assembly = loadedAssemblies.FirstOrDefault(a => AssemblyName.ReferenceMatchesDefinition(a.GetName(), assemblyName));
            if (assembly != null)
            {
                return assembly;
            }

            return null;
        }

I'm pretty sure there are more elegant and better fix, but just want y'all know of this issue. Here's the store I used for testing:
list-of-coordinate-frames.zip

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with Sources/Runtime/Microsoft.Psi/Common/TypeResolutionHelper.cs and the failing lookup in Microsoft.Psi.Serialization.KnownSerializers.GetUntypedHandler. Reproduce by reopening the attached list-of-coordinate-frames.zip store on Linux/.NET Core after recording on Windows/.NET Framework. Done means the templated MathNet stream can be deserialized without the mscorlib/System.Private.CoreLib resolution exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend, stream-processing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.