IronLanguages / IronLanguages/ironpython3
Parallel execution of multiple IronPython engines with one runtime
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 2.8k
- Forks
- 316
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 1
Description
Description
Hi,
we found a changed behaviour in IP 3.4.1 compared to the IP 2.x implementation.
We are using IP to execute python code in multiple threads.
To improve performance the work threads are precreated and share the same (global) runtime.
After switching to IP 3 we found that addings an assembly via the clr methodes and also imports to e.g. the decimal module sometimes fail. This is different for what happend in IP 2.x, there was no issue, running using mutiple python enginens in parallel threads with one global runtime.
We have been able to mitigate the issue with the addition of a process wide lock object used in the methods responsible for imports and adding assembly references. However, we can't tell for sure that all potential methods are
Steps to Reproduce
- Create a shared runtime
- Create multipe thread having their private python engine
- Ensure that the threads are started at the same time
- Have the engine execute script code including clr.AddReference calls and "from decimal import Decimal"
- Attached is a C# sample program (renamed to .cs.txt to allow upload) and a patch file with the changes we made to try to fix the issue
Program.cs.txt
mutliThreadedSuport.patch
Expected behavior:
All threads execute the script code without errors
Actual behavior:
Some threads fail to execute the script code with error like "module 'numbers' not in sys.modules" or failure to add the assembly via the clr method. The actual error can be different.
Version Information
- .NET platform used: .NET Framework
- Version 4.6.2
- Windows 11
- 3.4.1 DEBUG (3.4.1.1000) / [.NETFramework,Version=v4.6.2 on .NET Framework 4.8.9282.0 (64-bit)]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the attached Program.cs.txt and reproduce the shared-runtime case using multiple private engines, concurrent clr.AddReference calls, and decimal imports. Review mutliThreadedSuport.patch alongside the reproduction to identify the affected import and assembly-reference paths; done means all threads execute the script without errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, python
- Domain
- backend, compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100