JuliaPy / JuliaPy/PyCall.jl

Multiple local imports

Open
#896 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
1.5k
Forks
186
PR merge metrics
No merged PRs in 30d

Description

Hello!

I have a very, I think, specific issue and I would like to know, why PyCall does that and how to fix it.

Let's assume, that I have 2 folders `folder1` and `folder2` which are repositories written in python and each has a main script file `main1.py` and `main2.py`, and those two just have inside them multiple local imports and a function `main`.

Then I have two julia scripts `folder1.jl` and `folder2.jl`, which call those `main` functions from those two folders. The code for `folder1.jl` goes like this:

```julia
pushfirst!(PyVector(pyimport("sys")."path"), folder1)
execute_JL = pyimport("main1")
pyimport("importlib").reload(execute_JL)[:main](args)
```


And both julia scripts work fine. Now I want to combine them and call both scripts in the third script `combine.jl`.

This no longer works and here are two things that happen.

1. If the first element of `PyVector(pyimport("sys")."path")` is `folder1`, then I can import `main1`, if it's `folder2`, then no. But that I managed to fix this by just removing the folder from the path, once I ran the script.
2. If I do `pyimport("main1")`, I then cannot do `pyimport("main2")` and the opposite.

In both of these cases the first time you run the code the error doesn't even say anything - just that `PyObject(Ptr{PyCall.PyObject_struct} @0x00000000521c49f0), PyObject(Ptr{PyCall.PyObject_struct} @0x00000000650bf2e8), PyObject(Ptr{PyCall.PyObject_struct} @0x00000000650b58c8))`, for example. The second time you run it - all the local imports in the file that cannot be imported fail.

Can someone tell me why that happens?

P.S. As a workaround, I could probably create `setup.py` files in those folders and install them like python libraries, but I still wonder why right now I can't make this work.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the imports from folder1.jl, folder2.jl, and combine.jl using main1.py and main2.py, then inspect Python's sys.path and module state after the first import. Done means identifying why both local import trees cannot coexist through PyCall and documenting or validating a workable fix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.