clj-python / clj-python/libpython-clj

How can I call python functions that rely on "event_loop" inside go blocks?

Open
#208 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Clojure
Stars
1.2k
Forks
74
PR merge metrics
No merged PRs in 30d

Description

Huge thanks for the great library. I'm wondering if the following is possible.

The problem

I want to call python functions with asyncio.get_event_loop() inside go blocks.

The get_event_loop is buried deep in the python code base, so I can't simply swap every required event loop to some new event loop that I create.

I haven't come up with a way to write a wrapper, and setting the event loop manually does not help.

I'm not seeking to solve the exception, because I think my approach seems wrong and the exception is expected, but rather, I'm asking for a way for my goal.

Many Thanks!

A minimal example that crashes

(ns main.playasync
  (:require [libpython-clj2.require :refer [require-python]]
            [libpython-clj2.python :refer [py. py.. py.-] :as py]
            [clojure.core.async :as async :refer [>! <! chan go]]))

(require-python '[agen]
                '[builtins :as python]
                '[asyncio])

(comment
  (go
    (let [ev-loop (asyncio/new_event_loop)]
      (asyncio/set_event_loop ev-loop)
      (asyncio/get_event_loop))))

will raise an exception

Exception in thread "async-dispatch-6" java.lang.Exception: Traceback (most recent call last):
  File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/asyncio/events.py", line 642, in get_event_loop
    raise RuntimeError('There is no current event loop in thread %r.'
RuntimeError: There is no current event loop in thread 'Dummy-6'.

        at libpython_clj2.python.ffi$check_error_throw.invokeStatic(ffi.clj:703)
        at libpython_clj2.python.ffi$check_error_throw.invoke(ffi.clj:701)
        at libpython_clj2.python.ffi$simplify_or_track.invokeStatic(ffi.clj:960)
        at libpython_clj2.python.ffi$simplify_or_track.invoke(ffi.clj:941)
        at libpython_clj2.python.fn$call_py_fn.invokeStatic(fn.clj:197)
        at libpython_clj2.python.fn$call_py_fn.invoke(fn.clj:177)
        at libpython_clj2.python.bridge_as_jvm$generic_callable_pyobject$reify__17712.call(bridge_as_jvm.clj:427)
        at libpython_clj2.python.fn$call_kw.invokeStatic(fn.clj:217)
        at libpython_clj2.python.fn$call_kw.invoke(fn.clj:214)
        at libpython_clj2.python.fn$cfn.invokeStatic(fn.clj:282)
        at libpython_clj2.python.fn$cfn.doInvoke(fn.clj:273)
        at clojure.lang.RestFn.invoke(RestFn.java:410)
        at libpython_clj2.python.bridge_as_jvm$generic_callable_pyobject$reify__17712.invoke(bridge_as_jvm.clj:427)
        at main.playasync$eval31361$fn__31362$state_machine__28343__auto____31363$fn__31365.invoke(NO_SOURCE_FILE:14)
        at main.playasync$eval31361$fn__31362$state_machine__28343__auto____31363.invoke(NO_SOURCE_FILE:11)
        at clojure.core.async.impl.ioc_macros$run_state_machine.invokeStatic(ioc_macros.clj:978)
        at clojure.core.async.impl.ioc_macros$run_state_machine.invoke(ioc_macros.clj:977)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invokeStatic(ioc_macros.clj:982)
        at clojure.core.async.impl.ioc_macros$run_state_machine_wrapped.invoke(ioc_macros.clj:980)
        at main.playasync$eval31361$fn__31362.invoke(NO_SOURCE_FILE:11)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at clojure.core.async.impl.concurrent$counted_thread_factory$reify__22117$fn__22118.invoke(concurrent.clj:29)
        at clojure.lang.AFn.run(AFn.java:22)
        at java.base/java.lang.Thread.run(Thread.java:833)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the minimal core.async go example in the issue and the traceback showing asyncio.get_event_loop running on a Dummy thread. Read the libpython-clj2 Python call path and the core.async go execution context to determine whether event-loop access can be supported. Done would be a documented, working way to call these Python functions from go blocks, or a clear explanation that the approach is unsupported.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.