Need to handle Multi Threading for Manual Instrumentation in Faust Application and How to enable opentracing in Faust

Hi Team,

Need help on below problem/implementation for Manual Instrumentation in python for faust streaming application.

Currently on the streaming process we use async to manage concurrency on streaming part which manages the event properly with following code.

@app.agent(topic, concurrency=3)
async def topic_func(topic):
async for event in topic.events():

Now we have started using opentelemetry to span traces for the application. Since Faust doesnt support auto instrumentation yet, we progressed with Manual instrumentation.

For this we are managing and creating context for span in separate library which is outside this async scope and using the same across to create child span on different methods using decorator.

Due to streaming service has concurrency in place which creates multi thread, hence using the context variable which is acting as global variable getting conflicted in multiple threads.

Need to know if there is any existing library which can manage this multi threading and I can use the same context accross to create the child spans.

Also, if possible, could you please also help on , how can I enable opentracing library in faust to enable traces ?

Faust Ver - faust-streaming==0.8.4

Looking forward for the response.

Thanks.
Chirag

Start with the Faust 0.8.4 app.agent(concurrency=3) and topic_func event loop shown in the issue, then review how the external manual-instrumentation library stores context. Establish whether context is isolated per async task or thread and whether Faust has an existing OpenTracing integration; done means a documented, supported approach.