elastic / elastic/apm-agent-python

Standalone Celery and traces

Open
#1,452 1 comment 0 reactions 1 assignee Claimed by @beniwohli View on GitHub
agent-python community
Dominant language
Python
Stars
431
Forks
239
Avg merge
5d 10h
Merged PRs (30d)
7

Description

Hello,

We have 10 celery services (apps) working together with the same rabbitmq. Some task are send from outside the services and some tasks are called between them.
To track it we are using the apm standalone celery approach which is recommended. But it seems that the distributed trace implemented in this plugin is not suited for this. I'm wondering if its because celery is treated like a module rather than a framework ?

The results is that we can see the list of all our services. but some are empty, with no transaction at all. And when I'm looking at some services they have their own transaction listed as well as the one from other services. I'm guessing its the way it as been design for Django but not to work in a meshing of services ?

What I would expect is seeing every transaction listed in its own celery services. And when looking at a specific transaction, if a trace has been pass to another service, seeing it also in the waterfall of that transaction. Right now it is quite a mess, but I'm not sure why.

**Environment (please complete the following information)**
- OS: Linux
- Python version: 2.7
- Framework and version : Celery 4.4.7
- Agent version: elasticapm 5.10.1

**Additional context**

ServiceA and ServiceB are listed in Kibana APM. Both services are Celery apps.

```python
import elasticapm
from elasticapm import Client, instrument, events
from elasticapm.contrib.celery import register_exception_tracking, register_instrumentation

instrument()
apm_client = Client(service_name=app_name)
register_instrumentation(apm_client)
register_exception_tracking(apm_client)
app = Celery()
```

if ServiceB is called from ServiceA, transactions in serviceA will be empty, everything will be listed in ServiceB. So in ServiceB transactions list I'll see ServiceB.method and ServiceA.method

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.