Bogdanp / Bogdanp/dramatiq

StubBroker problems with Prometheus middleware.

Open
#529 1 comment 0 reactions 0 assignees View on GitHub
prometheus
Dominant language
Python
Stars
5.3k
Forks
383
Avg merge
9h 41m
Merged PRs (30d)
2

Description

# Issues[^discussion]

I’m trying to use the [StubBroker](https://dramatiq.io/reference.html#dramatiq.brokers.stub.StubBroker) in unit tests as described in [this section](https://dramatiq.io/guide.html#unit-testing). I noticed one problem and wanted to raise it for discussion: Using the default middlewares

https://github.com/Bogdanp/dramatiq/blob/a104a0bf999e6eb655c8d21d9d464e6697c7088b/dramatiq/middleware/__init__.py#L44-L47

for the `StubBroker` includes the [Prometheus](https://github.com/Bogdanp/dramatiq/blob/a104a0bf999e6eb655c8d21d9d464e6697c7088b/dramatiq/middleware/prometheus.py#L43) middleware which, when run in a `pytest` context, leaves the following errors after testing:

```
Exception ignored in: <_io.FileIO name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/gauge_livesum_25420.db' mode='ab+' closefd=True>
ResourceWarning: unclosed file <_io.BufferedRandom name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/gauge_livesum_25420.db'>
Exception ignored in: <_io.FileIO name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/histogram_25420.db' mode='ab+' closefd=True>
ResourceWarning: unclosed file <_io.BufferedRandom name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/histogram_25420.db'>
Exception ignored in: <_io.FileIO name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/counter_25420.db' mode='ab+' closefd=True>
ResourceWarning: unclosed file <_io.BufferedRandom name='/var/folders/2r/yyz1661s4j91vhdkzktg66z80000gn/T/dramatiq-prometheus/counter_25420.db'>
```
I’ve not yet dug into this issue, but could probably take a look.

For the time being I work around this by removing the `Prometheus` middleware from the `StubBroker`[^arg] as we discussed [here](https://groups.io/g/dramatiq-users/message/42):
```python
_, *middleware = default_middleware
broker = StubBroker(middleware=[m() for m in middleware])
```
I’m unsure if that’s related to the mentioned [Prometheus Gotchas](https://dramatiq.io/advanced.html#gotchas-with-prometheus).

## What OS are you using?

macOS Ventura 13.2

## What version of Dramatiq are you using?

```python
>>> dramatiq.__version__
'1.14.0'
````

## What did you do?

See above.

## What did you expect would happen?

No warnings from the Prometheus middleware.

## What happened?

Warnings from the Prometheus middleware.

[^discussion]: I think the template needs updating as it still references the retired Reddit instead of the [Groups.io](https://groups.io/g/dramatiq-users) link.

[^arg]: It might be a good idea to document the `middleware` arg for the `StubBroker` [here](https://dramatiq.io/reference.html#dramatiq.brokers.stub.StubBroker)?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.