canonical / canonical/cos-proxy-operator

cos_agent 0.25 breaks unit tests

Open
#236 0 comments 0 reactions 0 assignees View on GitHub
Status: Triage Type: Bug
Dominant language
Python
Stars
3
Forks
12
PR merge metrics
No merged PRs in 30d

Description

### Bug Description

If you `charmcraft fetch-lib` to obtain a recent cos_agent (e.g. 0.25), then cos-proxy unit tests fail.

### To Reproduce

```
charmcraft fetch-lib
tox -e unit
```

### Environment

Any

### Relevant log output

```shell
================================================== FAILURES ==================================================
_______________________ TestRelationMonitors.test_cos_agent_with_downstream_prometheus _______________________
Traceback (most recent call last):
File "/usr/lib/python3.14/unittest/case.py", line 58, in testPartExecutor
yield
File "/usr/lib/python3.14/unittest/case.py", line 669, in run
self._callTestMethod(testMethod)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 615, in _callTestMethod
result = method()
File "/home/resolute/code/cos-proxy-operator/tests/unit/test_outgoing_relation_data.py", line 271, in test_cos_agent_with_downstream_prometheus
self.assertEqual(
~~~~~~~~~~~~~~~~^
json.loads(prom_app_data["scrape_jobs"]), cos_agent_config["metrics_scrape_jobs"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/lib/python3.14/unittest/case.py", line 925, in assertEqual
assertion_func(first, second, msg=msg)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 1131, in assertListEqual
self.assertSequenceEqual(list1, list2, msg, seq_type=list)
~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 1113, in assertSequenceEqual
self.fail(msg)
~~~~~~~~~^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
raise self.failureException(msg)
AssertionError: Lists differ: [{'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_ama[4013 chars]'}]}] != [{'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_cos-proxy_[4108 chars]rt'}]

First differing element 0:
{'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_ama[965 chars]ort'}
{'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_cos-proxy_[450 chars])'}]}

- [{'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_conntrack_prometheus_scrape',
+ [{'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_cos-proxy_prometheus_scrape_dcd1032f',
+ 'relabel_configs': [{'regex': '(.*)',
+ 'separator': '_',
+ 'source_labels': ['juju_model',
+ 'juju_model_uuid',
+ 'juju_application',
+ 'juju_unit'],
+ 'target_label': 'instance'}],
+ 'static_configs': [{'labels': {'dns_name': 'localhost',
+ 'host': '127.0.0.1',
+ 'juju_application': 'cos-proxy',
+ 'juju_model': 'mymodel',
+ 'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
+ 'juju_unit': 'cos-proxy/0'},
+ 'targets': ['127.0.0.1:9090']}]},
+ {'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_telegraf_prometheus_scrape_9e4543dc',
+ 'relabel_configs': [{'regex': '(.*)',
+ 'separator': '_',
+ 'source_labels': ['juju_model',
+ 'juju_model_uuid',
+ 'juju_application',
+ 'juju_unit'],
+ 'target_label': 'instance'}],
+ 'static_configs': [{'labels': {'dns_name': 'localhost',
+ 'host': '10.181.49.93',
+ 'juju_application': 'telegraf',
+ 'juju_model': 'mymodel',
+ 'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
+ 'juju_unit': 'telegraf/0'},
+ 'targets': ['10.181.49.93:9103']}]},
+ {'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_conntrack_prometheus_scrape_aeecee48',
'metrics_path': '/export',
'params': {'command': ['check_conntrack'], 'ssl': [True]},
'relabel_configs': [{'regex': '(.*)',
'separator': '_',
'source_labels': ['juju_model',
'juju_model_uuid',
'juju_application',
'juju_unit'],
'target_label': 'instance'},
{'source_labels': ['__address__'],
'target_label': '__param_target'},
{'source_labels': ['__param_target'],
'target_label': 'instance'},
{'source_labels': ['__param_command'],
'target_label': 'command'},
{'replacement': '10.41.168.226:9275',
'target_label': '__address__'},
{'replacement': 'ubuntu-is-amazing/0',
'target_label': 'juju_unit'},
{'replacement': 'ubuntu-is-amazing',
'target_label': 'juju_application'}],
'static_configs': [{'labels': {'dns_name': 'localhost',
'host': '10.41.168.226',
'juju_application': 'nrpe',
'juju_model': 'mymodel',
'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
'juju_unit': 'nrpe/0'},
'targets': ['10.41.168.226:5666']}]},
- {'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_systemd_scopes_prometheus_scrape',
? ^^^ ----------

+ {'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_reboot_prometheus_scrape_31236bb4',
? ++++++++++ ^^^^^ +++++++++

- 'metrics_path': '/export',
- 'params': {'command': ['check_systemd_scopes'], 'ssl': [True]},
- 'relabel_configs': [{'regex': '(.*)',
- 'separator': '_',
- 'source_labels': ['juju_model',
- 'juju_model_uuid',
- 'juju_application',
- 'juju_unit'],
- 'target_label': 'instance'},
- {'source_labels': ['__address__'],
- 'target_label': '__param_target'},
- {'source_labels': ['__param_target'],
- 'target_label': 'instance'},
- {'source_labels': ['__param_command'],
- 'target_label': 'command'},
- {'replacement': '10.41.168.226:9275',
- 'target_label': '__address__'},
- {'replacement': 'ubuntu-is-amazing/0',
- 'target_label': 'juju_unit'},
- {'replacement': 'ubuntu-is-amazing',
- 'target_label': 'juju_application'}],
- 'static_configs': [{'labels': {'dns_name': 'localhost',
- 'host': '10.41.168.226',
- 'juju_application': 'nrpe',
- 'juju_model': 'mymodel',
- 'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
- 'juju_unit': 'nrpe/0'},
- 'targets': ['10.41.168.226:5666']}]},
- {'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_reboot_prometheus_scrape',
'metrics_path': '/export',
'params': {'command': ['check_reboot'], 'ssl': [True]},
'relabel_configs': [{'regex': '(.*)',
'separator': '_',
'source_labels': ['juju_model',
'juju_model_uuid',
'juju_application',
'juju_unit'],
'target_label': 'instance'},
{'source_labels': ['__address__'],
'target_label': '__param_target'},
{'source_labels': ['__param_target'],
'target_label': 'instance'},
{'source_labels': ['__param_command'],
'target_label': 'command'},
{'replacement': '10.41.168.226:9275',
'target_label': '__address__'},
{'replacement': 'ubuntu-is-amazing/0',
'target_label': 'juju_unit'},
{'replacement': 'ubuntu-is-amazing',
'target_label': 'juju_application'}],
'static_configs': [{'labels': {'dns_name': 'localhost',
'host': '10.41.168.226',
'juju_application': 'nrpe',
'juju_model': 'mymodel',
'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
'juju_unit': 'nrpe/0'},
'targets': ['10.41.168.226:5666']}]},
- {'job_name': 'juju_mymodel_fe2c9bb_cos-proxy_prometheus_scrape',
+ {'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_systemd_scopes_prometheus_scrape_baa46444',
+ 'metrics_path': '/export',
+ 'params': {'command': ['check_systemd_scopes'], 'ssl': [True]},
'relabel_configs': [{'regex': '(.*)',
'separator': '_',
'source_labels': ['juju_model',
'juju_model_uuid',
'juju_application',
'juju_unit'],
- 'target_label': 'instance'}],
? -

+ 'target_label': 'instance'},
+ {'source_labels': ['__address__'],
+ 'target_label': '__param_target'},
+ {'source_labels': ['__param_target'],
+ 'target_label': 'instance'},
+ {'source_labels': ['__param_command'],
+ 'target_label': 'command'},
+ {'replacement': '10.41.168.226:9275',
+ 'target_label': '__address__'},
+ {'replacement': 'ubuntu-is-amazing/0',
+ 'target_label': 'juju_unit'},
+ {'replacement': 'ubuntu-is-amazing',
+ 'target_label': 'juju_application'}],
'static_configs': [{'labels': {'dns_name': 'localhost',
- 'host': '127.0.0.1',
? --- ^

+ 'host': '10.41.168.226',
? ^^ ++++++

- 'juju_application': 'cos-proxy',
? ^^^^ ^^^^

+ 'juju_application': 'nrpe',
? ^^ ^

'juju_model': 'mymodel',
'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
- 'juju_unit': 'cos-proxy/0'},
? ^^^^ ^^^^

+ 'juju_unit': 'nrpe/0'},
? ^^ ^

- 'targets': ['127.0.0.1:9090']}]},
? --- ^ ^^^^ ^

+ 'targets': ['10.41.168.226:5666']}]}]
? ^^ ++++++ ^^^^ ^

- {'job_name': 'juju_mymodel_fe2c9bb_telegraf_prometheus_scrape',
- 'relabel_configs': [{'regex': '(.*)',
- 'separator': '_',
- 'source_labels': ['juju_model',
- 'juju_model_uuid',
- 'juju_application',
- 'juju_unit'],
- 'target_label': 'instance'}],
- 'static_configs': [{'labels': {'dns_name': 'localhost',
- 'host': '10.181.49.93',
- 'juju_application': 'telegraf',
- 'juju_model': 'mymodel',
- 'juju_model_uuid': 'fe2c9bbb-58ab-40e4-8f70-f27480093fca',
- 'juju_unit': 'telegraf/0'},
- 'targets': ['10.181.49.93:9103']}]}]

__________________________________ TestRelationMonitors.test_only_cos_agent __________________________________
Traceback (most recent call last):
File "/usr/lib/python3.14/unittest/case.py", line 58, in testPartExecutor
yield
File "/usr/lib/python3.14/unittest/case.py", line 669, in run
self._callTestMethod(testMethod)
~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 615, in _callTestMethod
result = method()
File "/home/resolute/code/cos-proxy-operator/tests/unit/test_outgoing_relation_data.py", line 254, in test_only_cos_agent
self.assert_outgoing_data_on_removal({"agent": rel_id_agent}, upstream_ids)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/resolute/code/cos-proxy-operator/tests/unit/test_outgoing_relation_data.py", line 195, in assert_outgoing_data_on_removal
self.assertEqual(
~~~~~~~~~~~~~~~~^
{"telegraf", "cos-proxy"}, self.get_app_names_from_scrape_jobs(scrape_jobs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/usr/lib/python3.14/unittest/case.py", line 925, in assertEqual
assertion_func(first, second, msg=msg)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 1185, in assertSetEqual
self.fail(self._formatMessage(msg, standardMsg))
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3.14/unittest/case.py", line 750, in fail
raise self.failureException(msg)
AssertionError: Items in the second set but not the first:
'nrpe'

FAILED tests/unit/test_outgoing_relation_data.py::TestRelationMonitors::test_cos_agent_with_downstream_prometheus - AssertionError: Lists differ: [{'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_ama[4013 chars]'}]}] !...
FAILED tests/unit/test_outgoing_relation_data.py::TestRelationMonitors::test_only_cos_agent - AssertionError: Items in the second set but not the first:
```

### Additional context

The first error,
```
AssertionError: Lists differ: [{'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_ama[4013 chars]'}]}] != [{'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_cos-proxy_[4108 chars]rt'}]
```
we can fix by modifying the unit test as follows:

```diff
- self.assertEqual(
- json.loads(prom_app_data["scrape_jobs"]), cos_agent_config["metrics_scrape_jobs"]
- )

+ downstream = sorted(json.loads(prom_app_data["scrape_jobs"]), key=lambda j: j.get("job_name"))
+ cos_agent = sorted(cos_agent_config["metrics_scrape_jobs"], key=lambda j: j.get("job_name"))

+ # NOTE: the scrape jobs are expected to be identical, except for the "job_name" field which is generated
+ # with a random suffix to avoid conflicts in cos-agent when multiple scrape jobs exist.
+ # So "job_name" field is compared separately and then removed before full-dict comparison.
+ # - {'job_name': 'juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_systemd_scopes_prometheus_scrape',
+ # + {'job_name': 'cos-proxy_juju_mymodel_fe2c9bb_ubuntu_ubuntu_is_amazing_0_check_systemd_scopes_prometheus_scrape_35630366'
+ for downstream_job, cos_agent_job in zip(downstream, cos_agent):
+ self.assertIn(
+ downstream_job.pop("job_name"),
+ cos_agent_job.pop("job_name"),
+ )

+ self.assertEqual(downstream, cos_agent)
```

The second error, I'm not sure about.

Contributor guide

No contributing guide indexed for this repository

Research direction

Run `charmcraft fetch-lib` followed by `tox -e unit`, then inspect `tests/unit/test_outgoing_relation_data.py`, especially `TestRelationMonitors.test_cos_agent_with_downstream_prometheus`. Compare the relation data produced with cos_agent 0.25 against the test expectations; done means the unit suite passes while the Prometheus scrape-job data remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.