oxidecomputer / oxidecomputer/propolis

oximeter log messages can intersperse with propolis-server log messages

Open
#512 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
270
Forks
42
Avg merge
4d 5h
Merged PRs (30d)
6

Description

This is the same thing as #407, but for oximeter; this call creates its own logger instead of using a child logger:
https://github.com/oxidecomputer/propolis/blob/b59b355a18691af30641ce2d12b723abb3d8b9c1/bin/propolis-server/src/lib/stats.rs#L147

which results in log message interleaving:

Aug 25 21:55:34.920 INFO listening, local_addr: 127.0.0.1:37557, component: dropshot
Aug 25 21:55:34.961 WARN Aug 25 21:55:34.961Could not connect to oximeter (retrying in Error registering as metric producer: Communication Error: error sending request for url (http://127.0.0.1:9000/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)s):
1 , INFOuri :received request to begin graceful shutdown , /instancelocal_addr, :method :127 .0PUT.0, .1req_id:37557:,  component62d5fd2e-f857-413a-827e-40e39666e014:,  remote_addrdropshot:
 127.0.0.1:46236, local_addr: 127.0.0.1:8000
Aug 25 21:55:35.963 INFO listening, local_addr: 127.0.0.1:60856, component: dropshot
Aug 25 21:55:35.993 Aug 25 21:55:35.993WARN  INFOCould not connect to oximeter (retrying in Error registering as metric producer: Communication Error: error sending request for url (http://127.0.0.1:9000/metrics/producers): error trying to connect: tcp connect error: Connection refused (os error 146)s):
1 , received request to begin graceful shutdownuri, :local_addr :/instance , 127method.0:.0 .1PUT:60856, , req_idcomponent::  62d5fd2e-f857-413a-827e-40e39666e014dropshot, 
remote_addr: 127.0.0.1:46236, local_addr: 127.0.0.1:8000

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at bin/propolis-server/src/lib/stats.rs around line 147 and compare the related issue #407. Reproduce the oximeter and propolis-server logging scenario, then verify that the resulting messages no longer intersperse.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
observability-sre
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.