ModusCreateOrg / ModusCreateOrg/devops-infra-demo

Enhance cloudwatch log capturing to include entire output of systemd logs

Open
#40 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement help wanted
Dominant language
Shell
Stars
34
Forks
19
PR merge metrics
No merged PRs in 30d

Description

I want everything that journalctl -f emits, in a systemd log group, for example:

Jul 30 16:17:48 ip-10-0-1-81.ec2.internal uwsgi[3620]: ImportError: Import by filename is not supported.
Jul 30 16:17:48 ip-10-0-1-81.ec2.internal uwsgi[3620]: unable to load app 1 (mountpoint='/api') (callable not found or import error)
Jul 30 16:17:48 ip-10-0-1-81.ec2.internal uwsgi[3620]: ImportError: Import by filename is not supported.
Jul 30 16:17:48 ip-10-0-1-81.ec2.internal uwsgi[3620]: unable to load app 1 (mountpoint='/api') (callable not found or import error)
Jul 30 16:17:48 ip-10-0-1-81.ec2.internal uwsgi[3620]: [pid: 3641|app: 0|req: 31/127] 10.0.1.127 () {38 vars in 444 bytes} [Tue Jul 30 16:17:48 2019] GET /api/spin => generated 150 bytes in 70 msecs (HTTP/1.1 200) 2 headers in 66 bytes (1 switches on core 0)
Jul 30 16:17:49 ip-10-0-1-81.ec2.internal uwsgi[3620]: [pid: 3685|app: 0|req: 86/128] 10.0.2.111 () {38 vars in 444 bytes} [Tue Jul 30 16:17:48 2019] GET /api/spin => generated 149 bytes in 808 msecs (HTTP/1.1 200) 2 headers in 66 bytes (1 switches on core 0)
Jul 30 16:18:00 ip-10-0-1-81.ec2.internal sudo[6351]:   centos : TTY=pts/0 ; PWD=/home/centos ; USER=root ; COMMAND=/bin/journalctl -f
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: ImportError: Import by filename is not supported.
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: unable to load app 1 (mountpoint='/api') (callable not found or import error)
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: [pid: 3685|app: 0|req: 87/129] 10.0.3.114 () {38 vars in 444 bytes} [Tue Jul 30 16:18:18 2019] GET /api/spin => generated 149 bytes in 74 msecs (HTTP/1.1 200) 2 headers in 66 bytes (1 switches on core 0)
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: ImportError: Import by filename is not supported.
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: unable to load app 1 (mountpoint='/api') (callable not found or import error)
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: ImportError: Import by filename is not supported.
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: unable to load app 1 (mountpoint='/api') (callable not found or import error)
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: [pid: 3669|app: 0|req: 1/130] 10.0.1.127 () {38 vars in 444 bytes} [Tue Jul 30 16:18:18 2019] GET /api/spin => generated 151 bytes in 99 msecs (HTTP/1.1 200) 2 headers in 66 bytes (1 switches on core 0)
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: announcing my loyalty to the Emperor...
Jul 30 16:18:18 ip-10-0-1-81.ec2.internal uwsgi[3620]: Tue Jul 30 16:18:18 2019 - [emperor] vassal infra-demo.ini is now loyal
Jul 30 16:18:19 ip-10-0-1-81.ec2.internal uwsgi[3620]: [pid: 3685|app: 0|req: 88/131] 10.0.2.111 () {38 vars in 444 bytes} [Tue Jul 30 16:18:18 2019] GET /api/spin => generated 149 bytes in 618 msecs (HTTP/1.1 200) 2 headers in 66 bytes (1 switches on core 0)

Once we get this working we should examine the data captured in the other log groups to see if there are redundant logs we can remove.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named; start by locating the CloudWatch log-group configuration and the entry point that captures systemd or journalctl output. Compare the other log groups, then verify that the full journalctl -f output reaches the systemd group and identify any redundant groups for removal.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, shell
Domain
cloud, devops, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.