Autodesk / Autodesk/hubble

Cannot parse missing logs after upgrade - KeyError http

Open
#198 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
180
Forks
47
PR merge metrics
No merged PRs in 30d

Description

We just upgraded to 2.18.8 and see this now when running Hubble. Maybe because the old (pre-upgrade) logs were moved. Certainly github-audit.log.1*.gz does not exist.

The KeyError: 'http' is an interesting one. Again, maybe related to missing logs?

```
...
gzip: /var/log/github-audit.log.1*.gz: No such file or directory

gzip: /var/log/haproxy.log.1*.gz: No such file or directory
Use of uninitialized value in new at -e line 1.

gzip: /var/log/haproxy.log.1*.gz: No such file or directory
Use of uninitialized value in new at -e line 1.

/git/tmpdata/hubble-data/git-download.tsv
/git/tmpdata/hubble-data/git-download-detailed.tsv
Finished update of git-download.tsv (runtime: 1.721 s)
Started update of git-protocol.tsv
gzip: /var/log/babeld/babeld.log.1*.gz: No such file or directory

gzip: /var/log/babeld/babeld.log.1*.gz: No such file or directory

Traceback (most recent call last):
File "./update-stats.py", line 121, in
main()
File "./update-stats.py", line 84, in main
ReportGitProtocol(configuration, dataDirectory, metaStats).update()
File "/apps/git/GitHubInsights/updater/reports/Report.py", line 168, in update
self.updateData()
File "/apps/git/GitHubInsights/updater/reports/ReportDaily.py", line 35, in updateData
self.updateDailyData()
File "/apps/git/GitHubInsights/updater/reports/ReportGitProtocol.py", line 20, in updateDailyData
protocols["http"],
KeyError: 'http'
```

Contributor guide

Open the contributing guide

Research direction

Start with update-stats.py and the ReportGitProtocol path shown in the traceback, especially Report.py and ReportDaily.py. Reproduce the Hubble update with the referenced rotated log files absent, then determine the expected behavior for missing logs and the resulting protocols data; the update should no longer fail with KeyError: 'http'.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
analytics
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.