Single log line is 30+ MB in size
- Dominant language
- Python
- Stars
- 13
- Forks
- 17
- Avg merge
- 6h 59m
- Merged PRs (30d)
- 1
Description
When debugging other things, I took a dump of the docker logs and discovered there to be a single line that, when saved to its own file, was 32.6 MiB in size.
This is insane.
this line came from postgres logging a query line (unsure whether it was an error or not because it was truncated)
This query was attempting to insert (at least from what i could see) at least 85422 records into the repo_labor table.
This logging is probably also made worse by the recursive way we process data in bulk_insert_dicts (causing failures to show up many times).
the affected line
https://github.com/chaoss/CollectOSS/blob/d289de1b69c3cc1640ff53b5401a03a9972c5af6/collectoss/tasks/git/scc_value_tasks/core.py#L53
Contributor guide
Assessment
This issue has not been assessed yet.