matomo-org / matomo-org/docker

Unable to update logs with log-analytics

Open
#223 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
1k
Forks
384
PR merge metrics
No merged PRs in 30d

Description

When I try to run /usr/bin/python2 /var/www/matomo/misc/log-analytics/import_logs.py --url=https://example.com --idsite=1 --recorders=4 --enable-http-errors --enable-http-redirects --enable-bots /var/log/containers/nginx/access.ota.log I get the following output :

0 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
Parsing log /var/log/containers/nginx/access.ota.log...
2020-11-11 19:54:44,301: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:44,302: [INFO] Retrying request, attempt number 2
2020-11-11 19:54:44,319: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:44,320: [INFO] Retrying request, attempt number 2
2020-11-11 19:54:44,322: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:44,322: [INFO] Retrying request, attempt number 2
2020-11-11 19:54:44,323: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:44,325: [INFO] Retrying request, attempt number 2
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
2020-11-11 19:54:54,362: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:54,363: [INFO] Retrying request, attempt number 3
2020-11-11 19:54:54,384: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:54,384: [INFO] Retrying request, attempt number 3
2020-11-11 19:54:54,386: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:54,386: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:54:54,387: [INFO] Retrying request, attempt number 3
2020-11-11 19:54:54,387: [INFO] Retrying request, attempt number 3
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
3202 lines parsed, 0 lines recorded, 0 records/sec (avg), 0 records/sec (current)
2020-11-11 19:55:04,412: [INFO] Error when connecting to Matomo: HTTP Error 400: Bad Request
2020-11-11 19:55:04,413: [INFO] Max number of attempts reached, server is unreachable!
Fatal error: Server returned status 400 (Bad Request).
Is the BulkTracking plugin disabled?
You can restart the import of "/var/log/containers/nginx/access.ota.log" from the point it failed by specifying --skip=2 on the command line.

It suddenly stopped working since last one week even though there were no noticeable changes. Any idea what could have gone wrong? BulkTracking in enabled

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with misc/log-analytics/import_logs.py and the shown command/output; inspect how requests reach Matomo and how HTTP 400 responses are reported. Check the BulkTracking setting and reproduce with the referenced nginx access log; done means the import records lines without the repeated 400 failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
nginx, python
Domain
api, backend, data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.