IQSS / IQSS/dataverse

Reduce log clutter from robots and security tests

Open
#7,706 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

At QDR, I often see many log entries that are from robots or security tests (I've checked IP addresses at times, for others I'm only assuming). While its good to log these, there are several that also print the stack trace. For example, I'm looking right now at many repeats of a NullPointerException stack trace from SendFeedbackDialog.java:185 because the Captcha/Math problem for the form wasn't filled in. UT Dorkbot also seems to try various calls that result in a stack trace.

I would propose that, in such cases, we just log the problem without also sending a stack trace (or move the stack trace inside an if clause to check for FINE logging. Are there even better options? (I don't want to ban all stack traces since they are important in catching rarer errors.))

If there's interest, and this sounds like a reasonable approach, I'll try to get to it at some point (though others are welcome).

I'd also like to request that site admins add their 'most annoying' cases of this here so we can start to build a list. I've start it off with the

  1. NPE at SendFeedbackDialog.java:185 (robots don't answer the math question)
  2. .UnsupportedOperationException: DataFile.getLatestPublishedFileMetadata(DataFile.java:572) (api call to get metadata on an unreleased file)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the logging around SendFeedbackDialog.java:185 and DataFile.java:572, then identify how these expected robot or unreleased-file cases are currently reported. The work is done when those cases no longer emit stack traces while rarer errors still retain them; no tests are named in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
observability-sre
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.