Structured logging of parameters interpolated in the audit log message
Nobody has claimed this yet.
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
How to use GitHub
- Please use the 👍 reaction to show that you are interested into the same feature.
- Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
- Subscribe to receive notifications on status change and new comments.
Is your feature request related to a problem? Please describe.
The admin_audit app produces logs of events which contain useful parameters interpolated in the message string:
{
"reqId": "2OaR2jAZibElRV0mpGAn",
"level": 3,
"time": "2022-07-08T13:59:14+00:00",
"remoteAddr": "172.17.0.1",
"user": "admin",
"app": "admin_audit",
"method": "POST",
"url": "/ocs/v2.php/apps/files_sharing/api/v1/shares",
"message": "The file \"/admin/files/Nextcloud.png\" with ID \"14\" has been shared via link with permissions \"17\" (Share ID: 1)",
"userAgent": "Mozilla/5.0...",
"version": "24.0.2.1"
}
While the messages themselves are useful, such logs are difficult to process and index (i.e. while working with the ELK stack). Let's say that one would want to filter out all events related to a specific file being shared via a link. In such a scenario, individual fields would have to either be extracted from the message string before being sent to Elasticsearch (i.e. using Filebeat regex processors) or the admin searching the logs would have to perform a text search on messages themselves, which is neither efficient nor easy to work with, especially if the message format is not known beforehand.
Describe the solution you'd like
Modify the admin_audit app so that it features structured logging of parameters that are included in the message string as well as a type of the action that is being logged. The IDataLogger interface (as used by the workflow engine; implicitly implemented by what is returned by getCustomPsrLogger method) could be utilized for that purpose.
Consequently, the the log entries could look like this:
{
"reqId": "2OaR2jAZibElRV0mpGAn",
"level": 3,
"time": "2022-07-08T13:59:14+00:00",
"remoteAddr": "172.17.0.1",
"user": "admin",
"app": "admin_audit",
"method": "POST",
"url": "/ocs/v2.php/apps/files_sharing/api/v1/shares",
"message": "The file \"/admin/files/Nextcloud.png\" with ID \"14\" has been shared via link with permissions \"17\" (Share ID: 1)",
"userAgent": "Mozilla/5.0...",
"version": "24.0.2.1",
"data": {
"eventType": "SHARE_LINK_CREATED",
"itemType": "file",
"itemSource": 14,
"permissions": 17,
"id": "1",
"path": "/admin/files/Nextcloud.png"
}
}
...or even going one step further and including all of the CriticalActionPerformedEvent parameters in the log message (unless obfuscateParameters is set to true):
{
"reqId": "2OaR2jAZibElRV0mpGAn",
"level": 3,
"time": "2022-07-08T13:59:14+00:00",
"remoteAddr": "172.17.0.1",
"user": "admin",
"app": "admin_audit",
"method": "POST",
"url": "/ocs/v2.php/apps/files_sharing/api/v1/shares",
"message": "The file \"/admin/files/Nextcloud.png\" with ID \"14\" has been shared via link with permissions \"17\" (Share ID: 1)",
"userAgent": "Mozilla/5.0...",
"version": "24.0.2.1",
"data": {
"eventType": "SHARE_LINK_CREATED",
"itemType": "file",
"itemSource": 14,
"shareType": 3,
"uidOwner": "admin",
"permissions": 17,
"fileSource": 14,
"expiration": null,
"token": "JFQqeCbTTLWK8Kq",
"id": "1",
"shareWith": null,
"itemTarget": "/Nextcloud.png",
"fileTarget": "/Nextcloud.png",
"path": "/admin/files/Nextcloud.png"
}
}
One additional consideration is that parameter names would ideally have to be unified. I have seen userID being referred to as uid as well as user in the app.
Describe alternatives you've considered
As described earlier, useful parameters can be extracted from the audit log messages using regex. I've seen it being done i.e. here (Filebeat config)
Let me know your feedback on this issue. I'm more than happy to prepare a PR.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the admin_audit app's use of getCustomPsrLogger and the IDataLogger interface, then inspect CriticalActionPerformedEvent and its parameters. Define the structured fields and naming rules, including obfuscation behavior, before changing the audit log output. Done means audit entries expose the action type and relevant interpolated parameters in a consistently named data structure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100