User access logs
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
Let's add **the user access logs** for auditing and security inspection.Conceptually, the access log should include at least:
- The user identity
- The keypair (and its relevant "realm") which is used to authenticate\* In the future, we may have multiple per-user keypairs issued for different realms such as web server, FastTrack, etc.
- The last access timestamp
- The client IP
To minimize performance and storage impacts, we need to limit the maximum history length for each keypair. Usually a single client session generates a lot of API requests, simply stripping the history by the timestamp would be useless; we should keep track of the unique client IPs per keypair up to a configurable number.
Currently, there are two ways to access the Backend.AI Manager:
```mermaid
flowchart TD
User(Users on Web Browser) -->|web credentials| W[WebServer]
W -->|"access key and secret key
(Case 1)"| M[Manager]
Client(Client SDK or CLI) -->|"access key and secret key
(Case 2)"| M
Client -->|"web credentials via
`backend.ai login`"| W
```
For the case 1 (via the web server), the observed client IP is always same to the IP of the WebServer from the Manager's perspective. We need to use the `X-Forwarded-For` header added by the webserver to get the actual client's IP to record as an access log.For the case 2 (direct API access), we could directly record the client IP.
The recording logic could be written in a similar way like the rate-limiter. Using Redis sorted sets, we could add a key-value pair set of `{client_ip`} → `{timestamp`} per `{access_key`} key. We could use `ZREMRANGEBYRANK {access_key} {N} -1` to preserve N latest-seen unique client IPs, while `ZADD` will update the timestamp of each client IP. To keep the latest entries in the low rank, we could multiply -1 to the timestamp when adding to the sorted set. When removing a keypair, this Redis sorted-set item should be removed as well.
The control panel may query this Redis database by each access key (`ZRANGE {access_key} 0 -1 WITHSCORES`) and join the access key with the user/keypair information fetched from the database, to display the access log with the user identity and keypair's realm. The final display will include a nested list like:
- For each user (a table in a screen/panel/tab)\* For each keypair (rows in the table) - fetched from PostgreSQL\* A list of latest N unique client IPs with their last-access timestamps (a collapsible inner-list of each row) - fetched from Redis
JIRA Issue: BA-125
Contributor guide
Assessment
This issue has not been assessed yet.