aquasecurity / aquasecurity/tracee

When enabling "exec-env" option, too many false positives happen

Open
#3,659 7 comments 0 reactions 1 assignee Claimed by @josedonizetti View on GitHub
kind/bug
Dominant language
Go
Stars
4.6k
Forks
507
Avg merge
2d 8h
Merged PRs (30d)
9

Description

## Description

When running tracee in a k8s environment, if I change the default configmap to have "exec-env" option enabled, I started getting too many "ld_preload" signatures triggered for many processes:

```yaml
$ microk8s.kubectl describe cm -n default tracee-config
...
Data
====
config.yaml:
----
no-containers: false
cache:
type: mem
size: 512
perf-buffer-size: 1024
healthz: true
metrics: true
pprof: false
pyroscope: false
listen-addr: :3366
log:
level: info
output:
json:
files:
- stdout
options:
parse-arguments: true
stack-addresses: false
exec-env: true
relative-time: true
exec-hash: false
sort-events: true
...
```

The false positives:

```json
{"timestamp":310041368905,"threadStartTime":310040881916,"processorId":3,"processId":30736,"cgroupId":2751,"threadId":30736,"parentProcessId":595,"hostProcessId":30736,"hostThreadId":30736,"hostParentProcessId":595,"userId":0,"mountNamespace":4026531841,"pidNamespace":4026531836,"processName":"snapctl","executable":{"path":""},"hostName":"vm02","containerId":"","container":{},"kubernetes":{},"eventId":"6006","eventName":"ld_preload","matchedPolicies":["default-policy"],"argsNum":2,"returnValue":0,"syscall":"","stackAddresses":null,"contextFlags":{"containerStarted":false,"isCompat":false},"threadEntityId":0,"processEntityId":0,"parentEntityId":0,"args":[{"name":"LD_LIBRARY_PATH","type":"const char *","value":"LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/microk8s/6070/lib:/snap/microk8s/6070/usr/lib:/snap/microk8s/6070/lib/x86_64-linux-gnu:/snap/microk8s/6070/usr/lib/x86_64-linux-gnu"},{"name":"triggeredBy","type":"unknown","value":{"args":[{"name":"cmdpath","type":"const char*","value":"/usr/bin/snapctl"},{"name":"pathname","type":"const char*","value":"/usr/lib/snapd/snapctl"},{"name":"dev","type":"dev_t","value":8388609},{"name":"inode","type":"unsigned long","value":16793},{"name":"ctime","type":"unsigned long","value":1697595129754118964},{"name":"inode_mode","type":"umode_t","value":33261},{"name":"interpreter_pathname","type":"const char*","value":null},{"name":"interpreter_dev","type":"dev_t","value":null},{"name":"interpreter_inode","type":"unsigned long","value":null},{"name":"interpreter_ctime","type":"unsigned long","value":null},{"name":"argv","type":"const char**","value":["snapctl","services","microk8s.daemon-kubelite"]},{"name":"interp","type":"const char*","value":"/usr/bin/snapctl"},{"name":"stdin_type","type":"string","value":"S_IFCHR"},{"name":"stdin_path","type":"char*","value":"/dev/null"},{"name":"invoked_from_kernel","type":"int","value":0},{"name":"env","type":"const char**","value":["SNAP_REVISION=6070","SNAP_REAL_HOME=/root","SNAP_USER_COMMON=/root/snap/microk8s/common","SNAP_INSTANCE_KEY=","SNAP_EUID=0","PWD=/var/snap/microk8s/6070","SYSTEMD_EXEC_PID=595","SNAP_CONTEXT=lGjw0Qm7x4cgAEd0QIOdClvVq4s-3mTB01eiuJmO5YMX9UAtzf1j","LANG=C.UTF-8","SNAP_ARCH=amd64","SNAP_INSTANCE_NAME=microk8s","SNAP_USER_DATA=/root/snap/microk8s/6070","INVOCATION_ID=0549ddae482040868d7df7a41ddf1ce2","SNAP_REEXEC=","SNAP_UID=0","PYTHONPATH=/snap/microk8s/6070/usr/lib/python3.8:/snap/microk8s/6070/lib/python3.8/site-packages:/snap/microk8s/6070/usr/lib/python3/dist-packages:","SNAP=/snap/microk8s/6070","SNAP_COMMON=/var/snap/microk8s/common","SNAP_VERSION=v1.27.6","SHLVL=0","SNAP_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void","SNAP_COOKIE=lGjw0Qm7x4cgAEd0QIOdClvVq4s-3mTB01eiuJmO5YMX9UAtzf1j","SNAP_DATA=/var/snap/microk8s/6070","LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/microk8s/6070/lib:/snap/microk8s/6070/usr/lib:/snap/microk8s/6070/lib/x86_64-linux-gnu:/snap/microk8s/6070/usr/lib/x86_64-linux-gnu","LC_ALL=C.UTF-8","SNAP_NAME=microk8s","JOURNAL_STREAM=8:15068","PATH=/snap/microk8s/6070/usr/sbin:/snap/microk8s/6070/usr/bin:/snap/microk8s/6070/sbin:/snap/microk8s/6070/bin:/snap/microk8s/6070/usr/bin:/snap/microk8s/6070/bin:/snap/microk8s/6070/usr/sbin:/snap/microk8s/6070/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin","_=/usr/bin/snapctl"]}],"id":713,"name":"sched_process_exec","returnValue":0}}],"metadata":{"Version":"1","Description":"LD_PRELOAD usage was detected. LD_PRELOAD lets you load your library before any other library, allowing you to hook functions in a process. Adversaries may use this technique to change your applications' behavior or load their own programs.","Tags":null,"Properties":{"Category":"persistence","Kubernetes_Technique":"","Severity":2,"Technique":"Hijack Execution Flow","external_id":"T1574","id":"attack-pattern--aedfca76-3b30-4866-b2aa-0f1d7fd1e4b6","signatureID":"TRC-107","signatureName":"LD_PRELOAD code injection detected"}}}
{"timestamp":310041420783,"threadStartTime":310040974073,"processorId":0,"processId":30737,"cgroupId":2751,"threadId":30737,"parentProcessId":595,"hostProcessId":30737,"hostThreadId":30737,"hostParentProcessId":595,"userId":0,"mountNamespace":4026531841,"pidNamespace":4026531836,"processName":"grep","executable":{"path":""},"hostName":"vm02","containerId":"","container":{},"kubernetes":{},"eventId":"6006","eventName":"ld_preload","matchedPolicies":["default-policy"],"argsNum":2,"returnValue":0,"syscall":"","stackAddresses":null,"contextFlags":{"containerStarted":false,"isCompat":false},"threadEntityId":0,"processEntityId":0,"parentEntityId":0,"args":[{"name":"LD_LIBRARY_PATH","type":"const char *","value":"LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/microk8s/6070/lib:/snap/microk8s/6070/usr/lib:/snap/microk8s/6070/lib/x86_64-linux-gnu:/snap/microk8s/6070/usr/lib/x86_64-linux-gnu"},{"name":"triggeredBy","type":"unknown","value":{"args":[{"name":"cmdpath","type":"const char*","value":"/snap/microk8s/6070/bin/grep"},{"name":"pathname","type":"const char*","value":"/snap/microk8s/6070/bin/grep"},{"name":"dev","type":"dev_t","value":7340037},{"name":"inode","type":"unsigned long","value":414},{"name":"ctime","type":"unsigned long","value":1696933677000000000},{"name":"inode_mode","type":"umode_t","value":33261},{"name":"interpreter_pathname","type":"const char*","value":"/snap/core20/2015/usr/lib/x86_64-linux-gnu/ld-2.31.so"},{"name":"interpreter_dev","type":"dev_t","value":7340033},{"name":"interpreter_inode","type":"unsigned long","value":6046},{"name":"interpreter_ctime","type":"unsigned long","value":1649294681000000000},{"name":"argv","type":"const char**","value":["grep","active"]},{"name":"interp","type":"const char*","value":"/snap/microk8s/6070/bin/grep"},{"name":"stdin_type","type":"string","value":"S_IFIFO"},{"name":"stdin_path","type":"char*","value":""},{"name":"invoked_from_kernel","type":"int","value":0},{"name":"env","type":"const char**","value":["SNAP_REVISION=6070","SNAP_REAL_HOME=/root","SNAP_USER_COMMON=/root/snap/microk8s/common","SNAP_INSTANCE_KEY=","SNAP_EUID=0","PWD=/var/snap/microk8s/6070","SYSTEMD_EXEC_PID=595","SNAP_CONTEXT=lGjw0Qm7x4cgAEd0QIOdClvVq4s-3mTB01eiuJmO5YMX9UAtzf1j","LANG=C.UTF-8","SNAP_ARCH=amd64","SNAP_INSTANCE_NAME=microk8s","SNAP_USER_DATA=/root/snap/microk8s/6070","INVOCATION_ID=0549ddae482040868d7df7a41ddf1ce2","SNAP_REEXEC=","SNAP_UID=0","PYTHONPATH=/snap/microk8s/6070/usr/lib/python3.8:/snap/microk8s/6070/lib/python3.8/site-packages:/snap/microk8s/6070/usr/lib/python3/dist-packages:","SNAP=/snap/microk8s/6070","SNAP_COMMON=/var/snap/microk8s/common","SNAP_VERSION=v1.27.6","SHLVL=0","SNAP_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void","SNAP_COOKIE=lGjw0Qm7x4cgAEd0QIOdClvVq4s-3mTB01eiuJmO5YMX9UAtzf1j","SNAP_DATA=/var/snap/microk8s/6070","LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/microk8s/6070/lib:/snap/microk8s/6070/usr/lib:/snap/microk8s/6070/lib/x86_64-linux-gnu:/snap/microk8s/6070/usr/lib/x86_64-linux-gnu","LC_ALL=C.UTF-8","SNAP_NAME=microk8s","JOURNAL_STREAM=8:15068","PATH=/snap/microk8s/6070/usr/sbin:/snap/microk8s/6070/usr/bin:/snap/microk8s/6070/sbin:/snap/microk8s/6070/bin:/snap/microk8s/6070/usr/bin:/snap/microk8s/6070/bin:/snap/microk8s/6070/usr/sbin:/snap/microk8s/6070/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin","_=/snap/microk8s/6070/bin/grep"]}],"id":713,"name":"sched_process_exec","returnValue":0}}],"metadata":{"Version":"1","Description":"LD_PRELOAD usage was detected. LD_PRELOAD lets you load your library before any other library, allowing you to hook functions in a process. Adversaries may use this technique to change your applications' behavior or load their own programs.","Tags":null,"Properties":{"Category":"persistence","Kubernetes_Technique":"","Severity":2,"Technique":"Hijack Execution Flow","external_id":"T1574","id":"attack-pattern--aedfca76-3b30-4866-b2aa-0f1d7fd1e4b6","signatureID":"TRC-107","signatureName":"LD_PRELOAD code injection detected"}}}
{"timestamp":310049980862,"threadStartTime":310049704987,"processorId":3,"processId":30743,"cgroupId":2751,"threadId":30743,"parentProcessId":595,"hostProcessId":30743,"hostThreadId":30743,"hostParentProcessId":595,"userId":0,"mountNamespace":4026531841,"pidNamespace":4026531836,"processName":"sleep","executable":{"path":""},"hostName":"vm02","containerId":"","container":{},"kubernetes":{},"eventId":"6006","eventName":"ld_preload","matchedPolicies":["default-policy"],"argsNum":2,"returnValue":0,"syscall":"","stackAddresses":null,"contextFlags":{"containerStarted":false,"isCompat":false},"threadEntityId":0,"processEntityId":0,"parentEntityId":0,"args":[{"name":"LD_LIBRARY_PATH","type":"const char *","value":"LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/microk8s/6070/lib:/snap/microk8s/6070/usr/lib:/snap/microk8s/6070/lib/x86_64-linux-gnu:/snap/microk8s/6070/usr/lib/x86_64-linux-gnu"},{"name":"triggeredBy","type":"unknown","value":{"args":[{"name":"cmdpath","type":"const char*","value":"/snap/microk8s/6070/bin/sleep"},{"name":"pathname","type":"const char*","value":"/snap/microk8s/6070/bin/sleep"},{"name":"dev","type":"dev_t","value":7340037},{"name":"inode","type":"unsigned long","value":445},{"name":"ctime","type":"unsigned long","value":1696933680000000000},{"name":"inode_mode","type":"umode_t","value":33261},{"name":"interpreter_pathname","type":"const char*","value":"/snap/core20/2015/usr/lib/x86_64-linux-gnu/ld-2.31.so"},{"name":"interpreter_dev","type":"dev_t","value":7340033},{"name":"interpreter_inode","type":"unsigned long","value":6046},{"name":"interpreter_ctime","type":"unsigned long","value":1649294681000000000},{"name":"argv","type":"const char**","value":["sleep","5"]},{"name":"interp","type":"const char*","value":"/snap/microk8s/6070/bin/sleep"},{"name":"stdin_type","type":"string","value":"S_IFCHR"},{"name":"stdin_path","type":"char*","value":"/dev/null"},{"name":"invoked_from_kernel","type":"int","value":0},{"name":"env","type":"const char**","value":["SNAP_REVISION=6070","SNAP_REAL_HOME=/root","SNAP_USER_COMMON=/root/snap/microk8s/common","SNAP_INSTANCE_KEY=","SNAP_EUID=0","PWD=/var/snap/microk8s/6070","SYSTEMD_EXEC_PID=595","SNAP_CONTEXT=lGjw0Qm7x4cgAEd0QIOdClvVq4s-3mTB01eiuJmO5YMX9UAtzf1j","LANG=C.UTF-8","SNAP_ARCH=amd64","SNAP_INSTANCE_NAME=microk8s","SNAP_USER_DATA=/root/snap/microk8s/6070","INVOCATION_ID=0549ddae482040868d7df7a41ddf1ce2","SNAP_REEXEC=","SNAP_UID=0","PYTHONPATH=/snap/microk8s/6070/usr/lib/python3.8:/snap/microk8s/6070/lib/python3.8/site-packages:/snap/microk8s/6070/usr/lib/python3/dist-packages:","SNAP=/snap/microk8s/6070","SNAP_COMMON=/var/snap/microk8s/common","SNAP_VERSION=v1.27.6","SHLVL=1","SNAP_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void","SNAP_COOKIE=lGjw0Qm7x4cgAEd0QIOdClvVq4s-3mTB01eiuJmO5YMX9UAtzf1j","SNAP_DATA=/var/snap/microk8s/6070","LD_LIBRARY_PATH=/var/lib/snapd/lib/gl:/var/lib/snapd/lib/gl32:/var/lib/snapd/void:/snap/microk8s/6070/lib:/snap/microk8s/6070/usr/lib:/snap/microk8s/6070/lib/x86_64-linux-gnu:/snap/microk8s/6070/usr/lib/x86_64-linux-gnu","LC_ALL=C.UTF-8","SNAP_NAME=microk8s","JOURNAL_STREAM=8:15068","PATH=/snap/microk8s/6070/usr/sbin:/snap/microk8s/6070/usr/bin:/snap/microk8s/6070/sbin:/snap/microk8s/6070/bin:/snap/microk8s/6070/usr/bin:/snap/microk8s/6070/bin:/snap/microk8s/6070/usr/sbin:/snap/microk8s/6070/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin","_=/snap/microk8s/6070/bin/sleep"]}],"id":713,"name":"sched_process_exec","returnValue":0}}],"metadata":{"Version":"1","Description":"LD_PRELOAD usage was detected. LD_PRELOAD lets you load your library before any other library, allowing you to hook functions in a process. Adversaries may use this technique to change your applications' behavior or load their own programs.","Tags":null,"Properties":{"Category":"persistence","Kubernetes_Technique":"","Severity":2,"Technique":"Hijack Execution Flow","external_id":"T1574","id":"attack-pattern--aedfca76-3b30-4866-b2aa-0f1d7fd1e4b6","signatureID":"TRC-107","signatureName":"LD_PRELOAD code injection detected"}}}
```

And I can list the binaries causing the triggering:

```console
$ microk8s.kubectl logs tracee-zc65w | jq -r '.processName' | sort -u
10-cni-restart
10-cni-restart-
90-calico-apply
bandwidth
calico
calico-ipam
chgrp
chmod
getent
grep
ip6tables
iptables
loopback
null
openssl
portmap
python3
runc
sleep
snapctl
```

```console
$ microk8s.kubectl logs tracee-jvbtb | jq -r '.processName' | sort -u
10-cni-restart
10-cni-restart-
90-calico-apply
bandwidth
calico
calico-ipam
chgrp
chmod
getent
grep
ip6tables
iptables
jq
loopback
null
openssl
portmap
python3
runc
sleep
snapctl
```

Microk8s runs using snaps (just like flatpaks) and that could be causing the fause positives (needs to check before confirming), but it could also be that the signature is being triggered by something else.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.