kubernetes / kubernetes/kops

Cluster Nodes & Masters Failing kube-bench Security Checks - Question

Open
#6,150 22 comments 11 reactions 0 assignees View on GitHub
lifecycle/frozen
Dominant language
Go
Stars
16.7k
Forks
4.7k
Avg merge
17h 21m
Merged PRs (30d)
68

Description

I just ran `kube-bench` (https://github.com/aquasecurity/kube-bench) across my KOPS 1.10 provisioned clusters (Kubernetes `1.10.11` running **kope.io/k8s-1.10-debian-jessie-amd64-hvm-ebs-2018-08-17**) and I was quite shocked to see a large number of tests failed for both nodes and masters.

I expected a few, but not quite this many.

Is there any reason why there seems to be so many security issues with both node and master machines provisioned with KOPS?

Some output

**Workers**
```
[INFO] 2 Worker Node Security Configuration
[INFO] 2.1 Kubelet
[FAIL] 2.1.1 Ensure that the --allow-privileged argument is set to false (Scored)
[FAIL] 2.1.2 Ensure that the --anonymous-auth argument is set to false (Scored)
[FAIL] 2.1.3 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
[FAIL] 2.1.4 Ensure that the --client-ca-file argument is set as appropriate (Scored)
[FAIL] 2.1.5 Ensure that the --read-only-port argument is set to 0 (Scored)
[FAIL] 2.1.6 Ensure that the --streaming-connection-idle-timeout argument is not set to 0 (Scored)
[FAIL] 2.1.7 Ensure that the --protect-kernel-defaults argument is set to true (Scored)
[PASS] 2.1.8 Ensure that the --make-iptables-util-chains argument is set to true (Scored)
[FAIL] 2.1.9 Ensure that the --hostname-override argument is not set (Scored)
[FAIL] 2.1.10 Ensure that the --event-qps argument is set to 0 (Scored)
[FAIL] 2.1.11 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
[PASS] 2.1.12 Ensure that the --cadvisor-port argument is set to 0 (Scored)
[FAIL] 2.1.13 Ensure that the --rotate-certificates argument is not set to false (Scored)
[FAIL] 2.1.14 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
[FAIL] 2.1.15 Ensure that the Kubelet only makes use of Strong Cryptographic Ciphers (Not Scored)
[INFO] 2.2 Configuration Files
[FAIL] 2.2.1 Ensure that the kubelet.conf file permissions are set to 644 or more restrictive (Scored)
[FAIL] 2.2.2 Ensure that the kubelet.conf file ownership is set to root:root (Scored)
[FAIL] 2.2.3 Ensure that the kubelet service file permissions are set to 644 or more restrictive (Scored)
[FAIL] 2.2.4 Ensure that the kubelet service file ownership is set to root:root (Scored)
[FAIL] 2.2.5 Ensure that the proxy kubeconfig file permissions are set to 644 or more restrictive (Scored)
[FAIL] 2.2.6 Ensure that the proxy kubeconfig file ownership is set to root:root (Scored)
[WARN] 2.2.7 Ensure that the certificate authorities file permissions are set to 644 or more restrictive (Scored)
[WARN] 2.2.8 Ensure that the client certificate authorities file ownership is set to root:root (Scored)
[FAIL] 2.2.9 Ensure that the kubelet configuration file ownership is set to root:root (Scored)
[FAIL] 2.2.10 Ensure that the kubelet configuration file has permissions set to 644 or more restrictive (Scored)
```

**Masters**
```
[INFO] 1 Master Node Security Configuration
[INFO] 1.1 API Server
[PASS] 1.1.1 Ensure that the --anonymous-auth argument is set to false (Scored)
[FAIL] 1.1.2 Ensure that the --basic-auth-file argument is not set (Scored)
[PASS] 1.1.3 Ensure that the --insecure-allow-any-token argument is not set (Scored)
[PASS] 1.1.4 Ensure that the --kubelet-https argument is set to true (Scored)
[FAIL] 1.1.5 Ensure that the --insecure-bind-address argument is not set (Scored)
[FAIL] 1.1.6 Ensure that the --insecure-port argument is set to 0 (Scored)
[PASS] 1.1.7 Ensure that the --secure-port argument is not set to 0 (Scored)
[FAIL] 1.1.8 Ensure that the --profiling argument is set to false (Scored)
[FAIL] 1.1.9 Ensure that the --repair-malformed-updates argument is set to false (Scored)
[PASS] 1.1.10 Ensure that the admission control plugin AlwaysAdmit is not set (Scored)
[FAIL] 1.1.11 Ensure that the admission control plugin AlwaysPullImages is set (Scored)
[FAIL] 1.1.12 Ensure that the admission control plugin DenyEscalatingExec is set (Scored)
[FAIL] 1.1.13 Ensure that the admission control plugin SecurityContextDeny is set (Scored)
[FAIL] 1.1.14 Ensure that the admission control plugin NamespaceLifecycle is set (Scored)
[FAIL] 1.1.15 Ensure that the --audit-log-path argument is set as appropriate (Scored)
[FAIL] 1.1.16 Ensure that the --audit-log-maxage argument is set to 30 or as appropriate (Scored)
[FAIL] 1.1.17 Ensure that the --audit-log-maxbackup argument is set to 10 or as appropriate (Scored)
[FAIL] 1.1.18 Ensure that the --audit-log-maxsize argument is set to 100 or as appropriate (Scored)
[PASS] 1.1.19 Ensure that the --authorization-mode argument is not set to AlwaysAllow (Scored)
[FAIL] 1.1.20 Ensure that the --token-auth-file parameter is not set (Scored)
[FAIL] 1.1.21 Ensure that the --kubelet-certificate-authority argument is set as appropriate (Scored)
[FAIL] 1.1.22 Ensure that the --kubelet-client-certificate and --kubelet-client-key arguments are set as appropriate (Scored)
[FAIL] 1.1.23 Ensure that the --service-account-lookup argument is set to true (Scored)
[FAIL] 1.1.24 Ensure that the admission control plugin PodSecurityPolicy is set (Scored)
[FAIL] 1.1.25 Ensure that the --service-account-key-file argument is set as appropriate (Scored)
[FAIL] 1.1.26 Ensure that the --etcd-certfile and --etcd-keyfile arguments are set as appropriate (Scored)
[PASS] 1.1.27 Ensure that the admission control plugin ServiceAccount is set(Scored)
[PASS] 1.1.28 Ensure that the --tls-cert-file and --tls-private-key-file arguments are set as appropriate (Scored)
[PASS] 1.1.29 Ensure that the --client-ca-file argument is set as appropriate (Scored)
[FAIL] 1.1.30 Ensure that the API Server only makes use of Strong Cryptographic Ciphers (Not Scored)
[FAIL] 1.1.31 Ensure that the --etcd-cafile argument is set as appropriate (Scored)
[FAIL] 1.1.32 Ensure that the --authorization-mode argument is set to Node (Scored)
[PASS] 1.1.33 Ensure that the admission control plugin NodeRestriction is set (Scored)
[FAIL] 1.1.34 Ensure that the --experimental-encryption-provider-config argument is set as appropriate (Scored)
[WARN] 1.1.35 Ensure that the encryption provider is set to aescbc (Scored)
[FAIL] 1.1.36 Ensure that the admission control plugin EventRateLimit is set (Scored)
[PASS] 1.1.37 Ensure that the AdvancedAuditing argument is not set to false (Scored)
[PASS] 1.1.38 Ensure that the --request-timeout argument is set as appropriate (Scored)
[FAIL] 1.1.39 Ensure that the API Server only makes use of Strong Cryptographic Ciphers ( Not Scored)
[INFO] 1.2 Scheduler
[FAIL] 1.2.1 Ensure that the --profiling argument is set to false (Scored)
[PASS] 1.2.2 Ensure that the --address argument is set to 127.0.0.1 (Scored)
[INFO] 1.3 Controller Manager
[FAIL] 1.3.1 Ensure that the --terminated-pod-gc-threshold argument is set as appropriate (Scored)
[FAIL] 1.3.2 Ensure that the --profiling argument is set to false (Scored)
[PASS] 1.3.3 Ensure that the --use-service-account-credentials argument is set to true (Scored)
[PASS] 1.3.4 Ensure that the --service-account-private-key-file argument is set as appropriate (Scored)
[PASS] 1.3.5 Ensure that the --root-ca-file argument is set as appropriate (Scored)
[FAIL] 1.3.6 Ensure that the RotateKubeletServerCertificate argument is set to true (Scored)
[PASS] 1.3.7 Ensure that the --address argument is set to 127.0.0.1 (Scored)
[INFO] 1.4 Configuration Files
[FAIL] 1.4.1 Ensure that the API server pod specification file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.2 Ensure that the API server pod specification file ownership is set to root:root (Scored)
[FAIL] 1.4.3 Ensure that the controller manager pod specification file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.4 Ensure that the controller manager pod specification file ownership is set to root:root (Scored)
[FAIL] 1.4.5 Ensure that the scheduler pod specification file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.6 Ensure that the scheduler pod specification file ownership is set to root:root (Scored)
[FAIL] 1.4.7 Ensure that the etcd pod specification file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.8 Ensure that the etcd pod specification file ownership is set to root:root (Scored)
[WARN] 1.4.9 Ensure that the Container Network Interface file permissions are set to 644 or more restrictive (Not Scored)
[WARN] 1.4.10 Ensure that the Container Network Interface file ownership is set to root:root (Not Scored)
[FAIL] 1.4.11 Ensure that the etcd data directory permissions are set to 700 or more restrictive (Scored)
[FAIL] 1.4.12 Ensure that the etcd data directory ownership is set to etcd:etcd (Scored)
[FAIL] 1.4.13 Ensure that the admin.conf file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.14 Ensure that the admin.conf file ownership is set to root:root (Scored)
[FAIL] 1.4.15 Ensure that the scheduler.conf file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.16 Ensure that the scheduler.conf file ownership is set to root:root (Scored)
[FAIL] 1.4.17 Ensure that the controller-manager.conf file permissions are set to 644 or more restrictive (Scored)
[FAIL] 1.4.18 Ensure that the controller-manager.conf file ownership is set to root:root (Scored)
[INFO] 1.5 etcd
[FAIL] 1.5.1 Ensure that the --cert-file and --key-file arguments are set as appropriate (Scored)
[FAIL] 1.5.2 Ensure that the --client-cert-auth argument is set to true (Scored)
[PASS] 1.5.3 Ensure that the --auto-tls argument is not set to true (Scored)
[WARN] 1.5.4 Ensure that the --peer-cert-file and --peer-key-file arguments are set as appropriate (Scored)
[WARN] 1.5.5 Ensure that the --peer-client-cert-auth argument is set to true (Scored)
[WARN] 1.5.6 Ensure that the --peer-auto-tls argument is not set to true (Scored)
[WARN] 1.5.7 Ensure that a unique Certificate Authority is used for etcd (Not Scored)
[INFO] 1.6 General Security Primitives
[WARN] 1.6.1 Ensure that the cluster-admin role is only used where required (Not Scored)
[WARN] 1.6.2 Create administrative boundaries between resources using namespaces (Not Scored)
[WARN] 1.6.3 Create network segmentation using Network Policies (Not Scored)
[WARN] 1.6.4 Ensure that the seccomp profile is set to docker/default in your pod definitions (Not Scored)
[WARN] 1.6.5 Apply Security Context to Your Pods and Containers (Not Scored)
[WARN] 1.6.6 Configure Image Provenance using ImagePolicyWebhook admission controller (Not Scored)
[WARN] 1.6.7 Configure Network policies as appropriate (Not Scored)
[WARN] 1.6.8 Place compensating controls in the form of PSP and RBAC for privileged containers usage (Not Scored)
[INFO] 1.7 PodSecurityPolicies
[WARN] 1.7.1 Do not admit privileged containers (Not Scored)
[WARN] 1.7.2 Do not admit containers wishing to share the host process ID namespace (Not Scored)
[WARN] 1.7.3 Do not admit containers wishing to share the host IPC namespace (Not Scored)
[WARN] 1.7.4 Do not admit containers wishing to share the host network namespace (Not Scored)
[WARN] 1.7.5 Do not admit containers with allowPrivilegeEscalation (Not Scored)
[WARN] 1.7.6 Do not admit root containers (Not Scored)
[WARN] 1.7.7 Do not admit containers with dangerous capabilities (Not Scored)
```

_UPDATED WITH FULL LIST AFTER VERSION 1.10.11 UPGRADE_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the reported kube-bench results against a KOPS 1.10 cluster running Kubernetes 1.10.11, then compare the worker and master failures with the cluster configuration. Done means determining which findings are expected defaults versus kOps security defects and documenting the required resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure, security
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.