Add seccomp default feature blog post
This adds the blog post about the new Kubernetes `SeccompDefault` alpha feature. Signed-off-by: Sascha Grunert <sgrunert@redhat.com>pull/28951/head
Sascha Grunert
parent
86aa6c434d
commit
84e472e95c
|
|
@ -0,0 +1,267 @@
|
||||||
|
---
|
||||||
|
layout: blog
|
||||||
|
title: "Enable seccomp for all workloads with a new v1.22 alpha feature"
|
||||||
|
date: 2021-08-25
|
||||||
|
slug: seccomp-default
|
||||||
|
---
|
||||||
|
|
||||||
|
**Author:** Sascha Grunert, Red Hat
|
||||||
|
|
||||||
|
This blog post is about a new Kubernetes feature introduced in v1.22, which adds
|
||||||
|
an additional security layer on top of the existing seccomp support. Seccomp is
|
||||||
|
a security mechanism for Linux processes to filter system calls (syscalls) based
|
||||||
|
on a set of defined rules. Applying seccomp profiles to containerized workloads
|
||||||
|
is one of the key tasks when it comes to enhancing the security of the
|
||||||
|
application deployment. Developers, site reliability engineers and
|
||||||
|
infrastructure administrators have to work hand in hand to create, distribute
|
||||||
|
and maintain the profiles over the applications life-cycle.
|
||||||
|
|
||||||
|
You can use the [`securityContext`][seccontext] field of Pods and their
|
||||||
|
containers can be used to adjust security related configurations of the
|
||||||
|
workload. Kubernetes introduced dedicated [seccomp related API
|
||||||
|
fields][seccontext] in this `SecurityContext` with the [graduation of seccomp to
|
||||||
|
General Availability (GA)][ga] in v1.19.0. This enhancement allowed an easier
|
||||||
|
way to specify if the whole pod or a specific container should run as:
|
||||||
|
|
||||||
|
[seccontext]: /docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context-1
|
||||||
|
[ga]: https://kubernetes.io/blog/2020/08/26/kubernetes-release-1.19-accentuate-the-paw-sitive/#graduated-to-stable
|
||||||
|
|
||||||
|
- `Unconfined`: seccomp will not be enabled
|
||||||
|
- `RuntimeDefault`: the container runtimes default profile will be used
|
||||||
|
- `Localhost`: a node local profile will be applied, which is being referenced
|
||||||
|
by a relative path to the seccomp profile root (`<kubelet-root-dir>/seccomp`)
|
||||||
|
of the kubelet
|
||||||
|
|
||||||
|
With the graduation of seccomp, nothing has changed from an overall security
|
||||||
|
perspective, because `Unconfined` is still the default. This is totally fine if
|
||||||
|
you consider this from the upgrade path and backwards compatibility perspective of
|
||||||
|
Kubernetes releases. But it also means that it is more likely that a workload
|
||||||
|
runs without seccomp at all, which should be fixed in the long term.
|
||||||
|
|
||||||
|
## `SeccompDefault` to the rescue
|
||||||
|
|
||||||
|
Kubernetes v1.22.0 introduces a new kubelet [feature gate][gate]
|
||||||
|
`SeccompDefault`, which has been added in `alpha` state as every other new
|
||||||
|
feature. This means that it is disabled by default and can be enabled manually
|
||||||
|
for every single Kubernetes node.
|
||||||
|
|
||||||
|
[gate]: /docs/reference/command-line-tools-reference/feature-gates
|
||||||
|
|
||||||
|
What does the feature do? Well, it just changes the default seccomp profile from
|
||||||
|
`Unconfined` to `RuntimeDefault`. If not specified differently in the pod
|
||||||
|
manifest, then the feature will add a higher set of security constraints by
|
||||||
|
using the default profile of the container runtime. These profiles may differ
|
||||||
|
between runtimes like [CRI-O][crio] or [containerd][ctrd]. They also differ for
|
||||||
|
its used hardware architectures. But generally speaking, those default profiles
|
||||||
|
allow a common amount of syscalls while blocking the more dangerous ones, which
|
||||||
|
are unlikely or unsafe to be used in a containerized application.
|
||||||
|
|
||||||
|
[crio]: https://github.com/cri-o/cri-o/blob/fe30d62/vendor/github.com/containers/common/pkg/seccomp/default_linux.go#L45
|
||||||
|
[ctrd]: https://github.com/containerd/containerd/blob/e1445df/contrib/seccomp/seccomp_default.go#L51
|
||||||
|
|
||||||
|
### Enabling the feature
|
||||||
|
|
||||||
|
Two kubelet configuration changes have to be made to enable the feature:
|
||||||
|
|
||||||
|
1. **Enable the feature** gate by setting the `SeccompDefault=true` via the command
|
||||||
|
line (`--feature-gates`) or the [kubelet configuration][kubelet] file.
|
||||||
|
2. **Turn on the feature** by enabling the feature by adding the
|
||||||
|
`--seccomp-default` command line flag or via the [kubelet
|
||||||
|
configuration][kubelet] file (`seccompDefault: true`).
|
||||||
|
|
||||||
|
[kubelet]: /docs/tasks/administer-cluster/kubelet-config-file
|
||||||
|
|
||||||
|
The kubelet will error on startup if only one of the above steps have been done.
|
||||||
|
|
||||||
|
### Trying it out
|
||||||
|
|
||||||
|
If the feature is enabled on a node, then you can create a new workload like
|
||||||
|
this:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Pod
|
||||||
|
metadata:
|
||||||
|
name: test-pod
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: test-container
|
||||||
|
image: nginx:1.21
|
||||||
|
```
|
||||||
|
|
||||||
|
Now it is possible to inspect the used seccomp profile by using
|
||||||
|
[`crictl`][crictl] while investigating the containers [runtime
|
||||||
|
specification][rspec]:
|
||||||
|
|
||||||
|
[crictl]: https://github.com/kubernetes-sigs/cri-tools
|
||||||
|
[rspec]: https://github.com/opencontainers/runtime-spec/blob/0c021c1/config-linux.md#seccomp
|
||||||
|
|
||||||
|
```bash
|
||||||
|
CONTAINER_ID=$(sudo crictl ps -q --name=test-container)
|
||||||
|
sudo crictl inspect $CONTAINER_ID | jq .info.runtimeSpec.linux.seccomp
|
||||||
|
```
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
{
|
||||||
|
"defaultAction": "SCMP_ACT_ERRNO",
|
||||||
|
"architectures": ["SCMP_ARCH_X86_64", "SCMP_ARCH_X86", "SCMP_ARCH_X32"],
|
||||||
|
"syscalls": [
|
||||||
|
{
|
||||||
|
"names": ["_llseek", "_newselect", "accept", …, "write", "writev"],
|
||||||
|
"action": "SCMP_ACT_ALLOW"
|
||||||
|
},
|
||||||
|
…
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
You can see that the lower level container runtime ([CRI-O][crio-home] and
|
||||||
|
[runc][runc] in our case), successfully applied the default seccomp profile.
|
||||||
|
This profile denies all syscalls per default, while allowing commonly used ones
|
||||||
|
like [`accept`][accept] or [`write`][write].
|
||||||
|
|
||||||
|
[crio-home]: https://github.com/cri-o/cri-o
|
||||||
|
[runc]: https://github.com/opencontainers/runc
|
||||||
|
[accept]: https://man7.org/linux/man-pages/man2/accept.2.html
|
||||||
|
[write]: https://man7.org/linux/man-pages/man2/write.2.html
|
||||||
|
|
||||||
|
Please note that the feature will not influence any Kubernetes API for now.
|
||||||
|
Therefore, it is not possible to retrieve the used seccomp profile via `kubectl`
|
||||||
|
`get` or `describe` if the [`SeccompProfile`][api] field is unset within the
|
||||||
|
`SecurityContext`.
|
||||||
|
|
||||||
|
[api]: https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context-1
|
||||||
|
|
||||||
|
The feature also works when using multiple containers within a pod, for example
|
||||||
|
if you create a pod like this:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Pod
|
||||||
|
metadata:
|
||||||
|
name: test-pod
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: test-container-nginx
|
||||||
|
image: nginx:1.21
|
||||||
|
securityContext:
|
||||||
|
seccompProfile:
|
||||||
|
type: Unconfined
|
||||||
|
- name: test-container-redis
|
||||||
|
image: redis:6.2
|
||||||
|
```
|
||||||
|
|
||||||
|
then you should see that the `test-container-nginx` runs without a seccomp profile:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo crictl inspect $(sudo crictl ps -q --name=test-container-nginx) |
|
||||||
|
jq '.info.runtimeSpec.linux.seccomp == null'
|
||||||
|
true
|
||||||
|
```
|
||||||
|
|
||||||
|
Whereas the container `test-container-redis` runs with `RuntimeDefault`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo crictl inspect $(sudo crictl ps -q --name=test-container-redis) |
|
||||||
|
jq '.info.runtimeSpec.linux.seccomp != null'
|
||||||
|
true
|
||||||
|
```
|
||||||
|
|
||||||
|
The same applies to the pod itself, which also runs with the default profile:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo crictl inspectp (sudo crictl pods -q --name test-pod) |
|
||||||
|
jq '.info.runtimeSpec.linux.seccomp != null'
|
||||||
|
true
|
||||||
|
```
|
||||||
|
|
||||||
|
### Upgrade strategy
|
||||||
|
|
||||||
|
It is recommended to enable the feature in multiple steps, whereas different
|
||||||
|
risks and mitigations exist for each one.
|
||||||
|
|
||||||
|
#### Feature gate enabling
|
||||||
|
|
||||||
|
Enabling the feature gate at the kubelet level will not turn on the feature, but
|
||||||
|
will make it possible by using the `SeccompDefault` kubelet configuration or the
|
||||||
|
`--seccomp-default` CLI flag. This can be done by an administrator for the whole
|
||||||
|
cluster or only a set of nodes.
|
||||||
|
|
||||||
|
#### Testing the Application
|
||||||
|
|
||||||
|
If you're trying this within a dedicated test environment, you have to ensure
|
||||||
|
that the application code does not trigger syscalls blocked by the
|
||||||
|
`RuntimeDefault` profile before enabling the feature on a node. This can be done
|
||||||
|
by:
|
||||||
|
|
||||||
|
- _Recommended_: Analyzing the code (manually or by running the application with
|
||||||
|
[strace][strace]) for any executed syscalls which may be blocked by the
|
||||||
|
default profiles. If that's the case, then you can override the default by
|
||||||
|
explicitly setting the pod or container to run as `Unconfined`. Alternatively,
|
||||||
|
you can create a custom seccomp profile (see optional step below).
|
||||||
|
profile based on the default by adding the additional syscalls to the
|
||||||
|
`"action": "SCMP_ACT_ALLOW"` section.
|
||||||
|
|
||||||
|
- _Recommended_: Manually set the profile to the target workload and use a
|
||||||
|
rolling upgrade to deploy into production. Rollback the deployment if the
|
||||||
|
application does not work as intended.
|
||||||
|
|
||||||
|
- _Optional_: Run the application against an end-to-end test suite to trigger
|
||||||
|
all relevant code paths with `RuntimeDefault` enabled. If a test fails, use
|
||||||
|
the same mitigation as mentioned above.
|
||||||
|
|
||||||
|
- _Optional_: Create a custom seccomp profile based on the default and change
|
||||||
|
its default action from `SCMP_ACT_ERRNO` to `SCMP_ACT_LOG`. This means that
|
||||||
|
the seccomp filter for unknown syscalls will have no effect on the application
|
||||||
|
at all, but the system logs will now indicate which syscalls may be blocked.
|
||||||
|
This requires at least a Kernel version 4.14 as well as a recent [runc][runc]
|
||||||
|
release. Monitor the application hosts audit logs (defaults to
|
||||||
|
`/var/log/audit/audit.log`) or syslog entries (defaults to `/var/log/syslog`)
|
||||||
|
for syscalls via `type=SECCOMP` (for audit) or `type=1326` (for syslog).
|
||||||
|
Compare the syscall ID with those [listed in the Linux Kernel
|
||||||
|
sources][syscalls] and add them to the custom profile. Be aware that custom
|
||||||
|
audit policies may lead into missing syscalls, depending on the configuration
|
||||||
|
of auditd.
|
||||||
|
|
||||||
|
- _Optional_: Use cluster additions like the [Security Profiles Operator][spo]
|
||||||
|
for profiling the application via its [log enrichment][logs] capabilities or
|
||||||
|
recording a profile by using its [recording feature][rec]. This makes the
|
||||||
|
above mentioned manual log investigation obsolete.
|
||||||
|
|
||||||
|
[syscalls]: https://github.com/torvalds/linux/blob/7bb7f2a/arch/x86/entry/syscalls/syscall_64.tbl
|
||||||
|
[spo]: https://github.com/kubernetes-sigs/security-profiles-operator
|
||||||
|
[logs]: https://github.com/kubernetes-sigs/security-profiles-operator/blob/c90ef3a/installation-usage.md#record-profiles-from-workloads-with-profilerecordings
|
||||||
|
[rec]: https://github.com/kubernetes-sigs/security-profiles-operator/blob/c90ef3a/installation-usage.md#using-the-log-enricher
|
||||||
|
[strace]: https://man7.org/linux/man-pages/man1/strace.1.html
|
||||||
|
|
||||||
|
#### Deploying the modified application
|
||||||
|
|
||||||
|
Based on the outcome of the application tests, it may be required to change the
|
||||||
|
application deployment by either specifying `Unconfined` or a custom seccomp
|
||||||
|
profile. This is not the case if the application works as intended with
|
||||||
|
`RuntimeDefault`.
|
||||||
|
|
||||||
|
#### Enable the kubelet configuration
|
||||||
|
|
||||||
|
If everything went well, then the feature is ready to be enabled by the kubelet
|
||||||
|
configuration or its corresponding CLI flag. This should be done on a per-node
|
||||||
|
basis to reduce the overall risk of missing a syscall during the investigations
|
||||||
|
when running the application tests. If it's possible to monitor audit logs
|
||||||
|
within the cluster, then it's recommended to do this for eventually missed
|
||||||
|
seccomp events. If the application works as intended then the feature can be
|
||||||
|
enabled for further nodes within the cluster.
|
||||||
|
|
||||||
|
## Conclusion
|
||||||
|
|
||||||
|
Thank you for reading this blog post! I hope you enjoyed to see how the usage of
|
||||||
|
seccomp profiles has been evolved in Kubernetes over the past releases as much
|
||||||
|
as I do. On your own cluster, change the default seccomp profile to
|
||||||
|
`RuntimeDefault` (using this new feature) and see the security benefits, and, of
|
||||||
|
course, feel free to reach out any time for feedback or questions.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
_Editor's note: If you have any questions or feedback about this blog post, feel
|
||||||
|
free to reach out via the [Kubernetes slack in #sig-node][slack]._
|
||||||
|
|
||||||
|
[slack]: https://kubernetes.slack.com/messages/sig-node
|
||||||
Loading…
Reference in New Issue