Runtime Kubernetes Files

Kata 1.12.0 is here! It includes several features including a couple of security fixes. Users are encouraged to upgrade to this release.

Security fixes: - Readonly bind-mounts are now mounted read-only on the host. With this fix, mounts are protected at VM boundary not just the guest kernel. If a container escape were to occur, one would be able to write to a directory or file that was mounted read-only. - Certain annotations in kata can be used to execute pre-exiting binaries. This could be used to execute arbitrary binaries with the onus of validating these paths left to the stack about Kata. In this release, we added appropriate validations so that an admin can configure a list of file system paths that can be used to filter annotations that represent valid file names.

Features:

• Added support for getOOMEvent GRPC agent API so OOM events can be retrieved from the agent.
• We now detect and support static ARP entries that may be created by a network plugin.
• Added support to hotplug block and vfio devices in cloud hypervisor.
• Fixes were made to make sure systemd cgroups are detected and handled correctly.
• OpenShift CI enabled on runtime repository.
• Added a debug-only capability to run a debug container in the agent PID namespace.
• Host cpuset support added for cpuset.cpus and cpuset.mems
• Kernel LTS 5.4.60 supported with this release
• Qemu updated to 5.0
• Cloud-hypervisor updated to 0.11.0

agent Changes

Shortlog

5af1d61 release: Kata Containers 1.12.0 8f7c782 release: Kata Containers 1.12.0-rc0 05298d0 github: Remove issue template and use central one 9804b1e device: Generalize PCI paths to any number of bridges 134f55a device: Reorganize TestPciPathToSysfs da4bc1d device: Introduce PciPath type, name things consistently 0eb612f device: Rename and clarify semantics of getDevicePCIAddress 8336b5b action: Improve porting checks 0a4d443 device: Simplify uevent matching in listenToUdevEvents() bd4dcc5 device: Rename pciDeviceMap in sandbox struct 27ebdc9 device: Check type as well as major:minor when looking up devices d88d468 device: Index all devices in spec before updating them a48a062 network: Fix Could not create destination mount point: /etc/resolv.conf 427dc4e action: Require PR porting labels 5cc719a action: Add issue to project and move to "In progress" on linked PR cef0a1e release: Kata Containers 1.12.0-alpha1 02d2f97 oci: Fix running of OCI hooks abb006c RFC: namespaces: Allow container with agent PID namespace 5dc7ae4 device: Ease device access for rootfs device to allow node creation 96d8dd3 actions: Add action to perform checks for pull requests b08eb7e release: Kata Containers 1.12.0-alpha0 c01192e device: Allow to use the predicted 'VmPath' when adding blk devices a88af32 device: Do not allow container access to the nvdimm rootfs 42438f9 network: Add grpc method to add static arp neighbors 756de79 Makefile: do not use LDFLAGS to avoid environment contamination 1eb1abe channel: fix the issue of epoll_wait interrupted by signal 2aa833f agent: add grpc endpoint to retrieve oom events

proxy Changes

Shortlog

27b2fdc release: Kata Containers 1.12.0 f4db666 release: Kata Containers 1.12.0-rc0 16cf58a github: Remove issue template and use central one e3df538 action: Improve porting checks 621fb82 action: Require PR porting labels 7e5a74c action: Fix in progress issue action 7dea9b4 action: Add issue to project and move to "In progress" on linked PR 57e322a release: Kata Containers 1.12.0-alpha1 9953a24 actions: Add action to perform checks for pull requests c9c4883 release: Kata Containers 1.12.0-alpha0

runtime Changes

Shortlog

00ff192c release: Kata Containers 1.12.0 1e6c6967 versions: Update cloud-hypervisor to release v0.11.0 d389fa42 tests: Update assets test to adapt to recent changes fd59f15f makefile: Enable hypervisor annotations by default b6f45c48 config: Rename 'runtime' to 'runtimeConfig' 18d9a1db config: Improve comments in configuration file templates 76a9542c config: Make configuration file comments consistent 40e22634 annotations: Correct unit tests to validate new protections 771865a1 annotations: Split addHypervisorOverrides to reduce complexity d4b8f610 annotations: Add unit test for checkPathIsInGlobs 9b733a9a annotations: Add unit test for regexpContains function ff869d5f runtime: Fix firecracker config 7a6cd2a5 makefile: Add missing generated vars to USER_VARS 622c2885 makefile: Improve names of config entries for annotation checks 90b7cfbd annotations: Give better names to local variabes in search functions 0609d2d7 annotations: Rename checkPathIsInGlobList with checkPathIsInGlobs 179325d4 config: Add better comments in the template files fc300a39 config: Whitelist hypervisor annotations by name b6d4683a config: Use glob instead of regexp to match paths in annotations 8c1199fa annotations: Fix typo in comment a3907283 config: Add makefile variables for path lists 06248125 config: Protect file_mem_backend against annotation attacks 3317bf70 config: Protect vhost_user_store_path against annotation attacks dc97a64f config: Add security warning on configuration examples 99ef2b6a config: Protect ctlpath from annotation attack 0243f409 config: Protect jailer_path annotation b7c8905b config: Add examples for path_list configuration f4dd7298 annotations: Simplify negative logic 75424056 config: Add hypervisor path override through annotations 0330aa07 config: Fix typo in function name 802bc999 config: Protect virtio_fs_daemon annotation 06369f23 config: Add 'List' alternates for hypervisor configuration paths 77399058 runtime: mount shared mountpoint readonly 509eb6f8 runtime: readonly mounts should be readonly bindmount on the host f03db9f8 static-checks: Correct the copyright format 7df99f30 arm64: correct bridge type for QEMUVIRT machine a8e9cff3 gitignore: Ignore cli/containerd-shim-kata-v2/config-generated.go b71211c8 runtime: Ignore ENOENT in kill/delete ebf5f95e runtime: Add s.newStore.Destroy before defer 44871d29 hypervisor: Remove unused methods f8e25a4a annotations: Improve asset annotation handling fb6ca1f9 annotations: Add missing hypervisor control annotation fa02f1b2 asset: Formatting, grammar and whitespace 3add5af9 release: Kata Containers 1.12.0-rc0 3f9f4b80 runtime: Don' call bindUnmountContainerRootfs for devicemapper device cfedf350 runtime: Fix /var/lib/vc/sbs/${sid} dir residual ab7f18d9 hypervisor: don't enforce a minimum memory setting ec964099 shimv2: handle ctx passed by containerd b90babb9 runtime: write oom file to notify CRI-O OOM occurred e5f3b6d3 ci: clear travis config warnings 1e91677e virtiofsd: fix typo in test code 321d28e5 version: upgrade qemu version to v5.1.0 for arm64 2f1219f8 virtiofs: Disable DAX e31c8345 versions: Add newest-version for OpenShift b5b8870e cpuset: don't set cpuset.mems in the guest 18c1a7f7 clh: Support VFIO device unplug 0f758018 clh: Remove unnecessary VmmPing 49bd1625 versions: cloud-hypervisor: Bump to version 6d30fe05 62b0d5ee clh: openapi: Tag the 'openapi-generator-cli' container to v4.3.1 3a1a70c4 github: Remove issue template and use central one 4cfaa8c6 versions: Update CLH to version v0.10.0 a7076083 kata-check: check for newer release 7d3fff46 scripts: Don't use hard-coded crio config 8ef2946c sandbox: consider cpusets if quota is not enforced 0e0ef633 cpuset: support setting mems for sandbox 598b4fe8 ci/openshift-ci: Enable openshift-ci 22d48232 virtcontainers: fix delete sandbox failed problem 67be9265 action: Require PR porting labels 5cb47f2f action: Add issue to project and move to "In progress" on linked PR 0868c2ad virtcontainers: Add unit test for utils/compare.go 227cba6b sandbox: Disconnect from agent after VM shutdown d3690ec1 release: Kata Containers 1.12.0-alpha1 dfb8ed7e clh: Disable the 'seccomp' option temporarily e529c010 kernel: move to the latest LTS kernel 5.4.60 9bb8e36a shimv2: Add a "--version" cli option ad78c6fa build: Fold long clean line 6bf93b23 drivers: Correct isPCIeDevice logic c87ff44f clh: Add some error handling for clh 3a0cd87d shimv2: fix the issue of close IO stream 44b58e41 clh: Add support to unplug block devices 03fb9c50 clh: Set 'Id' explicitly while hotplugging block device 39897867 clh: Provide cpu topology to API 40f49312 clh: opeanapi: update api for cloud hypervisor 0dcbbd8d versions: cloud-hypervisor 0.9.0 d803f077 versions: Update qemu-virtiofs to 5.0 3a4aec15 qemu: add annotations for iommu_platform for s390x virtio devices 9305ef72 vendor: Update govmm for s390x iommu_platform annoations 62529e3b virtcontainers: Add msg to existing utils unit tests 5debe065 virtcontainers: Add to utils unit tests e8e1124b virtcontainers: Add unit test for types/container.go cb49a571 namespace: Allow container to join pid namespace of agent 50085cae vendor: Vendor in github.com/kata-containers/agent a7b98ac4 initrd: Increase Alpine Version to 3.12 a162469c qemu: Set govmmQemu NoReboot config Knob b1cbf833 qemu: Add test for qemuConfig Knobs 0d5c05ea vendor: update govmm 8802bd32 qemu: remove multidev in qemu/fsdev parameter on arm64 1e2a3612 virtcontainers: Expand unit test coverage for asset 18fbde9d virtcontainers: Add function to capabilities test 695fa432 virtcontainers: 9p: shares multiple devices with only one export 50d96b3c vendor: update govmm d889e9cf virtcontainers: Add additional unit tests for sandbox 345d0c2a virtcontainers: Remove duplicate unit tests d2fac4cc virtcontainers: Move unit tests for types/sandbox.go 64bf3fe2 cgroups: remove unused SystemdCgroup variable and accessor/mutators ad5484ba cgroups: Add systemd detection when creating cgroup manager 790951a0 actions: Add action to perform checks for pull requests b8238ce0 versions: Use new kata tag for virtiofs kernel e71b05b9 virtcontainers: Add to bridges unit test 337f2e0c sandbox: Stop and clean up containers that fail to create 0f957fb4 virtcontainers: vhost-user-blk/scsi are block device nodes 8b4c299e sandbox: don't constrain cpus, mem only cpuset, devices 093aaa87 cgroups: add ability to update CPUSet 9fa2bf1c vendor: add cpuset package from kubernetes 1aa0cec2 virtcontainers: add method for calculating cpuset for sandbox e0dc806a shimv2: Removing function as no longer used 624d13d8 shimv2 : Remove workaround for sharedPidNs a3de4520 release: Kata Containers 1.12.0-alpha0 c139a667 versions: update QEMU to 5.0.0 30b40f55 clh: Remove the use of deprecated '--memory file=' parameter e02d5ef7 virtcontainers: print a warning when the device to append is not supported 5fccab74 virtcontainer/cgroup: create cgroup manager after creating the network 3c8c6505 virtcontainers/network: Change signature of Enpoint Attach method 581ff974 drivers: change BindDevicetoVFIO signature 970ef454 device: support vfio cold plug 6532eaa0 device: add ColdPlug flag 26f8c14d vendor: update govmm 53a9d005 virtcontainers: Fix structured logging in cgroups package c51baf8d shimv2: Use BUILDTAGS when building shimv2 651d5ff6 qemu: Fix kernel_irqchip=split option for IOMMU enabled sandbox 364435a6 clh: vsock: Use the updated VsockConfig 17d265af versions: Move to cloud-hypervisor v0.8.0 4ee382cd qemu: Report all errors on virtiofsd execution 5a3b6651 katatestutils: Use the configured virtiofs daemon path 2c342638 virtcontainers: Check the correct error variable for sandbox creation c19daa59 qemu: Fix travis build failure for Power 5d442a28 qemu_arm64: Fix build failure fdcd1f3a qemu: enable iommu on q35 66b54f88 qemu: support appending a vIOMMU device 401ad67c vendor: update govmm to bring iommu support 4645d3e6 virtiofsd: Use cache=auto 9ac39116 cli: Fix kata-env output on Power 6be76fcd kata_agent: Add unit tests 5b96e01f clh: Clear the "PCIAddr" field while blk device hotplug 50c1dce1 kata_agent: Pass "VirtPath" with "PCIAddr" of blk devices to agent aea29b64 kata_agent: Allow to use "VirtPath" as volume source for blk devices e5a3211c clh: Allow add virtiofs args and cache options from config 49ebaa88 virtcontainers: drop deferred func for GetAndSetSandboxBlockIndex 379f19f7 qemu: Fix rtc parameter is not set to qemu 20fe3bb9 shimv2: check correct error variable for deferred func in service#StartShim 54e8fdbc qemu: Fix Qemu binary path for Power across distros e855d8dd github: add auto comment bot a3dec262 vc: make host shared path readonly 1d3e1ea3 qemu: Remove hard-coding of Qemu machine options for ppc64le 67d3e2c5 network: Detect and add static ARP entries 412dcbfd vendor: Update agent to include AddARPNeighbors grpc method 6b32472c qemu: Remove PMU feature for Power (ppc64le) platform e07a932a ci: Do not install virtcontainers with podman clh f76d7391 virtcontainers: GetOOMEvent should have no timeout 5e552720 clh: Set 'virtio-blk' as the default block device driver c5f97b24 clh: Enable disk block device hotplug support 18662e16 qemu: Remove pmu limitation in nested virtualization of amd/ppc64le 41a06d49 build: Add "pmu=off" to default cpu_features option f03c17d1 annotations: add cpu_features 0100af18 qemu: add cpu_features option 0b3a9271 vendor: Update govmm 6c517548 clh: remove slow boot debug flags from kernel cmdline 160e3a7c clh: Remove vsock log port in kernel cmdline e1ee00d1 clh: Improve hypervisor logging 882a8239 virtiofsd: Improve logging 7b269ff7 qemu: Don't leak file descriptors in case of error 6aff0779 virtcontainers: x86: Support microvm machine type c98ef487 vendor: update govmm bec32f61 utils: Fix case version check for stable releases 86f58106 shim: exit out of oom polling if unimplemented b4833a48 virtcontainers: tests fix, nit fix db28dcf2 shim: retrieve oom events after starting sandbox 86686b56 virtcontainers: add support for getOOMEvent agent endpoint to sandbox ef8624bb vendor: update agent 619ada25 clh: vsock: Supply the right VsockConfig to Vmconfig 9dbd9298 versions: Move to cloud-hypervisor v0.7.0 3c4fe035 shm: handle shm mount backed by empty-dir memory volumes 7b5e8f66 clh: memory: remove pmem size argument d4a9282f versions: Move to latest cloud-hypervisor ee985a60 qemu: arm64: Set defaultGICVersion to 3 to limit the max vCPU number 4d4a153a qemu: arm64: Don't detect gic version by /proc/interrupts d0dbd048 virtcontainers: Fix structured logging in device/config package 8d9fa47e virtcontainers: constrain runtime after creating network 017ac55c virtcontainers: update sandbox's device cgroup 1da6f22b virtcontainers: remove all the code related to HasCRIContainerType 389b374e virtcontainers: apply constraints to the sandbox cgroup 6377fc47 pkg/cgroups: update the list of devices for the hypervisor 042e7a20 pkg/cgroups: add methods to add and remove device from the cgroup dc69d6e2 pkg/cgroups: implement functions to get information from a host device eee0b090 device: add GetHostPath() to generic device 23aa94e6 logging: Fix structured logging in store package 868f6871 versions: Remove golangci-lint and gometalinter entries e36389e2 dax: enable dax on arm64 7e470461 vc: Version support check is ineffective in createSandbox c4b5922d versions: Misc changes to descriptions

shim Changes

Shortlog

50e26ea release: Kata Containers 1.12.0 147a3ce release: Kata Containers 1.12.0-rc0 bdc7968 github: Remove issue template and use central one b1f77fa action: Require PR porting labels 01f1f12 action: Add issue to project and move to "In progress" on linked PR f8b3398 release: Kata Containers 1.12.0-alpha1 f5220a8 actions: Add action to perform checks for pull requests 866e33c release: Kata Containers 1.12.0-alpha0

Compatibility with Docker

Kata Containers 1.12.0 is compatible with Docker v18.06-ce

Compatibility with CRI-O

Kata Containers 1.12.0 is compatible with CRI-O [0eec45]

Compatibility with cri-containerd

Kata Containers 1.12.0 is compatible with cri-contaienrd [3a4acf]

OCI Runtime Specification

Kata Containers 1.12.0 support the OCI Runtime Specification v1.0.0-rc5

Compatibility with Kubernetes

Kata Containers 1.12.0 is compatible with Kubernetes 1.17.3-00

Kata Linux Containers image

Agent version: 1.12.0

Default Image Guest OS:

description: | Root filesystem disk image used to boot the guest virtual machine. url: "https://github.com/kata-containers/osbuilder" architecture: aarch64: name: "ubuntu" version: "latest" ppc64le: name: "centos" version: "latest" s390x: name: "ubuntu" version: "latest" x86_64: name: "clearlinux" version: "latest" meta: image-type: "clearlinux"

Default Initrd Guest OS:

description: | Root filesystem initrd used to boot the guest virtual machine. url: "https://github.com/kata-containers/osbuilder" architecture: aarch64: name: "alpine" version: "3.12" ppc64le: name: "alpine" version: "3.12" s390x: name: "alpine" version: "3.12" x86_64: name: "alpine" version: "3.12"

Kata Linux Containers Kernel

Kata Containers 1.12.0 suggest to use the Linux kernel v5.4.60 See the kernel suggested Guest Kernel patches See the kernel suggested Guest Kernel config

Installation

Follow the Kata installation instructions.

Issues & limitations

More information Limitations