A Virtual Machine Monitor for modern Cloud workloads. Features include CPU, memory and device hotplug, support for running Windows and Linux guests, device offload with vhost-user and a minimal compact footprint. Written in Rust with a strong focus on security.
Bot releases are visible (Hide)
Published by github-actions[bot] almost 2 years ago
This release has been tracked in our roadmap
project as iteration
v29.0. The following user visible changes have been made:
On x86-64
the binary included in releases supports both the KVM and MSHV
hypervisor with runtime detection to identify the correct hypervisor to use.
Improvements have been made to the data structures used for both live migration
and snapshot/restore. Unfortunately this has broken compatibility with older
versions (support for migrating between major versions is not yet officially
supported but has worked for some versions.)
Improvements have been made to the volume of heap allocations when running with
virtio-block
devices along with a reduction in the peak heap size.
ch-remote
ImprovementsSupport for "pinging" the VMM and shutting the VMM down have been added to
ch-remote
.
AArch64
Documentation IntegrationThe documentation for AArch64
support has been integrated into the main
README.
virtio-block
Counters EnhancementThe counters for the virtio-block
device has extended to include min/mean/max
latency counters.
The virtio-net
device has gained support for controlling the enabling of
checksum and offloading. This allows the device to be used in environments
where the hardware lacks support for the offloading.
linux-loader
that addresses anvirtio-net
including an integer overflow issuecpuid
information for L2 cache for older KVM on x86 (#4920)virtio
devices are now shutdown on reboot (#5095)No functionality has been removed in this release.
MemoryZoneConfig::file
orMemoryConfig::file
has been deprecated. This was originally used forMany thanks to everyone who has contributed to our release:
Published by github-actions[bot] almost 2 years ago
This is a bug fix release. The following issues have been addressed:
linux-loader
thatvirtio-net
including an integer overflow issuecpuid
information for L2 cache for older KVM on x86 (#4920)Published by github-actions[bot] almost 2 years ago
This release has been tracked in our new roadmap project as iteration v28.0.
Just a reminder that we have a new mailing list to support broader community discussions. Please consider subscribing. We plan to use this to announce a regular meeting for those interested in talking about Cloud
Hypervisor development.
This is the first version of Cloud Hypervisor to be released under the LTS release process. Point releases for bug fixes will be made for the next 18 months; live migration and live upgrade will be supported between the point releases of the LTS.
Support for adding an emulated CRB TPM has been added. This has it's own TPM documentation.
By default, but controllable through --memory thp=off
if it possible to back the guest memory with Transparent Huge Pages (no file backing/shared=off
) then this will be used resulting in improved boot performance.
The README has been refreshed with the quick start guide updated to reflect the different firmware options and to recommend the use of pre-built binaries.
virtio
device bug fixes found through fuzzing (#4859, #4799)The following functionality has been removed:
Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] about 2 years ago
This release has been tracked in our new roadmap
project as iteration
v27.0.
A new mailing list has been created to support broader community discussions.
Please consider subscribing; an
announcement of a regular meeting will be announced via this list shortly.
Prebuilt packages are now available. Please see this
document
on how to install. These packages also include packages for the different
firmware options available.
The MTU for the TAP device associated with a virtio-net
device is now exposed
to the guest. If the user provides a MTU with --net mtu=..
then that MTU is
applied to created TAP interfaces. This functionality is also exposed for
vhost-user-net
devices including those created with the reference backend
(#4658, #4676.)
Support for generating a trace report for the boot time has been added
including a script for generating an SVG from that trace (#4659.)
The set of feature flags, for e.g. experimental features, have been simplified:
msvh
and kvm
features provide support for those specific hypervisorskvm
enabled by default),tdx
provides support for Intel TDX; and although there is no MSHV supportmshv
feature (#4696,)tracing
adds support for boot tracing,guest_debug
now covers both support for gdbing a guest (formerly gdb
The following feature flags were removed as the functionality was enabled by
default: amx
, fwdebug
, cmos
and common
(#4679, #4632.)
AArch64 has gained support for loading the guest kernel asynchronously like
x86-64. (#4538)
GDB stub support (accessed through --gdb
under guest_debug
feature) is now
available on AArch64 as well as as x86-64.
virtio-queue
that addresses an issuevirtio-console
and serial devicesDeprecated features will be removed in a subsequent release and users should
plan to use alternatives.
Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] about 2 years ago
This release has been tracked through the v26.0
project.
--platform
--platform
and the appropriate API structure has gained support for supplying
OEM strings (primarily used to communicate metadata to systemd in the guest)
(#4319, #4446) and support for specifying the UUID (#4389.)
Support for both the MSHV and KVM hypervisors can be compiled into the same
binary with the detection of the hypervisor to use made at runtime.
SIGINT
and SIGTERM
signals are now handled before the VM has bootedvirtio-fs
driver in OVMF to be used (#4341, #4314)Deprecated features will be removed in a subsequent release and users should
plan to use alternatives.
kernel
and initramfs
members on the VmConfig
have beenPayloadConfig
as the payload
member. The OpenAPI documentThe following functionality has been removed:
poll_queue
parameter has been removed from --disk
andvhost-user-block
Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] over 2 years ago
This release has been tracked through the v25.0
project.
ch-remote
ImprovementsThe ch-remote
command has gained support for creating the VM from a JSON
config and support for booting and deleting the VM from the VMM.
Under the guest_debug
feature flag it is now possible to extract the memory
of the guest for use in debugging with e.g. the crash
utility. (#4012)
IFF_RUNNING
on TAP devices (#4279)The following functionality has been removed:
mergeable
option from the virtio-pmem
support has been removeddax
option from the virtio-fs
support has been removed (#3889)Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] over 2 years ago
This release has been tracked through the v24.0
project.
virtio-iommu
virtio-iommu
specification describes how a device can be attached by default
to a bypass domain. This feature is particularly helpful for booting a VM with
guest software which doesn't support virtio-iommu
but still need to access
the device. Now that Cloud Hypervisor supports this feature, it can boot a VM
with Rust Hypervisor Firmware or OVMF even if the virtio-block
device exposing
the disk image is placed behind a virtual IOMMU.
Multiple checks have been added to the code to prevent devices with identical
identifiers from being created, and therefore avoid unexpected behaviors at boot
or whenever a device was hot plugged into the VM.
Sparse mmap support has been added to both VFIO and vfio-user devices. This
allows the device regions that are not fully mappable to be partially mapped.
And the more a device region can be mapped into the guest address space, the
fewer VM exits will be generated when this device is accessed. This directly
impacts the performance related to this device.
A new serial_number
option has been added to --platform
, allowing a user to
set a specific serial number for the platform. This number is exposed to the
guest through the SMBIOS.
Deprecated features will be removed in a subsequent release and users should
plan to use alternatives
mergeable
option from the virtio-pmem
support has been deprecateddax
option from the virtio-fs
support has been deprecated (#3889)A new blog post Achieving Bare Metal Performance Within a Virtual
Machine
has been added to the Cloud Hypervisor website.
Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] over 2 years ago
This is a bug fix release. The following issues have been addressed:
virtio-fs
filesystem entries from config on removalvirtio-mem
resize if the guest doesn't activate the devicePublished by github-actions[bot] over 2 years ago
This release has been tracked through the v23.0
project.
A vDPA device has a datapath that complies with the virtio specification but
with a vendor specific control path. The addition of --vdpa
and the REST API
equivalent allows the use of these devices with Cloud Hypervisor.
The list of officially supported and tested OS versions has been updated to
include Ubuntu "jammy" 22.04 and EOLed versions removed.
AArch64
Memory Map ImprovementsThe memory map when running on AArch64
has been improved for the handling of
the UEFI region which means that the booted guest OS now has full access to its
allocated RAM. (#3938)
AMX
SupportUnder a compile time gate of amx
it is possible compile in support for the
AMX
instruction set extension for guest use. This also requires runtime
enabling with --cpu features=amx
.
virtio-mem
(#3883)virtio-iommu
backed PCI segments are now comprehensively placed behind thevirtio-fs
to support direct accessDeprecated features will be removed in a subsequent release and users should
plan to use alternatives
mergeable
option from the virtio-pmem
support has been deprecateddax
option from the virtio-fs
support has been deprecated (#3889)Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] over 2 years ago
This is a bug fix release. The following issues have been addressed:
virtio-net
control queue (#3829)Published by github-actions[bot] over 2 years ago
This is a bug fix release. The following issues have been addressed:
openat()
syscall from seccomp filter (#3609)virtio-net
control queue (#3829)Published by github-actions[bot] over 2 years ago
This release has been tracked through the v22.0
project.
Cloud Hypervisor can now be used as debug target with GDB. This is controlled
by the gdb
compile time feature and details of how to use it can be found in
the gdb
documentation.
virtio-iommu
Backed SegmentsIn order to facilitate hotplug devices that require being behind an IOMMU (e.g.
QAT) there is a new option --platform iommu_segments=<list_of_segments>
that
will place all the specified segments behind the IOMMU.
It is now possible to change the VM configuration (e.g. add or remove devices,
resize) before the VM is booted.
virtio-balloon
Free Page ReportingIf --balloon free_page_reporting=on
is used then the guest can report pages
that is it not using to the VMM. The VMM will then notify the host OS that
those pages are no longer in use and can be freed. This can result in improved
memory density.
Through the use of TD-Shim
lightweight firmware it is now possible to
directly boot into the kernel with TDX. The TDX
documentation
has been updated for this usage.
A PMU is now available on AArch64 for guest performance profiling. This will be
exposed automatically if available from the host.
The documentation is now licensed under the "Creative Commons Attribution 4.0
International" license which is aligned with the project charter under the
Linux Foundation.
virtiofsd
The use of the Rust based virtiofsd
is now recommended and we are no longer testing against the C based "classic"
version.
AF_INET
support (#3785)virtio-balloon
size is now validated against guest RAM size (#3689)virtio-net
device hotplug (#3607)vhost-user
features are correct across migration (#3737)Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] over 2 years ago
This release has been tracked through the v21.0 project.
In order to support fast live upgrade of the VMM an optimised path has been added in which the memory for the VM is not compared from source to destination. This is activated by passing --local
to the ch-remote send-migration
command. This means that the live upgrade can complete in the order of 50ms vs 3s. (#3566)
Due to an issue in the virtio-net
code in 5.14 the recommended Linux kernel is now 5.15. (#3530)
virtio-net
desciptor chain (#3548)direct=on
(O_DIRECT
) can now be used with a guest that makes unaligned accesses (e.g. firmware) (#3587)Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] almost 3 years ago
This is a bug fix release. The following issues have been addressed:
SIGWINCH
handler (for console resize)SIGWINCH
handler was not initialisedvirtio-vsock
blocking issue (#3497)Published by github-actions[bot] almost 3 years ago
This is a bug fix release. The following issues have been addressed:
virtio-net
(#3450)vfio-user
support (#3401)DeviceTree
on restorePublished by github-actions[bot] almost 3 years ago
This release has been tracked through the v20.0
project.
Cloud Hypervisor is no longer limited to 31 PCI devices. For both x86_64
and
aarch64
architectures, it is now possible to create up to 16 PCI segments,
increasing the total amount of supported PCI devices to 496.
For each vCPU, the user can define a limited set of host CPUs on which it is
allowed to run. This can be useful when assigning a 1:1 mapping between host and
guest resources, or when running a VM on a specific NUMA node.
Based on VFIO region capabilities, all regions can be memory mapped, limiting
the amount of triggered VM exits, and therefore increasing the performance of
the passthrough device.
Several sections containing unsafe Rust code have been replaced with safe
alternatives, and multiple comments have been added to clarify why the remaining
unsafe sections are safe to use.
The documentation related to VFIO has been updated while some new documents have
been introduced to cover the usage of --cpus
parameter as well as how to run
Cloud Hypervisor on Intel TDX.
Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] about 3 years ago
This release has been tracked through the v19.0 project.
virtio-console
The PTY support for serial has been enhanced with improved buffering when the
the PTY is not yet connected to. Using virtio-console
with PTY now results in
the console being resized if the PTY window is also resized.
Multiple optimisations have been made to the PCI handling resulting in
significant improvements in the boot time of the guest.
When using the latest TDVF firmware the ACPI tables created by the VMM are now
exposed via the firmware to the guest.
Live migration support has been enhanced to support migration with virtio-mem
based memory hotplug and the virtio-balloon
device now supports live
migration.
virtio-mem
support with vfio-user
The use of vfio-user
userspaces devices can now be used in conjunction with
virtio-mem
based memory hotplug and unplug.
virtio-iommu
A paravirtualised IOMMU can now be used on the AArch64 platform.
Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] about 3 years ago
This release has been tracked through the v18.0 project.
vfio-user
) supportExperimental support for running PCI devices in userspace via vfio-user
has been included. This allows the use of the SPDK NVMe vfio-user
controller
with Cloud Hypervisor. This is enabled by --user-device
on the command line.
vhost-user
devicesDevices exposed into the VM via vhost-user
can now be migrated using the live
migration support. This requires support from the backend however the commonly
used DPDK vhost-user
backend does support this.
Images using the VHDX disk image format can now be used with Cloud Hypervisor.
When running on the MSHV hypervisor it is possible to pass through devices from
the host through to the guest (e.g with --device
)
virtio-mem
The reference Linux kernel we recommend for using with Cloud Hypervisor now supports virtio-mem
on AArch64.
Live migration is now supported when running on the MSHV hypervisor including
efficient tracking of dirty pages.
The CPU topology (as configured through --cpu topology=
) can now be
configured on AArch64 platforms and is conveyed through either ACPI or device
tree.
Use of the ACPI power button (e.g ch-remote --api-socket=<API socket> power-button
)
is now supported when running on AArch64.
--serial pty --console pty
now works correctly (#3012)Many thanks to everyone who has contributed to our release:
Published by github-actions[bot] about 3 years ago
This release has been tracked through the v17.0
project.
The support for ACPI on ARM64 has been enhanced to include support for
specifying a NUMA configuration using the existing control options.
Seccomp
support for MSHV backendThe seccomp
rules have now been extended to support running against the MSHV
hypervisor backend.
macvtap
devicesHotplug of macvtap
devices is now supported with the file descriptor for the
network device if opened by the user and passed to the VMM. The ch-remote
tool supports this functionality when adding a network device.
The SGX support has been updated to match the latest Linux kernel support and
now supports SGX provisioning and associating EPC sections to NUMA nodes.
vhost-user
devicesSupport for handling inflight tracking of I/O requests has been added to the
vhost-user
devices allowing recovery after device reconnection.
vhost-user
devices no longer advertise theVIRTIO_F_RING_PACKED
feature as they are not yet supported in the VMMMany thanks to everyone who has contributed to our release:
Published by github-actions[bot] over 3 years ago
This release has been tracked through the v16.0 project.
The live migration support inside Cloud Hypervisor has been improved with the addition of the tracking of dirty pages written by the VMM to complement the tracking of dirty pages made by the guest itself. Further the internal state of the VMM now is versioned which allows the safe migration of VMs from one version of the VMM to a newer one. However further testing is required so this should be done with care. See the live migration documentation for more details.
vhost-user
supportWhen using vhost-user
to access devices implemented in different processes there is now support for reconnection of those devices in the case of a restart of the backend. In addition it is now possible to operate with the direction of the vhost-user-net
connection reversed with the server in the VMM and the client in the backend. This is aligns with the default approach recommended by Open vSwitch.
Cloud Hypervisor now supports using ACPI and booting from a UEFI image on ARM64. This allows the use of stock OS images without direct kernel boot.
virtio-net
queues than advertised is now supported. This appeared when using OVMF with an MQ enabled device (#2578).virtio
devices Cloud Hypervisor now enforces a minimum vCPU count which ensures that the user will not see adverse guest performance (#2563).The following formerly deprecated features have been removed:
bzImage
CONFIG_PVH=y
.Many thanks to everyone who has contributed to our release including some new faces.