IO virtualization is the most complex part of computer virtualization because it involves the coordination of CPUS, operating systems, hypervisors, and I/O devices. IO virtualization has also undergone a shift from software simulation virtualization and class virtualization to full hardware virtualization.
1.IO software simulation virtualization and class virtualization
In AN I/O device virtualization scenario, you need to focus on both I/O device simulation and interaction between vcpus and virtual I/O devices. Many conditions are intertwined, making the whole problem very complicated. The PERFORMANCE cost of I/O virtualization is mainly reflected in three aspects: the cost of the driver accessing the device register; The cost of device access to the driver through interrupts and DMA; The cost of device simulation itself. Therefore, I/O virtualization performance is optimized from the following aspects:
Reduce the cost of IO access to registers:
- On the one hand, part of the IO access to MMIO access, so that you do not need to be trapped in the Hypervisor; The other side is the cost of optimizing the VM-exit/VM-entry switch.
- Reduce I/O access times. For example, simplify the notification mechanism and virtualization device functions.
- Optimized interrupts: Mainly interrupt hardware virtualization like APIC or polling drivers that do not require interrupts.
- Reduce the cost of DMA access: Implement Pass Through mode via IOMMU etc.
- Reduce the cost of device simulation: the hardware device is mainly realized through the hardware SR-IOV mechanism.
As shown in Figure 3.30(a), the device seen in the virtual machine is typically modeled by the Hypervisor. Virtual devices can be less or more functional than physical devices, and can even simulate some non-existent features and non-existent hardware devices. Through the IO software simulation approach, we call it IO device software simulation virtualization. In the IO software simulation virtualization solution, the client VM uses the underlying hardware resources, and the Hypervisor is required to intercept each request instruction and simulate the behavior of those instructions. We all know that the Hypervisor intercepts instructions from VM-exit, processing simulation and then VM-entry. This process is very expensive, and the performance cost of each instruction must be very large.
As shown in Figure 3.30(b), Virtio provides a virtual-like mode in which the client completes the front-end driver of the device and the Hypervisor completes the corresponding back-end driver with the client. In this way, efficient virtualization process can be realized through an interactive mechanism between the two.
Figure 3.30 I/O device virtualization
The Virtio framework, as shown in Figure 3.31, uses Virtqueue to implement its IO mechanism. Each Virtqueue is a Queue carrying a large amount of data. VRing is the specific implementation of Virtqueue, and there will be corresponding descriptor table for VRing description. Virtio is a general driver and device interface framework, based on Virtio respectively implemented virtio-net, Virtio-BLk, virtio-scsi and many different types of analog devices and device drivers.
Figure 3.31 Virtio framework
The performance advantage of Virtio virtualization over traditional IO Device software simulation is that much of the control and state information does not need to be exchanged through register reads and writes. Instead, it is written to the relevant data structures in the Virtqueue to allow drivers and devices to interact. In data interaction, the driver notifies the device by writing the Kick register, and the device notifies the driver by interrupting.
2.I/O is fully hardware virtualization
Evaluate the performance and universality of IO virtualization technology.
Performance, of course, is closer to the NON-virtualization environment I/O performance is the best; The commonality is that the MORE transparent IO virtualization is to the guest operating system, the better. The most straightforward way to achieve high performance is to let clients use real hardware directly; To be universal, you need to find a way for the client operating system’s native drivers to discover and manipulate the device.
The client directly manipulating the device faces two problems:
- First, how to give the client direct access to the device’s real IO address space (including IO and MMIO);
- Second, how to give the device’s DMA direct access to the client’s memory space.
The EPT technology for memory hardware virtualization solves the first problem. Vt-d solves the second problem. Vt-d technology mainly introduces address remapping (IOMMU+IOTLB), which is responsible for providing remapping and direct device allocation. DMA access from the device goes into address remapping for address translation so that the device can access the memory region specific to the corresponding client.
Vt-d technology can transparently transmit physical I/O devices to VMS. However, a computer system is limited by interfaces and can connect only a limited number of physical devices. Therefore, PCIe SR-IOV technology came into being. Using the PCIe SR-IOV technology, one physical I/O device can create multiple virtual devices and assign them to VMS.
As shown in Figure 3.30(c), SR-IOV introduces two PCIe function types:
- Physical Functions (PFs) : manages all PCIe devices, including sr-IOV.
- Virtual Functions (VFs) : lightweight PCIe devices that perform only required configurations and transfer data.
The Hypervisor assigns the VF to VMS and directly transfers data between VMS and hardware devices through DMA data mapping provided by hardware assistive technologies such as IOMMU.
3.IO Virtualization summary
The IO virtualization technology is summarized in terms of compatibility, performance, cost, and scalability. For details, see Table 3.5.
Table 3.5 Comparison of DIFFERENT I/O virtualization modes
How can the software and hardware of cloud computing mix more efficiently?
How will the data center of the future be built? Let’s learn about Fusion of Hardware and Software.
