Xen: performance of full virtualization vs. paravirtualization

I wanted to check what is the difference in performance between virtualization and paravirtualization using Xen. Before going any further some basic info about paravirtualization and full virtualization is given.

Under paravirtualization the kernel of the guest operating system is modified specifically to run on the hypervisor [from here], where hypervisor is a program that runs directly on the hardware of the host system in ring 0. The task of the hypervisor is to handle resource and memory allocation for the virtual machines in addition to providing interfaces for higher level administration and monitoring tools [from here]. Full virtualization provides support for unmodified guest operating systems. The term unmodified refers to operating system kernels which have not been altered to run on a hypervisor and therefore still execute privileged operations as though running in ring 0 of the CPU. In this scenario, the hypervisor provides CPU emulation to handle and modify privileged and protected CPU operations made by unmodified guest operating system kernels [from here].


To compare the performance of paravirtualized system with full virtualized syststem I created two virtual machines (two domU's). The first one was full virtualization with CentOS 5.2 x86_64, while the second one was paravirtualization with the same CentOS 5.2 x86_64. CentOS's were installed with identical sets of packages and configurations. The only difference between the two, was that one was installed using full virtualization, while the second was installed using paravirtualization. Both were installed with four virtual CPU's and 1GB of RAM. The host (dom0) was CentOS 5.2 x86_64 with 24 CPUs (4 physical Xeon CPUs, each with 6 cores) and 24GB of RAM. It must be noted the my CPUs support hardware virtualization (Intel VT-x). Hardware virtualization leverages virtualization features built into the latest generations of CPUs from both Intel and AMD [from here]. Therefore, the following title of this post could be better: Xen: performance of hardware virtualization vs. paravirtualization. However, I basically refer to the fact that in paravirtualization the kernel of the guest (i.e. domU) is modified, while in full and hardware virtualizations the kernel of the guest is not modified.
To compare the performance the following benchmarks were used: nbench, bench and primes 0.9. nbench and bench are single-threaded benchmarks, while primes is multi-threaded benchmark. Primes benchmark was executed with four CPUs and option #15 (i.e. find all prime number less than 256,000,000). The tests on one virtual machine were performed while the other was not running.

Some results are presented below. Full results are here (PV) and here (FV). Better results between the two virtualization method were marked using bold font. For complicity, results for dom0 are also given. Dom0 was the same CentOS 5.2 x86_64 as was used in virtual machines. However, there were differences in configuration, installed packages and services. In another words, dom0 was little more loaded than any domU. During the tests on dom0, virtual machines were shutdown. The full results for dom0 are here.

nbench results

Index (higher is better)	Full virtualization	Paravirtualization	dom0
MEMORY INDEX	18.892	18.991	21.046
INTEGER INDEX	16.921	17.062	17.969
FLOATING-POINT INDEX	28.499	28.781	29.917

Update 17/03/09: I tested fully virtualized ubuntu-server 8.10 using nbench and got: 19.037,18.017 and 28.756.

Primes results

Time [s] (lower is better)	Full virtualization	Paravirtualization	dom0*
Time	116.423	112.432	104.67

* - four CPUs were used.

Update 17/03/09: I tested fully virtualized ubuntu-server 8.10 using primes, and got 113.297 s.

bench results

Index (higher is better)	Full virtualization	Paravirtualization	dom0
Arithmetic Test (type = double)	409.6	411.6	461.3
Dhrystone 2 without register variables	484.4	489.8	539.2
Process Creation Test [lps]	614.0	3176.7	2855.6
Shell scripts (4 concurrent) [lpm]	190.6	1250.0	1261.3

Conclusions

Paravirtualization is marginally faster than full virtualization in my case. Therefore, from the results obtained it does not appear that paravirtualization exhibits much greater performance over full virtualization. One reason for this may be the fact that my CPU's support hardware virtualization. However, my tests were not full. Things such as network bandwich, disk operations etc. were not tested. More tests is required.