Yenya's World

Mon, 19 Feb 2007

Virtualization Overhead

In a followup to my previous article I want to sum up the speed penalty introduced by the two virtualization systems, KVM and lguest. The measurements are by no means meaningful for everyone, just exact data I have measured. And yes, it sometimes compares apples-to-oranges - read on at your own risk :-)

The test systems were the following:

Dual AMD Opteron 2220 (4 cores total, 2.8GHz), 6GB RAM, 4-way RAID-10 disk, gigabit LAN, Gentoo Linux/AMD64.
KVM-based virtual AMD64 machine running at Pyrrha, 10 GB disk image, 512 MB RAM, bridged LAN to the main Pyrrha's LAN, Fedora 6/AMD64.
Pentium 4 3.0GHz HT, 1 GB RAM, single SATA drive, 100baseTx LAN, Fedora Rawhide/IA32.
lguest-based virtual IA32 machine running at Scylla, 10 GB disk image, 512 MB RAM, routed/masqueraded LAN to the main Scylla's LAN, Fedora 6/IA32.
Test Pyrrha Terminus KVM overhead Scylla Glauke lguest overhead
bc 6.286 5.876 -6.52 % 8.130 8.240 1.35 %
wget 0.441 10.885 2368.25 % 3.732 3.770 1.02 %
tar unpack 15.118 20.322 34.42 % 27.566 40.701 47.65 %
rm -rf 0.538 0.634 17.84 % 0.477 0.640 34.17 %
compile 6.410 21.929 242.11 % 126.005 184.833 46.69 %

The numbers in the above table are in seconds (so lower is better). I ran each test five times and used the lowest time from these five runs. I did not bother to reboot between the tests or switch the system daemons off.

Description of the tests

time sh -c 'echo 2^1000000|bc >/dev/null'
A simple test of a CPU-intensive job. Why is Terminus faster than Pyrrha? Maybe the clock skew inside the guest? Or Gentoo-compiled bc being slower than Fedora-prebuilt one?
time wget -q -O /dev/null
Network performance. KVM (having to emulate the real-world NIC) is waaay slower. However, Pyrrha has a gigabit NIC, so the baseline is 10 times off. But still, raw bandwidth used for KVM was ~22 Mbit/s, while lguest has filled the Scylla's 100 Mbit pipe without trouble. lguest could be even faster in the future, if they use a bounce buffer bigger than a single page (which is what they use now).
tar unpack and rm
time tar xjf /usr/src/linux-2.6.20.tar.bz2 ; time rm -rf linux-2.6.20.tar.bz2
A simple filesystem-related test. Nothing to see here (KVM is a bit faster).
make clean; time make modules > /dev/null
A simple part of kernel compile. Both the architecture and kernel config was different between Pyrrha+Terminus and Scylla+Glauke, so do not try to compare the absolute times between those two groups. Interestingly enough, KVM was much slower than lguest.

From a subjective point of view, lguest feels faster, which was totally unexpected. I am looking forward to the further development (especially lguest with AMD64 support). Anyway, James Morris did a more in-depth measurements of the lguest network performance.

Section: /computers (RSS feed) | Permanent link | 3 writebacks

3 replies for this story:

Adelton wrote: Paravirt being faster unexpectedly ...

I didn't do any exact measures but having worked with both fullvirt and paravirt Xen guests in the last few months, indeed, paravirt feels faster for similar workloads. My naive internal explanation is that paravirtualized kernel knows how to be nice to the hypervisor, while the vanilla kernel does not case and the hypervisor has to try harder. The nice thing about Xen is (besides it being in FC6) that you can easily switch from paravirtual to fully virtual guests -- just boot different kernel from the grub menu.

Adelton wrote: vmware

I don't really know whether you're looking for some particular virtualization solution, but you should definitely check vmware as well -- the server is now free as in beer and having the full computer together with BIOS and PXE boot is just nice.

Yenya wrote: Re: (Adelton)

Yes, when I think about it, it looks obvious that paravirtualization should be faster than full virtualization. As for vmware - from my point of view it is even worse than Xen, being available without source code. Xen has at least some prospect of being included in the vanilla kernel.

Reply to this story:

URL/Email: [http://... or mailto:you@wherever] (optional)
Title: (optional)
Key image: key image (valid for an hour only)
Key value: (to verify you are not a bot)


Yenya's World: Linux and beyond - Yenya's blog.


RSS feed

Jan "Yenya" Kasprzak

The main page of this blog



Blog roll:

alphabetically :-)