Taking SPICE for a Spin

I finally figured out how to get SPICE working on Fedora 14.

What is SPICE? - It stands for "Simple Protocol for Independent Computing Environments". What does that mean exactly? SPICE is a remote display protocol designed specifically for use with the Linux kernel's built-in virtualization hypervisor KVM. SPICE is similar to terminal services but rather than multiple users sharing a single, remote physical machine, SPICE allows you to graphically connect to and use a local or a remote KVM virtual machine.

For those who want to just watch a video, here it is. Please note that I kept bumping the tripod by accident and autofocus can be annoying in some spots... and it isn't the highest quality... BUT it does give you a good idea of how well SPICE works.

If you can't see it, your browser probably doesn't support the WEBM video format yet. Right-click on any of the links below (webm and ogv) and download. Then play the file you downloaded in a recent version of VLC.

http://www.archive.org/download/TakingSpiceForASpin/spice-for-a-spin.webm
http://www.archive.org/download/TakingSpiceForASpin/spice-for-a-spin.ogv

Here's my setup.

Hardware: Dell Optiplex 755, 2.66GHz Intel Core Duo, ATI Radeon HD2400XT video, Dell 22" widescreen @ 1680x1050
Host OS: Fedora 14 x86_64
Guest OS: Fedora 14 i686

Early Road Blocks

The Fedora 14 SPICE feature page has some instructions but they aren't very detailed. The main issue is that I'm used to using virt-manager for creating and managing KVM virtual machines. SPICE has not yet been integrated into virt-manager so setting it up is quite a manual process. If you are a qemu-kvm command line tool ninja then it helps. I'm not currently one of those.

At first I tried borrowing a command line that virt-manager was generating to start up a VM... and then modifying it to add the SPICE options. There are a ton of options and I'm guessing some of them conflict with each other because I could never get it to work. Finally I just simpled the command line by stripping off a lot of options, making sure I had the important basics like the disk image file, the amount of RAM, etc. Another thing to point out is that the Fedora 14 features page says to use the qemu command with the -enable-kvm option. That did work for me but it was very, very slow. Using qemu-kvm rather than the qemu really makes a huge difference.

qemu-kvm /vm/montanalinux32.img -m 2048 -usbdevice tablet -soundhw ac97 -vga qxl -spice port=5930,password={password}

For comparison, here's the full command line that virt-manager constructs to start the same VM:

/usr/bin/qemu-kvm -S -M pc-0.13 -enable-kvm -m 2048 -smp 2,sockets=2,cores=1,threads=1 -name montanalinux32 -uuid e01b1e3e-9a88-f101-f0ab-87028ff65a5d -nodefconfig -nodefaults -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/montanalinux32.monitor,server,nowait -mon chardev=monitor,mode=readline -rtc base=utc -boot c -drive file=/vm/montanalinux32.img,if=none,id=drive-virtio-disk0,boot=on,format=raw -device virtio-blk-pci,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -device virtio-net-pci,vlan=0,id=net0,mac=52:54:00:78:aa:9d,bus=pci.0,addr=0x3 -net tap,fd=40,vlan=0,name=hostnet0 -chardev pty,id=serial0 -device isa-serial,chardev=serial0 -usb -device usb-tablet,id=input0 -vnc 127.0.0.1:0 -vga cirrus -device AC97,id=sound0,bus=pci.0,addr=0x4 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6

The other part of the instructions that had me lost was this blurb:

This should let you access the machine. You should now install the qxl driver and optionally the agent (only available for windows) in the guest. If you do not do this you're running in vga mode which is quite slow and inefficient.

I saw the xorg-x11-drv-qxl package in Fedora 14 and knew that is what they were talking about, and I had that installed but X.org wasn't using it. Obviously I needed to find a way to generate an /etc/X11/xorg.conf file. X.org hasn't been using a config file for a while. In the past when I needed to generate a xorg.conf file I'd run the system-config-display tool... but that was deprecated some time ago and is no longer available in Fedora 14. Every attempt to create an xorg.conf file by hand resulted in an X.org that wouldn't start up. Then I found a reference that said to use "Xorg -configure". You have to do that from the text console and it worked for me. It even setup the qxl driver by default. After learning that trick it was all down hill.

Working with the SPICE client

The SPICE client (spicec) is a little rough still, at least the Linux version is, because it doesn't have a popup menu and the hotkeys are not obvious. While running spicec without any parameters does give a blank screen with some labeled fields to populate, I'm not sure how functional that is yet. You should run spicec from the command line passing all of the options. The Fedora 14 SPICE feature page instructions worked fine for that.

spicec -h localhost -p 5930 -w {password}

I can't remember where I found them but I'm only aware of two SPICE client hotkeys so far... Shift + F11 will switch between full-screen and windowed mode, and Shift + F12 will release a trapped keyboard/mouse. I'm still not sure how to pass certain keys from within the spice client. For example, how does one get to the virtual consoles? Alt + Control + Fkey should work but that isn't trapped by the client and instead is passed to the underlying physical machine so I get the virtual console of my physical machine rather than that of my VM. I'm guessing there is a way to do it but I'm not sure what that is yet. If anyone knows, please comment.

SPICE Performance

I've used quite a few remote display protocols. The last month or so I've been using a lot of No Machine's NX server/client and was very impressed with it. I use Microsoft's RDP for managing remote Windows boxes. I've been using one flavor or another of VNC since 1998. VNC works fairly well over a LAN but over WAN, it is really slow. I like NX but it is unfortunate that development stalled on FreeNX. Google has a fork of FreeNX but no one seems to be packaging that yet. NX 4.0 looks like it will kick butt but since it is completely closed source... I really wish Red Hat would buy it and free it... because I don't like using proprietary software.

Please note though that I have NOT tried Citrix ICA with the HDX extension... I have NOT tried VMware View... and I have not tried Microsoft's brand new RemoteFX extension to RDP which requires the server have a GPU. Supposedly those protocols offer improved performance too.

Anyway... back to SPICE performance. All I can say is that so far my experience with SPICE has been amazing. While SPICE doesn't offer 100% of the speed of a physical machine, it is the closest of anything I've used thus far. I did not tailor the test environment down at all... still using background images and all of the tool tips and popups offered in KDE 4 and GNOME... again at 1680x1050 widescreen resolution.

Display settings - I was able to adjust the display size easily from both KDE and GNOME's display management apps. I had a wide range of available resolutions including a few larger than my monitor could handle. I adjusted the display to the max value my monitor could handle which was 1680x1050. I only tried a few different resolutions but they all worked as expected.

Keyboard and Mouse - Interactive typing is normal... which is amazing given the slowdowns that happen in almost all other remote display protocols. The mouse pointer is full speed although wheeling around with the mouse wheel is slower than usual and I find myself using the scrollbars more than on a physical host.

Web Browser and Desktop Usage - Text scrolling and browser scrolling, even with images, was full speed. The web browser experience is pretty darn close to that of a physical machine. Moving windows around on the desktop was speedy. GNOME and KDE's panels were full speed. Application menus and dialogs were full speed.

Audio - Works. If the distro you run in your VM supports multiplexed audio, that works too. I tried two audio sources at the same time and they worked fine.

Streaming video - Flash works fine but don't go full-screen. Traditional in-browser sizes are fine. They are a little choppy but really not that bad. A/V sync is actually great so videos are totally watchable. I watched an episode of Glee on Hulu and the playback was good and the A/V sync was near perfect. I tried .ogv and .webm in-browser playback with some videos on this site and they worked ok but seemed more jittery than Flash videos.

Local video - I tried a few .webm videos and they worked fairly well... but again, don't try to go full-screen or it will be frame drop city. All of the videos I tried were 640 in width or smaller and they worked fine... a little choppy but A/V was near perfect. HD video? I don't think it is quite here yet... but then again, I'm not sure my experience was completely optimized.

Other apps - Inkscape worked well. Marble worked very well and it is really slow in all of the other remote display protocols I've tried. The Google Maps website works well including panning and scrolling. OpenOffice.org's window seems to redraw noticeably slower when switching from one virtual desktop without the OOo window and back to the desktop with the OOo window. The only app that seemed to perform unusually slow was GIMP. Not sure why.

Too much stuff? - After I had been in my SPICE session for a couple of hours I got to a point where I had a video playing in totem, a song playing in VLC, and two flash videos going... and audio got very unhappy with me... and my browser froze. There was a period of very choppy interactivity but closing the multimedia apps, including the browser, seemed to fix it. The point here is don't try to over due it or you might run into some issues. Even on a physical box I don't typically watch a local video while playing a song and watching two in-browser videos... so should I expect SPICE to handle that? Probably not. Leason - yes you can overload SPICE if you try, but probably not with regular use patterns.

Over all, I'd give SPICE a 9.5 out of 10. Given my experience thus far, I wouldn't have a problem using it on a daily basis.

Some Remote Connections

After updating my iptables filewall on the physical host I discovered that I could indeed connect to my VMs with SPICE from remote clients. I started up a second VM, which happened to be Windows 7 32-bit. Then I went into a computer lab in the same building, logged into two different lab computers, and installed the SPICE client app on them.

At first I was a bit confused and thought that I'd need to connect to the IP address of each VM but that is NOT the case. The host you connect to is physical host... and you use the appropriate port for whichever machine you want to connect to.

With one client I connected to port 5930 which was the Fedora 14 VM. On the other client I connected to port 5931 which was the Windows 7 VM. Of course these port numbers are arbitrary and are specified by the user when they start the VM.

I spent a little while playing with each VM. I had to adjust the VM displays to 1280x1024 since that was the maximum resolution available from the lab monitors.

The Windows 7 VM wasn't setup completely because it was using the generic VGA display driver rather than the QXL driver. Also the Windows VM didn't like the audio card I had assigned to the VM so sound didn't work. I'm guessing if I specify a different sound device when starting up the VM, I could get wound to work in the Windows VM but I'm not sure what to put there either.

Anyway... the Fedora 14 VM performed just like it did on the localhost. I couldn't really tell a difference.

The Windows VM, given it's shortcomings (generic VGA and no sound) did quite well. I'd say it performed as well as RDP does if not slightly better. Once I get the driver situation figured out, I think it'll get even better.

Here's a second video in .webm format I recorded showing the two remote connections:

Download links:
http://www.archive.org/download/TakingSpiceForASpin2/spice-for-a-spin2.webm
http://www.archive.org/download/TakingSpiceForASpin2/spice-for-a-spin2.ogv

Things I Still Need to Learn

I want to become a qemu-kvm command line ninja because currently I don't know how control the network setup. The simplified qemu-kvm command lines I used to start the Fedora and Windows 7 VMs gave them 10.0.2.x IP addresses. If I use virt-manager and NAT, I get 192.168.122.x IP addresses. virt-manager also makes it easy to use bridged networking so the VMs can use public IP addresses if desired.

I don't know what audio device to specify for Windows VMs.

I'm fairly ignorant when it comes to proxies and connection redirection. I know a lot of that can be done with ssh but what I'd like to understand is how to engineer a reasonable connection broker. One feature that sets the commercial products apart is that they provide a connection broker. For example, at work I've been doing a trial of Virtual Bridge's VERDE. VERDE is a Virtual Desktop Infrastructure (VDI) product that includes a hypervisor platform (KVM on Linux), server clustering and load balancing of VMs, as well as an advanced connection broker providing multiple display protocols for Windows and Linux virtual machines. VERDE virtual desktops can share the physical host's IP address, use NAT, or use a public IP via bridging. It would really be awesome to figure out how to duplicate some of that functionality in a home brew setup.

Looking to the future of SPICE

So far I've just been playing around with a single VM connecting to it with a client on the same physical host... but how does SPICE scale? How much bandwidth does it use? How much CPU? There isn't a whole lot of information available on those subjects yet but there is one study that compares RDP and SPICE which is a fair indicator of how well SPICE performs in its current incarnation. According to the figures provided in that study, it seems that a single GB NIC could provide quite a few remote desktop experiences.

VDI looks quite doable with SPICE. Red Hat already has SPICE deployed in their Red Hat Enterprise Virtualization for Desktops product. Virtual Bridges has VERDE 5.0 due out any time now that adds SPICE. It does appear though that the main target audience for SPICE is on the LAN and not the WAN. We'll have to see how well SPICE holds up over WAN and at the slower speeds as the SPICE components mature.

What is SPICE missing?

Currently SPICE does not have remote USB features but the SPICE developers (mostly from Red Hat) are working on that. RHEV for Desktops and VERDE appear to use an additional protocol for USB device access. SPICE has been developing rapidly and I expect it to have native USB support within 6 months.

I'm guessing a lot of people would like to use SPICE as a general purpose remote display protocol BUT currently SPICE is designed for remote access to KVM virtual machines only. Some people claim that the three prong design of SPICE (client, server, gxl driver in the VM) can't be easily adapted into a general purpose display protocol. I'm not sure if that is true or not. If that turns out to be the case, what does one use for general, non-VMs situations? VNC works well over a LAN but NX works a lot better. It would be so fantastic if NX were made open source but I won't hold my breath for that.

SPICE is here and folks are just starting to use it. I believe that Fedora 15 and Red Hat Enterprise Linux 6.1 will have SPICE features integrated into virt-manager and the various other virt-tools. That will do a lot to increase adoption. I wonder what other distros might have in store. Will they depend solely on the tools that Red Hat is providing or will we see some third-party tools start to appear? We shall see.