Interview: OpenVZ Project Manager Kir Kolyshkin


I had the opportunity to be part of the OpenVZ booth at the recent LinuxWorld Expo in San Francisco where I met the OpenVZ Project Manager, Kir Kolyshkin. He was kind enough to answer some questions for me via email.

About the OpenVZ Project

ML: Please tell me a little bit about yourself... education, hobbies, family... what jobs did have before SWsoft / the OpenVZ project?

Kir KolyshkinKir KolyshkinKir: I graduated in CS from Ukhta State Technical University, it's in the city where I was born and lived, somewhere 1500 km north from Moscow, the capital of Russia where I live now.

While in the university I had a chance to work and play with not only boring DOS/Windows, but also OS/2, HP-UX, SCO, Novell Netware, and some other operating systems, including Linux of course. My first Linux distro was some ancient version of Slackware, I only remember it came with kernel 1.0.9 and the CD also contained patches for up to 1.1.50. I immediately fell in love with Linux and free software model.

Before I became the project manager for OpenVZ, I worked at a few companies, including positions at Deutsche Bank and at the Russian natural gas giant Gazprom. As for SWsoft, I did a few projects there -- a search engine (ASPseek, now mostly in oblivion), a few projects for Virtuozzo (lead development of vzctl, then kernel testing, then template tools).

Now I live in Moscow with my wife and a 6-year-old son who will go to school in a few days. Of course he runs Linux, his favorite game was SuperTuxKart, now he's playing some Wesnoth. Aside from computers, he likes riding his bike, drawing, reading books, going to banya (russian sauna). That covers most of my hobbies as well -- let me just add skiing (cross-country, not mountain) and music (listening only, not playing).

ML: How did you get involved with the OpenVZ project?

Kir: Naturally, as I was working with different Virtuozzo subsystems I knew it rather well, and since I was a known proponent of free software and its philosophy I was invited to lead the project and met that with enthusiasm.

ML: Why did SWsoft decide to create OpenVZ as a separate project?

Kir: First, it was always been there since Virtuozzo (as the core of it is a modification of the Linux Kernel and thus must be under the GPL), it was just not existing as a separate entity, a separate project -- and now it is. Why? Because virtualization, and containers in particular, will become a commodity in a few years and we want to be a part of the process since we do have some expertise. Finally, this is our contribution back to Linux community.

ML: Who picked the name OpenVZ and who created the logo?

Kir: I do not remember who found the name -- a few people discussed different names on the mailing list. We tried a few names like VZOpen and Open Virtuozzo but didn't like them. OpenVZ seemed quite reasonable, being short, unique, and to the point (as we sometimes refer to Virtuozzo as just VZ). Finally, domain was available.

As for the logo, I was in a rush to open the to the world and mocked up a logo using the Inkscape editor in just a couple of hours, using its "star" tool. I chose the same greenish color as the Virtuozzo logo (which has changed since then). I thought it might be a temporary solution before some professional artist came out with something better for us, but apparently people liked it so it stayed :)

As for the green color -- OpenVZ helps to increase server utilization, thus saving power in a datacenter. By not adding much overhead per se it further helps to save more power. It's a green technology with a green logo!

ML: You are using GIT for source code management. Has the OpenVZ project used any other SCMs? Why did you decide to go with GIT?

Kir: We used CVS before (and still use it for some stuff) -- and it's just way too old and inconvenient for the 21st century! GIT is a few orders of magnitude more powerful and pretty damn fast. If you want to know why, see the Linus Torvalds talk about it:

ML: Are there any areas in the OpenVZ project that you wish you had a bunch of volunteers to work on?

Kir: We have already seen some good contributions here and there, but there's always room for more! I would really like people to work more on tools, especially template tools and OpenVZ control libraries (a.k.a. vzctl-lib). A lot of people already contribute OpenVZ templates, and I'd like that to continue with not only OS templates, but also some kind of virtual appliances (i.e. a pre-installed set of applications for a specific purpose, like running a mail server).

I wish we could have some help with the mainstream integration -- if anyone would like to join the fun, start with subscribing to containers-at-linux-foundation-dot-org.

ML: Who is using OpenVZ and why? How many users do you think there are?

Kir: It's hard to know -- OpenVZ has users not customers, so we may not hear from users at all unless they have some problem with or a question about the software. I would guess -- tens of thousands of users. As for the technology itself, we should also include Virtuozzo here which is clearly the number two virtualization solution, seconded only by VMware.

ML: What direction do you see the project heading in the future? Are there any up and coming features we can expect to see in the not-too-distant future?

Kir: The port to the 2.6.22 Linux kernel just went out -- it was our short-term goal for the period. Our long-term goal is integrating OpenVZ into the mainline kernel; so far we have merged some tiny bits a pieces, and more will come soon (network virtualization, beancounters a.k.a. resource management).

ML: An Ubuntu user wanted me to ask if the OpenVZ project would eventually build kernels for the Ubuntu distribution... or if that would be something the Ubuntu project would be more responsible for?

Kir: I know the Ubuntu repository already contains packages for vzctl and vzquota; as for the kernel, one can use the OpenVZ kernel provided by Debian available from Surely it won't hurt to ask Ubuntu/Canonical people to include OpenVZ in Ubuntu.

OpenVZ's Relationship to SWsoft and Virtuozzo

ML: Are all of the core OpenVZ developers employees of SWsoft? How much, if any, development work comes from outside of SWsoft?

Kir: Well, SWsoft's kernel hackers are working on OpenVZ and Virtuozzo at the same time, because Virtuozzo is a superset of OpenVZ.

As for the outside development, we greatly benefit from what you call the "outside" people. Let me use the chance to say thanks to the following people:

Benedikt Boehm, Christian Heim -- Gentoo ebuilds
Ola Lundqvist,Thorsten Schifferdecker -- Debian packages
Dmitry Levin, Konstantin A. Lepikhov -- ALTLinux packages

[img_assist|nid=382|title=|desc=LinuxWorld 2007|link=popup|align=right|width=100|height=75]
A lot of other people are quite active on our support forums and/or mailing lists. Some people provide download mirror services. Some people are blogging We had a great team of OpenVZ users at our booth at LinuxWorld Expo 2007 recently. (Editor's Note: Thanks, we really enjoyed it!)

Having said that, most of the OpenVZ code is the kernel, which is a complex thing, thus there are not that many kernel hackers on this planet. Still though, we had a few OpenVZ bugs fixes and features added by outside people and I hope this will increase in the future.

ML: What features separate OpenVZ and Virtuozzo?

Kir: Virtuozzo is for businesses, while OpenVZ is for geeks, thus the difference: Virtuozzo comes with a bunch of GUI-based and Web-based control panels, while OpenVZ sticks only to the command line tools.

ML: I've often heard that Virtuozzo scales even better and allows for even higher VPS/VE density than OpenVZ. Why is this true and what feature differences between OpenVZ and Virtuozzo account for this?

Kir: There is a VZFS filesystem in Virtuozzo but not in OpenVZ. It is similar to unionfs or aufs, the idea here is to make a single copy of files for applications, binaries, libraries and data. If there is a single copy of such files on disk for different VEs, there will be a single copy in memory holding the read-only code and data. It helps to save some memory and thus increase running VE density a bit -- but only the in cases where all (or most of) the VEs are identical (i.e. have the exactly same versions of programs installed).

So, VZFS is not a silver bullet and it comes with a price -- not conforming to POSIX by adding an ability to have additional permission and ownership attributes for symlinks. Because of that Virtuozzo ships with its own modified versions of tar, cpio, rpm, and deb which are patched to understand the added filesystem semantics. Besides, it complicates template tools. We do not want that sort of complexity in OpenVZ.

ML: How much financial support does SWsoft give to the OpenVZ project and do they ever expect OpenVZ to become a self-sustaining, truly independent project?

Kir: SWsoft sponsors pretty much all of OpenVZ, and there is no goal for the project to become self sustaining. Still, I can not say SWsoft (or anybody else) is controlling OpenVZ -- the code is all licensed under the GNU GPL and free/open to all.

ML: Do you think there is a chance that the OpenVZ community (and/or third-party commercial developers) will start adding features to OpenVZ that make it more competitive with Virtuozzo? ...and if that were to start happening, how would SWsoft react?

Kir: I think that would be just great, and I hope things like that will start happening.

As for the SWsoft, I'm probably not the one to ask, but still: I see SWsoft's future in providing high-level virtualization management and automation tools on top of technologies such as OpenVZ.

The Linux Kernel and Containers

ML: Much of the information I've found on the subject of OS virtualization and containers comes from Jon Corbet's Linux Kernel reports from the Linux Weekly News website. If I understand correctly there are at least three stakeholders in container technologies. Andrew Morton mentioned in his most recent talk about the Linux kernel that the only features he could predict for sure that were coming to the mainline Linux kernel were related to containers and he specifically mentioned the OpenVZ project.

Could you please list the various stake holders in container technology and how they relate to each other? From LWN's coverage I've seen:

IBM - Chandra Seetharaman
Google - Paul Menage
The OpenVZ Project / SWsoft

Kir: There are many more people at IBM from different countries involved in this: Cedric Le Goater, Daniel Lezcano, Dave Hansen, Serge Hallyn, Balbir Singh, Sukadev Bhattiprolu, Srivatsa Vaddagiri, maybe someone else I missed.

Also, a lot of work is done by Eric Biederman, and some wise fixes are coming from Oleg Nesterov. Surely, there are more people involved -- see mailing list archives for more.

ML: Do each of the stake holders have a similar vision for containers or are they working in different areas? Is there much overlap between projects? Do they work together or are they completely independent?

Kir: Indeed, different parties have different views of this or that subtopic of containerization. Still, we all work together on the mailing list, trying to find a common ground and consensus, by exchanging ideas, usage scenarios, actual code (in the form of patch sets). Such collaboration definitely helps to find the solutions all the interesting parties will like.

We will hold a containers mini-summit this September in Cambridge, UK (just before the Linux Kernel Summit which will also be in Cambridge), plus a couple of sections of the Kernel Summit itself will be devoted to containers-type virtualization.

ML: Often the mainline kernel seems to reject patches for a few common reasons:

  1. They are too obtrusive/disruptive and change too much of the kernel at once and need to be rewritten to be more incremental and general purpose,
  2. The kernel developers do not like the technical design of one or more pieces and request it be re-engineered
  3. The patch is very special purpose and is not seen as a feature the general community would be interested in

What if any feedback have you gotten from the higher mainline kernel developers regarding the OpenVZ kernel patches?

Kir: I guess we met all the three reasons above before, but we are adaptive and responsive -- always reacting to the constructive comments in a positive way, fixing the issues, cleaning up the code etc.

ML: Can you update us on the current status of OpenVZ integration into the mainline kernel? Do you expect anything to happen in the near future regarding integration?

Kir: Most notable is the addition of the PID namespace patchset by Pavel Emelyanov into -mm (Andrew Morton's) tree -- it means the code will be in Linus' kernel in a few months. PID namespaces is a feature that makes it possible to have different sets of PIDs in different containers. The code was mostly developed by OpenVZ's Pavel Emelyanov, with some pieces from IBM's Sukadev Bhattiprolu. With the first version sent back in May, it was rewritten a few times to incorporate comments, suggestions and feature requests from everyone who was interested.

This is a good example of how we work together with other teams (this is truly a cooperative work between OpenVZ and IBM, with some help from other parties), and a good example of complexity -- after all, PID namespaces accounts for only about 5% of all OpenVZ code. IPC and utsname() virtualization which have been in the kernel since the release of 2.6.19, both account for another 5% or so. That means we have a long (and exciting!) way to go.

Anther good thing that happened was the acceptance of the user namespace patches from IBM (Serge Hallyn and Cedric Le Goater). This code is much smaller than the PID namespaces code -- what it does is add a basic ability for a container to have its own set of users, including root.

While those two patchsets will appear in the mainstream kernel soon, we have already backported those to our 2.6.22 kernel in order to give this new code a broader testing coverage. That actually helps -- we already found a small bug in the user namespaces code.

As for the future, I think that the rate of patches (and the rate of accepted patches!) will increase. I would guess that some kind of memory accounting (from OpenVZ beancounters) and some kind of network virtualization will be merged in a few months.

ML: If most of the functionality of OpenVZ does end up merging into the mainline Linux kernel, what if any changes to the project do you see happening? Will development be as active or do you anticipate it going into a maintenance only mode?

Kir: Your previous question itself gives us the answer -- I guess there will be some parts of OpenVZ functionality that will never be merged so we will keep maintaining those off the mainline tree. More to say, it's not just a kernel, it's also the tools which I guess will still live at

OS Virtualization vs. Other Methods

ML: Do you think that most users of virtualization products understand the benefits and drawbacks of OS Virtualization vs. Machine Virtualization?

Kir: I'm afraid most people don't, so here's a little explanation.

Virtual Machines (as in Xen, KVM, or VMware) are providing some kind of a virtual hardware, and you can run any OS on top of it. This is flexible, but it comes with a price -- virtualization overhead is still quite high (especially for I/O intensive apps like databases). From the other side, Virtual Environments (a.k.a. containers, as in OpenVZ) are much more efficient, having a native performance and negligible overhead, but from the other side it can only run one OS (in the case of OpenVZ it's Linux, in case the of Solaris Zones it's Solaris, etc.)

Additional reasons to use VEs rather than VMs (not counting the higher VE density and performance) are: dynamic resource management (you can change all of the VEs parameters like RAM limits during runtime) and no issues with scalability (you can make a VE which uses all of the hardware resources, like 64 CPUs and 64 GB of RAM. Besides that, container technology is pretty much hardware-independent and thus are easily portable to other architectures.

ML: Where do you see the future of virtualization going? Do you see any other virtualization types springing up?

Kir: The future of virtualization is in automation and management, i.e. making it simple and transparent to benefit from virtualization, whatever its type. As I said, different types of virtualization are suitable for different workloads and applications -- so there must be a way to manage physical machines, virtual machines and virtual environments in a uniform way not caring about the underlying virtualization technology (if any).

As for the other types of virtualization, there is on-going work on multi-server virtualization (such as in OpenMOSIX or Kerrighed), which is like containers inside out: instead of splitting a single physical server into many smaller virtual servers, such approaches try to consolidate the resources of a few physical servers to provide a bigger virtual server. One can dream about a mix of both technologies -- to combine a few servers into a single one, and then to partition it in any way. For now it looks more like a dream, inefficient from the practical standpoint -- but who knows what will happen in, say, 20 years?

ML: SWsoft is the parent company of Parallels, Inc. Do the Virtuozzo and Parallels teams collaborate at all... and do you think we'll ever see a hybrid product?

Kir: As I said, different virtualization types are for different workloads. As for the hybrid project -- Parallels is developing its server product, and Virtuozzo tools will be able to manage both Virtuozzo VEs and Parallels VMs.

ML: If I understand correctly, the OpenVZ patches do not conflict with the Xen patches... and it is perfectly fine to have a single kernel that supports both Xen and OpenVZ virtual machines... and I know that OpenVZ works fine on a fully virtualized Xen guest. What, if any, practical benefits do you see using the two together?

Kir: So far we have been successful at combining OpenVZ and Xen functionality in one kernel -- we were able to run Xen/OpenVZ kernel in both Xen Dom0 and DomU. That means you can have Xen guests and OpenVZ VEs at the same time on the same server.

From the one side, that means you can benefit from both technologies -- run Linux instances as OpenVZ VEs for better performance and density, and run other OSs as Xen guests. From the other side, there's some overhead even for OpenVZ because it still runs under a Xen hypervisor.

In Closing...

ML: I greatly appreciate you taking the time to answer my questions. Are there any questions I missed... or anything additional you'd like to mention... or address to the OpenVZ community at large?

Kir: If you haven't done it yet -- go to and try this great virtualization technology for yourself, it's really easy to set up and get running. There are many different ways you can benefit from it -- even I may not know all of them.

If you are already using OpenVZ -- please tell me (kir-at-openvz-dot-org) how, I'm really interested in hearing your user story.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Nice Interview

I really respect the developers of OpenVZ and appreciated this review.
I think OpenVZ is still one of the best projects and is even suitable for mission critical applications in my experience.

Outdated info...

"From the other side, Virtual Environments (a.k.a. containers, as in OpenVZ) are much more efficient, having a native performance and negligible overhead, but from the other side it can only run one OS (in the case of OpenVZ it's Linux, in case the of Solaris Zones it's Solaris, etc.)"

Seems like he haven't heard about Solaris BrandZ, one of which is a linux Zone.... also freeBSD and other BSDs can be made into a zone, not only Solaris...

about the relationship between a container and a hypervisor, it's the same that have been happening on solaris with Zones and xVM.. so it's hardly a new idea to have both kinds of technologies on the same kernel :s

VServer isn't exactly

VServer isn't exactly "supremely useful" -- it won't cross your way for the most time but it won't advance you considerably either. At least when you've already had a longer look at OpenVZ.

I've used to run 1.x and 2.4, i586 and x86_64, but moving off to ovz is just so much better. This one does have its troubles (most notable ones for us were around NFS) but these are being taken on and solved, and if you don't happen to run into something really obscure (as they say frequently a bug in kernel proper which didn't manifest in usual circumstances), ovz is clearly better due to its UBC, online quotas and thus far better isolation and provisioning abilities.

As a friend of mine would put such situations, "it's a different kind of sports". Still lightweight one though.

If you want to test the waters without messing with kernels, utilities and so forth, ALT Linux 4.0 Server comes with OpenVZ-enabled kernel, utilities, and web-based container creation/administration tools as well (ML here).

OpenVZ forums being linked to corresponding mailing lists are also robust aides.

Submited by : Dietas

Scott Dowdle's picture

And the winner is?

Your comment has reminded me again that I am planning to write a comparison piece between Linux-VServer and OpenVZ. While there are many similarities between the two... there are also big differences. I must admit that I am more familiar with OpenVZ but I did get into Linux-VServer far enough to realize that it has several plus areas and the two are different enough... with their own pluses and minuses... that there isn't a clear winner.

So, I've made an outline of what I want to write about... now I just have flesh it out. Stay tuned... maybe this weekend.

The list of "stake holders"

The list of "stake holders in container technology" definitely misses VServer, which is the current choice in Debian (there are precompiled kernels with VServer, unlike OpenVZ which is distributed as a separate patch).

Scott Dowdle's picture

Linux-Vserver working with mainline?

I must admit that I'm way behind development news regarding Linux-Vserver... but yes, one would assume that the Linux-Vserver folks would be in there somewhere too. Oddly enough, I got my list of people from stories about containers on Linux Weekly News and information about container patches that have made their way into various Linux kernel developer's trees... and oddly enough, I didn't see Linux-Vserver mentioned anywhere. Not being familiar with their developers (other than the lead who is named Herbert I think?), I'm not sure that I've overlooked anyone... but if so, I certainly didn't do it on purpose.

That begs the question if the Linux-Vserver developers are happy being a separate patch to the mainline or if they want to go through the trouble of trying to get it merged into mainline... and working with the other container folks. Again, I'm ignorant on the subject so I can't speculate.

The fact that the Debian distro developers provide prebuild kernels with Linux-Vserver support doesn't mean anything beyond the availability of .deb packages unless the Debian developers are themselves trying to contribute the patches they apply to their kernels upstream... and I would be very surprised if the maintainer of the Debian Linux-Vserver kernel package(s) were trying to do that... unless they themselves were a member of the Linux-Vserver development team.

As mentioned in the interview, there are pre-built OpenVZ kernel packages for Debian available... but not an official part of the Debian distribution... but like you said... officially the Debian folks are maintaining an OpenVZ patch that applies to their kernels... or at least that is my understanding.

Please feel free, although I realize I'm replying to an anonymous comment, to come back and get me updated on Linux-Vserver development.

I did not meant to blame you

I did not meant to blame you for not mentioning VServer, I just wanted to remind, that VServer is still here and is quite popular (there are many hostings running on VServer), but sadly does not have as much press coverage as OpenVZ.

One specific example: Last year there was an "official" announcement, that OpenVZ is now a part of the Debian distribution. Nobody mentioned that it was packaged just as a patch against the kernel. Nobody wondered that VServer had the whole vserver-patched-kernel packages in the archive for several months already... Yes, it is probably deficiency on the VServer PR department, but still...

About your points:

* Yes, Herbert Poetzl is the current lead. Hint: interview him?

* Yes, VServer developers do not want to maintain the patch forever and are trying to push bits to the kernel, but it is a long and tedious path.

* With the previous point is connected the fact that VServer developers are following kernel development really closely and immediately update the VServer code to use every bit of new container functionality landed in the kernel (no matter whether it was developed by the other parties) and have patches for several current kernels. On the other hand it seems that OpenVZ folks stick with one particular kernel for a while and then skip the line to another chosen-at-that-time kernel.

I do not want to look like I'm just bashing OpenVZ, so I have to mention that OpenVZ has some nice points not found in VServer (quotas, live migration, virtualized network stack), but for some setups they are just overkill (IP aliasing used in VServer instead of full virtualized network stack is much faster...).

Scott Dowdle's picture

Thanks for the follow up... I have a suggestion

I really like to have original content on this site... and I'm really interested in learning more about Linux-Vserver BUT I'm not familiar with it enough to ask intelligent questions.

Would you consider contacting me via email so I have a better idea who you are... and you can contact Herbert (or I can if you want me to) to see if he would be interested in doing an email interview? If you want any help writing the questions, I can lend a hand.

If all goes well, we'll have another interesting content item for this site... and we can help get the word out about Linux-Vserver.

Thanks for both interview and OpenVZ!

Thanks Kirill, thanks ML, this reading has been very interesting to me -- even after using OpenVZ in production for more than a year (we've done things like isolated LTSP servers with it among the rest), and after having heard Kirill amazingly *playing* guitar at LinuxFest even if he'd only mention listening to the music... ;-)

Scott Dowdle's picture

Kir and Kirill

Just to clarify, there are two developers with those names... so when you mentioned that Kirill plays the guitar... I don't think you are talking about Kir.

Hmm, I wonder if Kir plays the guitar?


Unfortunately I do not play guitar, although it was definitely me there on LinuxFest'07.

We were sitting near the fire in the night and the guy sitting next to me was playing guitar, while I was just singing. I wish I could play...

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.