FastNetMon

Friday 10 June 2016

11 reasons why you should avoid OpenVZ and Virtuozzo in any cases!


I want to share few words about me. I'm CTO of FastVPS Eesti OU with ~8 years of overall experience with containers and OpenVZ. I'm author of following tools for OpenVZ:

  1. Ploop userspace (user mode implementation of ploop filesystem)
  2. Fast incremental difference algorithm for ploop images.
  3. Author of script for TC based shaper for OpenVZ
And also, I'm author of 175 bug and feature requests at OpenVZ bug tracker.

From the operational side we have about few hundreds of physical servers with OpenVZ (VZ6 based) with pretty huge density and I'm trying to keep my hands dirty and dig into details of any issues.

What you should expect following technical issues when you are working with OpenVZ:

  1. Very often and repetitive kernel faults for different reasons even on top of top-notch hardware with ECC, redundant power and battery backed RAID's
  2. Huge disk space (up 30-40% in some cases) overuse if you decide to use ploop filesystem (that's true for VZ7 too). 
  3. Pretty often storage faults with complete data corruption if you are using ploop. You could read more details about it here.
  4. If you want to get security updates in timely manner you should not rely on OpenVZ team. You should use some external toolkit for applying patches to running kernel (Ksplice, Kpatch or Kernel Care).
  5. You should expect a lot of issues already fixed in recent kernels because OpenVZ project are relying on already outdated RHEL 6 or RHEL7 (based on top of 3.10 but current version is 4.6) kernel. You definitely know about huge backport work of Red Hat but these kernels are really outdated and you could get significant speedup with update to recent vanilla kernel (as Oracle or Ubuntu already doing). But please keep in mind if you are running OpenVZ (VZ7) you are running already outdated toolkit.
  6. You should expect significant changes without backward compatibility in next major release of OpenVZ (for more details please read more details about prlctl and vzctl). In some nice moment you should rewrite all your integration code from billing, configuration management and support control subsystems.
  7. Very poor QA and issues with pretty obvious things.
  8. Almost each upcoming systemd update in container may require new kernel from OpenVZ / Virtuozzo team. Actually, this updates could break your running kernel. 


If we are speaking about overall experience about project I could share my ten cents below:

  • You should not expect any openness in the decision making of OpenVZ project development. That's still true for day-to-day issues (bug priority) and for helicopter view (strategic planning). That's still commercial company and they will not discuss roadmaps with the community. If they decide to do something they will do it even if community hate their decisions. You should check "simfs deprecation" conversation in OpenVZ Users maillist for details.
  • You should have very experienced system administrator who should dig into details of each issue and system instability. 
We are discussed a lot of issues about OpenVZ project. But I want to highlight few issues abut container aware isolation technologies. We have two options if we want to isolate applications in containers:

  • OpenVZ (VZ6, VZ7)
  • Linux upstream containers (managed by LXC or Docker)

Unfortunately, Linux upstream containers could not run whole OS properly they are pretty insecure in compare with OpenVZ. But if you want to isolate your own applications I want to recommend to use Docker because they have a lot of benefits and haven't almost all OpenVZ issues.

But if you want to isolate whole operating systems (for making VPS service) I want to recommend KVM technology. But KVM has significant overhead in compare with containers! Actually, it's true only partially because Intel have been doing a lot of things for improving performance of hardware based isolation and recent CPU's (Xeon E3, E5) has lightning fast KVM implementation! But please be careful and do not use RHEL 7 (do you remember? It's already outdated and you could host more VM's per host with recent kernel!) and look at vanilla kernel, Oracle kernel or some other project which offer recent and stable kernel.

Finally, I really like container isolation and I will be happy when all mentioned issues will be fixed ;)

Thanks for attention! 

8 comments :

  1. А если не секрет, какую ос порекомендовали из своих предпочтений по хостингу?
    Мы сейчас масштабно используем дебиан, но многие вещи в нем смущают и очень похожи на описанное выше :).

    ReplyDelete
    Replies
    1. Лично я предпочитаю Ubuntu 14.04 либо 16.04 LTS. Но там тоже проблемы с поддержкой пакетов, которые не в "main" - баги в них просто не будут чинить. А Дебиян - да, там плохо со сроком поддержки. Но у Убунты - плюс, что постоянно новые ядра с поддержкой железа.

      Delete
  2. А федору пробовали?

    ReplyDelete
    Replies
    1. Это не серверный дистрибутив все же - слишком короткий срок поддержки.

      Delete
  3. Pavel, What is your opinion on the future of Openvz (7)? Openvz.org site looks stalled (as in no news) since middle of 2016. Only updates for openvz 6 are appearing. But we can't assume that this old site is only about VZ6, since all wiki pages are updated to use prlctl instead of vzctl. Their claims that they "will merge Openvz with Virtuozzo 7" sounds like marketingally correct rephrasing of "we will abandon openvz, you are on your own now". On openvz.org top link to 'Installation' leads to 'Using Virtuozzo in the Vagrant box' - year old image which "does not work from the box". This is very impressing when textbook examples fail. To install "openvz" 7 on centos you can not just replace kernel and install few vz utils - now it is completely separated another RHEL-based distro "Virtuozzo" - you'll need fresh install or convert your system to this mess. (Yes RHEL-based distro and not centos-based.) Fresh install of of VZ7 is never worked for me, because of obscure (as in 'not explained anywhere') hardware requirements of "unrestricted guest".

    ReplyDelete
    Replies
    1. I do not work close with OpenVZ anymore. But my overall experience confirms that OpenVZ 6 is not alive anymore. And OpenVZ is never born.

      Also, community lead of OpenVZ (Kir Kolishkin) recently moved from Parallels/Visuozzo to Docker Inc. So, it's also strong sign.

      Delete
    2. Thanks for your answer!

      Delete

Note: only a member of this blog may post a comment.