Tuesday, May 27, 2014

Install and run a full functional VMware ESX Hypervisor within a KVM virtual machine

This post is about how to setup and configure a VMware ESX Hypervisor within a KVM virtual machine. The resulting ESX Host is going to be fully functional allowing to create (nested) virtual machines with a working network connection. 

Some (funny) facts:

  • Running VMware ESX within a KVM virtual machine allow memory over-commitment. That means even if your KVM Host system only has 2 GB memory you can assign e.g. 4 GB memory to the ESX Host VM.
  • KVM provides the hardware requirements for VMware ESX even if ESX does not supports the KVM Hosts hardware e.g. it allows to install and run a VMware ESX system on almost any kind of Desktop hardware which is normally not supported by ESX e.g. because of an unsupported network card model.

 Steps to setup VMware ESX within a KVM VM

  • Install Linux on a physical system (I used latest Ubuntu 14.04)
  • Install the qemu/kvm package 
                            apt-get install qemu-kvm
  •  Edit /etc/modprobe.d/kvm-intel (or kvm-amd for AMD cpus) and add

                            options kvm ignore_msrs=1
                            options kvm-intel nested=y ept=y
   pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, xen_enabled(),

   pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, TRUE,

  • Configure the qemu 2.0.0 source with the following command:
./configure --enable-kvm --target-list=x86_64-linux-user,x86_64-softmmu
  • Create /usr/local/etc/qemu/bridge.conf
                     allow all
  • Install the bridge-utils package
                    apt-get install bridge-utils
  • Configure a network bridge in /etc/network/interfaces as following:
auto br0
iface br0 inet static
bridge_ports eth0
bridge_fd 0
bridge_hello 2
bridge_maxage 12
bridge_stp off

 Please notice that your ip-configuration may be different
  • Choose a directory for the KVM virtual machine and create the virtual machine harddisk with the following qemu-img command:
                     qemu-img create -f raw esxi00.img 16G
  • Download VMware ESX Hypervisor 5.5 from the VMware website and put the install iso file (VMware-VMvisor-Installer-5.5.0.update01-1623387.x86_64.iso) into the same directory as the harddisk file was created a step before
  • Start the KVM virtual machine with:
/usr/local/bin/qemu-system-x86_64 -enable-kvm -cpu host -m 4096 -smp 2 -cdrom VMware-VMvisor-Installer-5.5.0.update01-1623387.x86_64.iso -netdev tap,helper=/usr/local/libexec/qemu-bridge-helper,id=hostnet0 -device vmxnet3,netdev=hostnet0,id=net0 -hda esxi00.img

Please notice the "vmxnet3" virtual network device type!
  • Go through the (normal) VMware ESX installation procedure
  • After the installation enable the SSH service on the ESX Host with is now running with the KVM VM. Then ssh into the ESX Host and set the following two parameters in /etc/vmware/config:
                      vhv.allow = "TRUE"

Now please reboot the ESX Host system to activate the new configuration!
  • When creating a virtual machine on the (nested virtualized) ESX Host please make sure to add the following two parameters to the VM's vmx configuration file:

                       vhv.enable = "TRUE"
                       vmx.allowNested = "TRUE"


You have successfully virtualized a VMware ESX Hypervisor with KVM!

I hope you now enjoy your fully functional VMware ESX Host

Additional informations on the VMware parameters used in this howto
  • vhv.allow = "TRUE" does nothing on ESX 5.1 and later
  • vhv.enable = "TRUE" is only going to be useful if you want to run VMs under the VMs running under ESXi running under kvm
  • You can add vmx.allowNested = "TRUE" to /etc/vmware/config in the ESXi VM to avoid having to put it in every nested VM's configuration file.
This quick howto was created with informations from https://communities.vmware.com/thread/451412
Special thanks to jmattson for the additional details on the VMware parameters


Unknown said...

Matt, please could you explain how you chose this IP setup for the bridge?

Suppose the host's IP is, and I want ESXi to be accessible from other machine (at, which bridge setup I should have?

(this is supposed that I can use any IP in the range 10.0.3.xxx for the bridge)

Thanks so much!

Gleb Frank said...

Hi Matt,

Were you able to run 64-bit VMs under the nested ESXi?

Gleb Frank

Christopher Wong said...

I am getting kvm "vcpu0 ignored rdmsr" errors
Are there any ways to supress those messages?

Unknown said...

Remember, the feeling you get from a good is far better than the feeling you get from sitting around wishing you were running. See the link below for more info.


Unknown said...

Thanks, qemu has merged the patch in.

Unknown said...

very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing. Apply vmware experienced jobs in hyderabad

Unknown said...

I cannot seem to be able to find the VMX file. Where is that located?

Thank you.

ajay prakash said...

Thank you so much for providing information on this. It was very useful.
aviation courses in Bangalore
aviation institute in Bangalore
best aviation academy in India
aviation academy in India

lekha mathan said...

Very useful information, Keep posting more blog likes this, Thank you.
best aviation academy in chennai
Air Hostess Training Institute in chennai
Airline Courses in Chennai
airport ground staff training in chennai
medical coding course
weekend fashion designing courses in chennai
Interior design courses in Chennai

Praylin S said...

Great article! It's really a pleasure to visit your site. Thanks for the detailed explanation. Really useful. Keep sharing more. Regards.
VMware Training in Chennai
VMware Course in Chennai
Oracle Training in Chennai
Oracle Training institute in chennai
JavaScript Training in Chennai
JavaScript Course in Chennai
VMware Training in OMR
VMware Training in Porur

Sadhana Rathore said...

I want to thank for sharing this blog, really great and informative. Share more stuff like this.
Machine Learning Course in Chennai
AWS Training in Chennai
DevOps Training in Chennai
Data Science Course in Chennai
R Programming Training in Chennai
RPA Training in Chennai
ccna course in Chennai
Python Training in Chennai
Angularjs Training in Chennai

manisha said...

Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information.
VMware Training in Gurgaon
VMware Course in Gurgaon
VMware Training institute in Gurgaon

Ramya Krishnan said...

This is the exact information I am been searching for, Thanks for sharing the required infos with the clear update and required points. To appreciate this I like to share some useful information regarding Microsoft Azure which is latest and newest,


Azure Training in Chennai
Azure Training Center in Chennai
Best Azure Training in Chennai
Azure Devops Training in Chenna
Azure Training Institute in Chennai
Azure Training in Chennai OMR
Azure Training in Chennai Velachery
Azure Online Training
Azure Training in Chennai Credo Systemz
DevOps Training in Chennai Credo Systemz


Thanks for sharing this blog. This is so informative blog
Learned a lot of new things from your post! Good creation .
Vmware Training institute in Noida

Vmware Training in Noida

asha katol said...
This comment has been removed by the author.
asha katol said...

Thanks for useful information on Vmware,keep Posting.

raj khan said...

Soma pill is very effective as a painkiller that helps us to get effective relief from pain. This cannot cure pain. Yet when it is taken with proper rest, it can offer you effective relief from pain.
This painkiller can offer you relief from any kind of pain. But Soma 350 mg is best in treating acute pain. Acute pain is a type of short-term pain which is sharp in nature. Buy Soma 350 mg online to get relief from your acute pain.


Buy Soma 350 mg
Soma Pill
Buy Soma 350 mg online

Buy Soma 350 mg online
Soma Pill
Buy Soma 350 mg

manisha said...

Thanks for sharing such a great blog Keep posting.. 
VMware Training in Delhi
VMware Course in Delhi

firstfly said...

Thanks for sharing a valuable information to us. keep on sharing like this.
Best Aviation Academy in Chennai
Best Air hostess Training in Chennai
Pilot Training in Chennai
Airport Ground staff Training in Chennai
Airport Flight Dispatcher Trainee in Chennai
RTR - Aero Training in Chennai
Cabin Crew Training in Chennai
Aviation Academy in Chennai
Aviation training institute in Chennai
Aviation Course Training in Chennai
Ground staff Training institute in Chennai
Airhostess Training institute in Chennai
Cabin Crew Course

Navinika said...

Thanks for sharing your knowledge. It's very useful for me and detailed and informative one. Keep doing the same.
Vmware Traning in Electronic City

Larry said...

Applicable to data center migration . Nice.

datasciencecourse said...

data science course bangalore is the best data science course