Tuesday, May 27, 2014

Install and run a full functional VMware ESX Hypervisor within a KVM virtual machine


This post is about how to setup and configure a VMware ESX Hypervisor within a KVM virtual machine. The resulting ESX Host is going to be fully functional allowing to create (nested) virtual machines with a working network connection. 


Some (funny) facts:

  • Running VMware ESX within a KVM virtual machine allow memory over-commitment. That means even if your KVM Host system only has 2 GB memory you can assign e.g. 4 GB memory to the ESX Host VM.
  • KVM provides the hardware requirements for VMware ESX even if ESX does not supports the KVM Hosts hardware e.g. it allows to install and run a VMware ESX system on almost any kind of Desktop hardware which is normally not supported by ESX e.g. because of an unsupported network card model.

 Steps to setup VMware ESX within a KVM VM

  • Install Linux on a physical system (I used latest Ubuntu 14.04)
  • Install the qemu/kvm package 
                            apt-get install qemu-kvm
  •  Edit /etc/modprobe.d/kvm-intel (or kvm-amd for AMD cpus) and add

                            options kvm ignore_msrs=1
                            options kvm-intel nested=y ept=y
   pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, xen_enabled(),
      0x4);

          to
   pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, TRUE,
       0x4);

  • Configure the qemu 2.0.0 source with the following command:
./configure --enable-kvm --target-list=x86_64-linux-user,x86_64-softmmu
  • Create /usr/local/etc/qemu/bridge.conf
                     allow all
  • Install the bridge-utils package
                    apt-get install bridge-utils
  • Configure a network bridge in /etc/network/interfaces as following:
auto br0
iface br0 inet static
address 192.168.88.135
netmask 255.255.255.0
network 192.168.88.0
broadcast 192.168.88.255
gateway 192.168.88.1
dns-nameservers 192.168.88.1
bridge_ports eth0
bridge_fd 0
bridge_hello 2
bridge_maxage 12
bridge_stp off
 

 Please notice that your ip-configuration may be different
  • Choose a directory for the KVM virtual machine and create the virtual machine harddisk with the following qemu-img command:
                     qemu-img create -f raw esxi00.img 16G
  • Download VMware ESX Hypervisor 5.5 from the VMware website and put the install iso file (VMware-VMvisor-Installer-5.5.0.update01-1623387.x86_64.iso) into the same directory as the harddisk file was created a step before
  • Start the KVM virtual machine with:
/usr/local/bin/qemu-system-x86_64 -enable-kvm -cpu host -m 4096 -smp 2 -cdrom VMware-VMvisor-Installer-5.5.0.update01-1623387.x86_64.iso -netdev tap,helper=/usr/local/libexec/qemu-bridge-helper,id=hostnet0 -device vmxnet3,netdev=hostnet0,id=net0 -hda esxi00.img


Please notice the "vmxnet3" virtual network device type!
  • Go through the (normal) VMware ESX installation procedure
  • After the installation enable the SSH service on the ESX Host with is now running with the KVM VM. Then ssh into the ESX Host and set the following two parameters in /etc/vmware/config:
                      vhv.allow = "TRUE"
                      hv.assumeEnabled="TRUE"

Now please reboot the ESX Host system to activate the new configuration!
  • When creating a virtual machine on the (nested virtualized) ESX Host please make sure to add the following two parameters to the VM's vmx configuration file:

                       vhv.enable = "TRUE"
                       vmx.allowNested = "TRUE"

Congratulations! 

You have successfully virtualized a VMware ESX Hypervisor with KVM!

I hope you now enjoy your fully functional VMware ESX Host

Additional informations on the VMware parameters used in this howto
  • vhv.allow = "TRUE" does nothing on ESX 5.1 and later
  • vhv.enable = "TRUE" is only going to be useful if you want to run VMs under the VMs running under ESXi running under kvm
  • You can add vmx.allowNested = "TRUE" to /etc/vmware/config in the ESXi VM to avoid having to put it in every nested VM's configuration file.
This quick howto was created with informations from https://communities.vmware.com/thread/451412
Special thanks to jmattson for the additional details on the VMware parameters

32 comments:

Unknown said...

Matt, please could you explain how you chose this IP setup for the bridge?

Suppose the host's IP is 10.0.3.100, and I want ESXi to be accessible from other machine (at 10.0.3.200), which bridge setup I should have?

(this is supposed that I can use any IP in the range 10.0.3.xxx for the bridge)

Thanks so much!

Gleb Frank said...

Hi Matt,

Were you able to run 64-bit VMs under the nested ESXi?

Thanks,
Gleb Frank

Christopher Wong said...

Hello
I am getting kvm "vcpu0 ignored rdmsr" errors
Are there any ways to supress those messages?

Unknown said...

Remember, the feeling you get from a good is far better than the feeling you get from sitting around wishing you were running. See the link below for more info.


#running
www.ufgop.org

ritzk said...

Thanks, qemu has merged the patch in.

Unknown said...

very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing. Apply vmware experienced jobs in hyderabad

Unknown said...

I cannot seem to be able to find the VMX file. Where is that located?

Thank you.

ajay prakash said...

Thank you so much for providing information on this. It was very useful.
aviation courses in Bangalore
aviation institute in Bangalore
best aviation academy in India
aviation academy in India

Sadhana Rathore said...

I want to thank for sharing this blog, really great and informative. Share more stuff like this.
Machine Learning Course in Chennai
AWS Training in Chennai
DevOps Training in Chennai
Data Science Course in Chennai
R Programming Training in Chennai
RPA Training in Chennai
ccna course in Chennai
Python Training in Chennai
Angularjs Training in Chennai

manisha said...

Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information.
VMware Training in Gurgaon
VMware Course in Gurgaon
VMware Training institute in Gurgaon

Test My internet Speed said...

Thanks for sharing this blog. This is so informative blog
Learned a lot of new things from your post! Good creation .
Vmware Training institute in Noida

Vmware Training in Noida

asha katol said...
This comment has been removed by the author.
asha katol said...

Thanks for useful information on Vmware,keep Posting.

manisha said...

Thanks for sharing such a great blog Keep posting.. 
VMware Training in Delhi
VMware Course in Delhi

Unknown said...

Thanks for sharing your knowledge. It's very useful for me and detailed and informative one. Keep doing the same.
Vmware Traning in Electronic City

Larry said...

Applicable to data center migration . Nice.

sudhan said...

Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating

Robotic Process Automation (RPA) Training in Chennai | Robotic Process Automation (RPA) Training in anna nagar | Robotic Process Automation (RPA) Training in omr | Robotic Process Automation (RPA) Training in porur | Robotic Process Automation (RPA) Training in tambaram | Robotic Process Automation (RPA) Training in velachery

Devi said...

I went through your blog its really interesting and holds an informative content. oracle training in chennai

OGEN Infosystem (P) Limited said...

Great Post, thanks for sharing such an amazing blog with us. Visit Ogen Infosystem for creative website design and PPC Services in Delhi, India.
Website Designing Company in India

OGEN Infosystem (P) Limited said...

Great Post, thanks for sharing such an amazing blog with us. Visit Ogen Infosystem for creative website design and PPC Services in Delhi, India.
Website Designing Company in India

Unknown said...

Thanks for sharing this amazing blog

Bis Consultant in delhi

Excelr Tuhin said...

I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it!
Data Science course

manjot singh said...


Thanks for the Valuable information.Really useful information. Thank you so much for sharing. It will help everyone.

SASVBA Delhi provides extensive MERN stack training in Delhi. The extensive hands-on session, hosted by the MERN Stack Training Institute in Delhi, includes live projects and simulations.

FOR MORE INFO:

Data Science Training in Hyderabad said...

Excellent blog, I wish to share your post with my folks circle. It’s really helped me a lot, so keep sharing post like this…

Data Science Training in Hyderabad

Data Science Course in Varanasi - 360DigiTMG said...

Thanks for bringing out such innovative content which truly attracts the readers towards you. Certainly, your blog competes with your co-bloggers to come up with the newly updated info. Finally, kudos to you.

Data Science Course in Varanasi

Deekshitha said...

Informartive blog
best digital marketing institute in hyderabad

360DigiTMG said...

Pleasant data, important and incredible structure, as offer great stuff with smart thoughts and ideas, loads of extraordinary data and motivation, the two of which I need, because of offer such an accommodating data here.
data analytics course in hyderabad

Ramesh Sampangi said...

Really an awesome blog and useful content. Keep updating more blogs again soon. Thank you.
Data Science Course in Hyderabad

Unknown said...

Thanks for sharing this information. I really like your blog post very much. You have really shared a informative and interesting blog post with people.. data science training in surat

traininginstitute said...

Thanks for your post. I’ve been thinking about writing a very comparable post over the last couple of weeks, I’ll probably keep it short and sweet and link to this instead if thats cool. Thanks.
cyber security course malaysia

Links For You said...

lovely information perform article. This announce is as a matter of fact the top notch cruelly this imperative circumstance. Office 365 Keygen

cyber pc said...

The most beguiling sound I have ever heard in that framework is your voice. The most flawless spot I have ever been to is in your arms. https://wishesquotz.com/happy-birthday-wishes-for-my-love/