Tuesday, May 27, 2014

Install and run a full functional VMware ESX Hypervisor within a KVM virtual machine


This post is about how to setup and configure a VMware ESX Hypervisor within a KVM virtual machine. The resulting ESX Host is going to be fully functional allowing to create (nested) virtual machines with a working network connection. 


Some (funny) facts:

  • Running VMware ESX within a KVM virtual machine allow memory over-commitment. That means even if your KVM Host system only has 2 GB memory you can assign e.g. 4 GB memory to the ESX Host VM.
  • KVM provides the hardware requirements for VMware ESX even if ESX does not supports the KVM Hosts hardware e.g. it allows to install and run a VMware ESX system on almost any kind of Desktop hardware which is normally not supported by ESX e.g. because of an unsupported network card model.

 Steps to setup VMware ESX within a KVM VM

  • Install Linux on a physical system (I used latest Ubuntu 14.04)
  • Install the qemu/kvm package 
                            apt-get install qemu-kvm
  •  Edit /etc/modprobe.d/kvm-intel (or kvm-amd for AMD cpus) and add

                            options kvm ignore_msrs=1
                            options kvm-intel nested=y ept=y
   pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, xen_enabled(),
      0x4);

          to
   pc_basic_device_init(isa_bus, gsi, &rtc_state, &floppy, TRUE,
       0x4);

  • Configure the qemu 2.0.0 source with the following command:
./configure --enable-kvm --target-list=x86_64-linux-user,x86_64-softmmu
  • Create /usr/local/etc/qemu/bridge.conf
                     allow all
  • Install the bridge-utils package
                    apt-get install bridge-utils
  • Configure a network bridge in /etc/network/interfaces as following:
auto br0
iface br0 inet static
address 192.168.88.135
netmask 255.255.255.0
network 192.168.88.0
broadcast 192.168.88.255
gateway 192.168.88.1
dns-nameservers 192.168.88.1
bridge_ports eth0
bridge_fd 0
bridge_hello 2
bridge_maxage 12
bridge_stp off
 

 Please notice that your ip-configuration may be different
  • Choose a directory for the KVM virtual machine and create the virtual machine harddisk with the following qemu-img command:
                     qemu-img create -f raw esxi00.img 16G
  • Download VMware ESX Hypervisor 5.5 from the VMware website and put the install iso file (VMware-VMvisor-Installer-5.5.0.update01-1623387.x86_64.iso) into the same directory as the harddisk file was created a step before
  • Start the KVM virtual machine with:
/usr/local/bin/qemu-system-x86_64 -enable-kvm -cpu host -m 4096 -smp 2 -cdrom VMware-VMvisor-Installer-5.5.0.update01-1623387.x86_64.iso -netdev tap,helper=/usr/local/libexec/qemu-bridge-helper,id=hostnet0 -device vmxnet3,netdev=hostnet0,id=net0 -hda esxi00.img


Please notice the "vmxnet3" virtual network device type!
  • Go through the (normal) VMware ESX installation procedure
  • After the installation enable the SSH service on the ESX Host with is now running with the KVM VM. Then ssh into the ESX Host and set the following two parameters in /etc/vmware/config:
                      vhv.allow = "TRUE"
                      hv.assumeEnabled="TRUE"

Now please reboot the ESX Host system to activate the new configuration!
  • When creating a virtual machine on the (nested virtualized) ESX Host please make sure to add the following two parameters to the VM's vmx configuration file:

                       vhv.enable = "TRUE"
                       vmx.allowNested = "TRUE"

Congratulations! 

You have successfully virtualized a VMware ESX Hypervisor with KVM!

I hope you now enjoy your fully functional VMware ESX Host

Additional informations on the VMware parameters used in this howto
  • vhv.allow = "TRUE" does nothing on ESX 5.1 and later
  • vhv.enable = "TRUE" is only going to be useful if you want to run VMs under the VMs running under ESXi running under kvm
  • You can add vmx.allowNested = "TRUE" to /etc/vmware/config in the ESXi VM to avoid having to put it in every nested VM's configuration file.
This quick howto was created with informations from https://communities.vmware.com/thread/451412
Special thanks to jmattson for the additional details on the VMware parameters

54 comments:

Unknown said...

Matt, please could you explain how you chose this IP setup for the bridge?

Suppose the host's IP is 10.0.3.100, and I want ESXi to be accessible from other machine (at 10.0.3.200), which bridge setup I should have?

(this is supposed that I can use any IP in the range 10.0.3.xxx for the bridge)

Thanks so much!

Gleb Frank said...

Hi Matt,

Were you able to run 64-bit VMs under the nested ESXi?

Thanks,
Gleb Frank

Christopher Wong said...

Hello
I am getting kvm "vcpu0 ignored rdmsr" errors
Are there any ways to supress those messages?

Unknown said...

Remember, the feeling you get from a good is far better than the feeling you get from sitting around wishing you were running. See the link below for more info.


#running
www.ufgop.org

Unknown said...

Thanks, qemu has merged the patch in.

Unknown said...

very nice blogs!!! i have to learning for lot of information for this sites...Sharing for wonderful information.Thanks for sharing this valuable information to our vision. You have posted a trust worthy blog keep sharing. Apply vmware experienced jobs in hyderabad

Unknown said...

I cannot seem to be able to find the VMX file. Where is that located?

Thank you.

ajay prakash said...

Thank you so much for providing information on this. It was very useful.
aviation courses in Bangalore
aviation institute in Bangalore
best aviation academy in India
aviation academy in India

lekha mathan said...

Very useful information, Keep posting more blog likes this, Thank you.
best aviation academy in chennai
Air Hostess Training Institute in chennai
Airline Courses in Chennai
airport ground staff training in chennai
medical coding course
weekend fashion designing courses in chennai
Interior design courses in Chennai

Praylin S said...

Great article! It's really a pleasure to visit your site. Thanks for the detailed explanation. Really useful. Keep sharing more. Regards.
VMware Training in Chennai
VMware Course in Chennai
Oracle Training in Chennai
Oracle Training institute in chennai
JavaScript Training in Chennai
JavaScript Course in Chennai
VMware Training in OMR
VMware Training in Porur

Sadhana Rathore said...

I want to thank for sharing this blog, really great and informative. Share more stuff like this.
Machine Learning Course in Chennai
AWS Training in Chennai
DevOps Training in Chennai
Data Science Course in Chennai
R Programming Training in Chennai
RPA Training in Chennai
ccna course in Chennai
Python Training in Chennai
Angularjs Training in Chennai

manisha said...

Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information.
VMware Training in Gurgaon
VMware Course in Gurgaon
VMware Training institute in Gurgaon

Ramya Krishnan said...

This is the exact information I am been searching for, Thanks for sharing the required infos with the clear update and required points. To appreciate this I like to share some useful information regarding Microsoft Azure which is latest and newest,

Regards,
Ramya

Azure Training in Chennai
Azure Training Center in Chennai
Best Azure Training in Chennai
Azure Devops Training in Chenna
Azure Training Institute in Chennai
Azure Training in Chennai OMR
Azure Training in Chennai Velachery
Azure Online Training
Azure Training in Chennai Credo Systemz
DevOps Training in Chennai Credo Systemz

Test My internet Speed said...

Thanks for sharing this blog. This is so informative blog
Learned a lot of new things from your post! Good creation .
Vmware Training institute in Noida

Vmware Training in Noida

asha katol said...
This comment has been removed by the author.
asha katol said...

Thanks for useful information on Vmware,keep Posting.

raj khan said...


Soma pill is very effective as a painkiller that helps us to get effective relief from pain. This cannot cure pain. Yet when it is taken with proper rest, it can offer you effective relief from pain.
This painkiller can offer you relief from any kind of pain. But Soma 350 mg is best in treating acute pain. Acute pain is a type of short-term pain which is sharp in nature. Buy Soma 350 mg online to get relief from your acute pain.

https://globalonlinepills.com/product/soma-350-mg/


Buy Soma 350 mg
Soma Pill
Buy Soma 350 mg online



Buy Soma 350 mg online
Soma Pill
Buy Soma 350 mg

manisha said...

Thanks for sharing such a great blog Keep posting.. 
VMware Training in Delhi
VMware Course in Delhi

firstfly said...

Thanks for sharing a valuable information to us. keep on sharing like this.
Best Aviation Academy in Chennai
Best Air hostess Training in Chennai
Pilot Training in Chennai
Airport Ground staff Training in Chennai
Airport Flight Dispatcher Trainee in Chennai
RTR - Aero Training in Chennai
Cabin Crew Training in Chennai
Aviation Academy in Chennai
Aviation training institute in Chennai
Aviation Course Training in Chennai
Ground staff Training institute in Chennai
Airhostess Training institute in Chennai
Cabin Crew Course
PRIVATE PILOT LICENCE (PPL) Training in Chennai
COMMERCIAL PILOT LICENCE (CPL) Training in Chennai

Navinika said...

Thanks for sharing your knowledge. It's very useful for me and detailed and informative one. Keep doing the same.
Vmware Traning in Electronic City

Larry said...

Applicable to data center migration . Nice.

datasciencecourse said...

data science course bangalore is the best data science course

priyash said...

wonderful article. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article resolved my all queries.
Data science Interview Questions
Data Science Course

Excelr Tuhin said...

keep up the good work. this is an Ossam post. This is to helpful, i have read here all post. i am impressed. thank you. this is our Data Science course in Mumbai
data science course in mumbai | https://www.excelr.com/data-science-course-training-in-mumbai

datasciencecourse said...

I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.

data science course

Mithun said...

Thanks for the informative article About Java. This is one of the best resources I have found in quite some time. Nicely written and great info. I really cannot thank you enough for sharing.
Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

CloudLearn ERP said...

Great writing! You have a flair for informational writing. Your content has impressed me beyond words. I have a lot of admiration for your writing. Thank you for all your valuable input on this topic.
Best Data Science training in Mumbai

Data Science training in Mumbai

datasciencecourse said...

I am really enjoying reading your well written articles. It looks like you spend a lot of effort and time on your blog. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work.

Correlation vs Covariance

priyash said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance

Dubai Dandb said...

Very interesting blog. Many blogs I see these days do not really provide anything that attracts others, but believe me the way you interact is literally awesome.You can also check my articles as well.

Data Science In Banglore With Placements
Data Science Course In Bangalore
Data Science Training In Bangalore
Best Data Science Courses In Bangalore
Data Science Institute In Bangalore

Thank you..

sudhan said...

Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating

Robotic Process Automation (RPA) Training in Chennai | Robotic Process Automation (RPA) Training in anna nagar | Robotic Process Automation (RPA) Training in omr | Robotic Process Automation (RPA) Training in porur | Robotic Process Automation (RPA) Training in tambaram | Robotic Process Automation (RPA) Training in velachery

EXCELR said...

Your blog is very informative. It is nice to read such high-quality content.
Data Science Course in Hyderabad

priyanka said...

Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Correlation vs Covariance
Simple linear regression
data science interview questions

Devi said...

I went through your blog its really interesting and holds an informative content. oracle training in chennai

priyash said...

Amazing Article ! I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Simple Linear Regression
Correlation vs covariance
data science interview questions
KNN Algorithm
Logistic Regression explained

EXCELR said...

Awesome blog. I enjoyed reading your articles. This is truly a great read for me. I have bookmarked it and I am looking forward to reading new articles. Keep up the good work! data scientist courses

priyanka said...

very well explained. I would like to thank you for the efforts you had made for writing this awesome article. This article inspired me to read more. keep it up.
Logistic Regression explained
Correlation vs Covariance
Simple Linear Regression
data science interview questions
KNN Algorithm
Bag of Words

Pankaj Singh said...

Great Post, thanks for sharing such an amazing blog with us. Visit Ogen Infosystem for creative website design and PPC Services in Delhi, India.
Website Designing Company in India

Pankaj Singh said...

Great Post, thanks for sharing such an amazing blog with us. Visit Ogen Infosystem for creative website design and PPC Services in Delhi, India.
Website Designing Company in India

Unknown said...

Thanks for sharing this amazing blog

Bis Consultant in delhi

jhansi said...

Really, this article is truly one of the best, information shared was valuable and resourceful Very good work thank you.
tally training in chennai

hadoop training in chennai

sap training in chennai

oracle training in chennai

angular js training in chennai

Excelr Tuhin said...

I've read this post and if I could I desire to suggest you some interesting things or suggestions. Perhaps you could write next articles referring to this article. I want to read more things about it!
Data Science course

Mallela said...

Thanks for posting the best information and the blog is very informative.data science interview questions and answers

data scientist said...

Terrific post thoroughly enjoyed reading the blog and more over found to be the tremendous one. In fact, educating the participants with it's amazing content. Hope you share the similar content consecutively.

360DigiTMG Data Analytics Course

Mallela said...

Thanks for posting the best information and the blog is very helpful.Data science course in Faridabad

Mallela said...

Thanks for posting the best information and the blog is very helpful.data science courses in Bangalore

manjot singh said...


Thanks for the Valuable information.Really useful information. Thank you so much for sharing. It will help everyone.

SASVBA Delhi provides extensive MERN stack training in Delhi. The extensive hands-on session, hosted by the MERN Stack Training Institute in Delhi, includes live projects and simulations.

FOR MORE INFO:

Unknown said...

Thanks for sharing such nice info. I hope you will share more information like this. please keep on sharing!

Python Training In Bangalore | Python Online Training

Artificial Intelligence Training In Bangalore | Artificial Intelligence Online Training

Data Science Training In Bangalore | Data Science Online Training

Machine Learning Training In Bangalore | Machine Learning Online Training

AWS Training In Bangalore | AWS Online Training

IoT Training In Bangalore | IoT Online Training

Adobe Experience Manager (AEM) Training In Bangalore | Adobe Experience Manager (AEM) Online Training

Oracle Apex Training In Bangalore | Oracle Apex Online Training

Data Science Institute in Bangalore - 360DigiTMG said...

Truly mind blowing blog went amazed with the subject they have developed the content. These kind of posts really helpful to gain the knowledge of unknown things which surely triggers to motivate and learn the new innovative contents. Hope you deliver the similar successive contents forthcoming as well.

data science in bangalore

Maneesha said...

Hey there! Do you know if they make any plugins to help with Search Engine Optimization? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good gains. If you know of any please share. Thank you!|
data scientist training and placement in hyderabad

Aishwariya said...

Thank you for sharing your article. Great efforts put it to find the list of articles which is very useful to know, Definitely will share the same to other forums. Amazon Web Services Training in Chennai

Maneesha said...

All of these posts were incredible perfect. It would be great if you’ll post more updates.
data scientist training and placement in hyderabad

Devi said...

Did you want to set your career towards Oracle? Then Infycle is with you to make this into reality. Infycle Technologies gives the combined and best Oracle course in Chennai, which offers various stages of Oracle such as Oracle PL/SQL, Oracle DBA, etc., along with 100% hands-on training guided by professional tutors in the field. Along with that, the mock interviews will be given to the candidates to face the interviews with complete confidence. Apart from all, the candidates will be placed in the top MNC's with an excellent salary package. To get it all, call 7502633633 and make this happen for your happy life.Best Oracle Training in Chennai | Infycle Technologies

ram said...

Learn Amazon Web Services for excellent job opportunities from Infycle Technologies, the Excellent AWS Training in Chennai. Infycle Technologies gives the most trustworthy AWS course in Chennai, with full hands-on practical training from professional trainers in the field. Along with that, the placement interviews will be arranged for the candidates, so that, they can meet the job interviews without missing them. To transform your career to the next level, call 7502633633 to Infycle Technologies and grab a free demo to know more