VMware VMotion, how fast can we go?

9 11 2009

This article is moved towards my new site:





10 responses

9 11 2009

Nice post Kenneth !!!

9 11 2009
Kenneth van Ditmarsch

Thanks, but without a real explanation YET…. 😉

9 11 2009
Steve Cleveland

Sorry if it’s a dumb question, but how are you getting a 7Gbit network? And I assume the troubles you’re having will translate to 10GBit as well, so definitely something that needs to be solved. Good research on this!

10 11 2009
Kenneth van Ditmarsch

Hi Steve,

If you read my post about Flex-10 it should clear up how I got the 7 Gb link. (it’s actually a 10Gb network that is shared into pieces from within the HP hardware)

10 11 2009

Very interesting…
Nothing on the page size side ?

10 11 2009
Kenneth van Ditmarsch

Not that I’m aware of. The transferred VM isn’t doing anything and I use the .vswp file on the datastore stored with the VM itself. (no reservations, no limits)

10 11 2009

Seems like the network device in the vmkernel is limited. Is this network stack based on the vmx or the linux stack?

10 11 2009
Kenneth van Ditmarsch

Well I’m assuming this is based on the linux stack. Whenever I’m testing the iperf speed between VM’s it would be based on the VMX since obviously the OS (and thus the OS overhead) is in between the transfers.

Last night I was thinking about this and maybe CPU has something to do with it. I can remember that during the ESX courses (yeaaars ago, so very old info from either ESX 2.5 or 3.0) they told me that transferring on 1 Gb speed would take up 1000 MHz of CPU cycles. If this information is still relevant (which I doubt) I could explain it this way: I’m running 2.26 GHz procs (8 cores) assuming I’m getting only 1 core that is serving my VMotion the math goes like this: 2260 MHz = 2.2 Gb speed = approx. the values I see when VMotionning.

Anyone any thought on that? I haven’t checked esxtop to check this behavior yet BTW

17 11 2009

are jumbo frames enabled? vmotion nic’s directly connected between hosts or via a switch?

17 11 2009
Kenneth van Ditmarsch

No Jumbo isn’t enabled, however I did the same test on a native 10 Gb line (with Jumbo Frames enabled) and my speed didn’t came much higher.
The VMotion nic’s are connected via a switch, but as stated my line is able to do higher speeds with the VM to VM so my line obviously isn’t the bottleneck in this test.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: