Volume Number: 26
Issue Number: 01
Column Tag: Virtualization
By Neil Ticktin, Editor-in-Chief/Publisher
< Previous Page Start | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 Next Page>
One of the common problems with File and Network IO Tests is caching. In fact, it's common for benchmarkers to think they are avoiding caching when in fact, they aren't. In the case of File and Network IO tests, there are two types of caching: at the host OS, and the guest OS. Furthermore, sometimes the caching can appear to happen even after restarts of both the host and guest OS.
The data set we used for testing was 4 files adding up to a total of 3.7GB. While avoiding caching, the same data files were used for all tests on all machines to keep things completely consistent. The benefit to the same set of files being used for all the copies is that you can see the differences between the different methods of copying. (Most of which were in the same relative time frame, except for copying to a USB Flash Drive. See the chart.)
Networking was done via the default NAT setup in both virtualized environments, and the defaults for disk setup for each application were used as well.
On the duplicate files, VMware performed significantly faster on the file copy after the copies had been done more than once (even with a full host and guest OS restart in between), sometimes even beating Parallels Desktop 5. However, since the test represents a more real world experience, the measurements are testing "Adam" tests - in other words, ones that are only done once and avoid caching.
Here were the results:
Or shown much better on a graph, it looks like this:
Figure 11: Virtual Machine File and Network I/O Performance
When we first started out to test 3D and HD Graphics, we were looking for ways to measure in a benchmark setting. What we found were a couple of things. First, in some cases, the performance on both platforms was so good; there was nothing that we could measure. Second, the common metric, frames per second, can be a poor method for measuring unless you have reliable and repeatable ways to measure this metric. Let's look at each case to explain.
3DMark06
3DMark06 by FutureMark is a globally recognized and comparable measurement of the 3D performance. Gamers use 3DMark as way to test their machines and tweak them for performance. See http://www.futuremark.com/
There are 3 main aggregate scores. The most important of which is the 3DMark score. In addition, SM2.0 Score measures 3D ShaderModel 2.0 performance, and the HDR/SM3.0 Score measures the 3D ShaderModel 3.0 & HDR performance.
Clearly, if you are going to play games and be serious about it, then running in BootCamp is your best choice. However, Parallels Desktop is fairly close and of course, you don't have to reboot with a virtualization solution. VMware Fusion 3 did ok, and certainly was improved over VMware Fusion 2, but in 3DMark06 and other testing, there were places where frames were not fully rendered, or the screen simply flashed black at times. See the examples of a Snow Scene in 3DMark's "Deep Freeze" tests.
Figure 12: Deep Freeze Scene, Parallels Desktop
Figure 13: Deep Freeze Scene, VMware Fusion
As for VMware Fusion running 3DMark06 on the MacBook and MacBook Pro under Windows 7, it simply doesn't work for some of the tests. We queried both FutureMark and VMware tech support and confirmed this. Furthermore, under Windows 7 (both 32-bit and 64-bit) running with two virtual CPUs, 3DMark06 crashes (BSoD) under VMware Fusion. Interestingly, it works just fine with one virtual CPU on the exact same VM. Parallels Desktop had no issues in these configurations.
The MacBook Pro and the Mac Pro were the clear winners here. The extra graphics hardware horsepower are noticeably faster (e.g., used) in both virtualization solutions.
The results for 3DMark06 vary greatly by the hardware they are on because of the graphics hardware of each machine. As such, the best way to look at these is on charts. If you'd like to see the detailed results for each model, see the full spreadsheet with collected data points. This speed chart will give you an idea of some of the things that VMware Fusion did well in, and the overwhelming winner that Parallels Desktop is in this graphics competition.
Figure 14 (renumber): Details from 3DMark06 Suite of Tests
< Previous Page Start | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 Next Page>
