(c) 2008 by Darek Mihocka, founder,

March 14 2008

[Part 1]  [Part 2]  [Part 3]  [Part 4]  [Part 5]  [Part 6]  [Part 7]  [Part 8]  [Part 9]  [Part 10]  [Part 11]  [Part 12]  [Part 13]  [Part 14]  [Part 15]  [Part 16]  [Part 17]  [Next]   [Return to]

The Future Is Almost Here Much Sooner!

Happy "Pi Day", better known as Albert Einstein's birthday.

There was a sort of relativistic space-time effect going on last time I posted on Leap Day, as little did I know that as I was writing up my wish list of things I wanted to see happen in time for next Leap Day, VMware was holding another VMworld event, in France, and already previewing that future! There was a smaller VMware event here in Seattle on March 4th that I did attended, and am very glad I did, because at that event I learned of the news from France. I thank Kevin Connor of VMware for filling me in on the details and pointing me at some links about the information, which you can now find here:

Looking at the three technologies they announced and previewed, virtual management and application virtualization don't really excite. But, the middle announcement does. "Offline Virtual Desktop Interface" they call it. It needs a sexier name, perhaps "Remote Virtualization" or as I was calling it the "Virtual Machine in the Cloud" concept. But this is fact the beginnings of the technology vision I've been blogging about for the last few months - the ability to connect to a remote server, seamlessly pull down a virtual machine to your local machine to use without a further connection to the server, then to later be able to sync up your local virtual machine with the one stored on the server. This is what I was asking for last summer, and VMware has stepped up and listened to me and no doubt to many other paying customers.

It is funny because I actually went to the event to heckle VMware, he he. The presentation Kevin gave started off with the familiar recap of VMware's existing technologies - the VMware Workstation that we've all in one way or another played around with in its various forms that include VMware Fusion for Mac and VMware Player, the VMware ACE technology that allows pre-canned encrypted virtual machines to be deployed for demo purposes and such, and the VMware VDI technology which allows you to remotely access a virtual machine running on a server farm hosting VMware Server. These are the products that are VMware's bread and butter and why they're so insanely popular these days.

But as I have been criticizing, these products are sort of at opposite ends of the spectrum. You are either running a virtual machine completely locally on your desktop or notebook computer, or you are remotely accessing a virtual machine running on some big iron server. This latter one is the "thin client" scenario not unlike the old mainframe computing days, where one can use a relatively inexpensive "thin client" terminal to access their real desktop running on some big fat server. But, if you lose your network connection, you lose your interactive desktop. It's like losing your internet connection in the middle of accessing your Gmail or being in the middle of an online shopping transaction. You must either have everything copied down to your local machine to work offline, or, you put your trust on a reliable network connection.

VMware and other companies do have a technology called "vMotion" or "live migration" which allows a virtual machine to be moved from one host server to another host server. This is to support local balancing on the host servers for the purpose of saving power, or to allow bad servers to be shut down and replaced by offloading their virtual machines. But the "killer app" scenario to me is to be able to offload the virtual machine right down to the client itself.

This conference in Seattle was actually sponsored by NEC, who most of use know as a monitor manufacturer, but who also make "thin client" hardware, inexpensive little PCs that you connect a monitor to and use strictly for the purpose of remotely accessing your session on the server. NEC was showing off their devices and a clever little hack for the purpose of accelerating remote media playback. Anyone who has tried to use the Windows "Remote Desktop" feature to view movies or video clips remotely knows what I am talking about. To take high framerate video and compress it to an RDP channel just doesn't work very well. And what you end up with on the remote client is very choppy video. NEC was showing off devices which use a second data channel to send the video content in its original form across the network and then to have it decompressed and played back on the local thin client. Clever, but to me this is a hack to solve a one-off problem, and not a general solution.

So being the muck disturber that I am, I raised my hand and asked both the NEC rep and Kevin why VMware does not simply build this back channel media trick into VMware's products directly. Before I could make the next logical step of asking about the remote virtualization and adding a back channel to migrate the virtual machine state on, Kevin dropped the fantastic news of Offline VDI. I just about fell out of my chair in shock and joy.

This is very tricky stuff and is by no means the full package. As I have pointed out, VMware's reliance on hardware virtualization means that both vMotion and Offline VDI will currently have some severe technical limitations, such as not being able to live migrate between AMD and Intel host machines, or even between radically different releases of the same brand's microprocessors, or say, to my Sony Playstation 3. There are some tricks called "CPUID masking" that VMware describes (and which I believe is also used by at least Xen) which hide this issue to a point, but this is still limited to a specific range of AMD and Intel microprocessors. At this point VMware has only previewed the feature, I do not believe it is available as a beta download yet, and my guess is that VMware is working hard to solve these technical challenges. Playstation 3, Apple TV, Xbox 360, there are the kind of clients I want to pull my virtual machines to when I am sitting in front of the living room big-screen TV/monitor.

What Offline VDI also does not address is the full vision of what I described last year, which is peer-to-peer migration between client machines. I go on vacation but say I want to take my work with me. So I connect to my server at work and pull down my personal virtual machine to my personal Dell laptop and jump on a plane to Europe. I now use my virtual machine locally on my Dell laptop while flying. A few hours later the battery start to run low. I now want to power up my Sony VAIO and "squirt" the virtual machine running on the Dell over to the VAIO via either wi-fi or some such temporary connection. You simply can't do that yet today with existing virtualization products, especially say if the Dell and Sony are running completely different CPUs, and it does not appear to me that Offline VDI supports this.

The solution of course as I've said a hundred times is to not rely on vendor-specific hardware virtualization but to implement virtualization using true binary translation techniques. I will not beat this to death any more, as I have blogged about this enough, and am writing up a paper on the subject for that conference in Beijing I mentioned last time.

VMware also needs to set up some kind of hosting deal with say Google to make such virtual machine hosting not just live in the domain of enterprise IT servers, but in a manner where anyone could just access the Gmail account and sync up with their virtual machines from any computer in the world. That vision still remains. But I do thank VMware for listening and for taking the leadership role again to take virtualization into unexplored territory. Of course you've just killed my startup idea. :-)

One thing I don't understand though, if VMware won't hire me to fix their virtualization issues, at least buy out the company Transitive ( Transitive are the guys who do the PowerPC legacy application emulation on the Intel based Macs. They also do other non-Intel processor emulation, and do so using tried and tested binary translation. I've met some of Transitive's engineers, and talking with them a couple of years ago I know that they implement their technology pretty much exactly how I would go about doing it. They use binary translation to provide true isolation of the virtual machine guest code from the host hardware, which is not true of the current products from VMware, Microsoft, or XenSource. VMware, meet Transitive. Transitive, meet VMware. Now please get a room and mate for the good of mankind.

Parallels Catches up to 2006

Another virtualization company that has been making news lately is Parallels ( Formerly called, um, I don't know, something forgettable, they apparently decided to change their name to the name of their one-trick-pony product. Do I sound a little down on Parallels? You bet. Ever since I download the first VMware Fusion beta a year and a half ago, it has simply been no contest between Parallels and Fusion. Parallels didn't support Vista, didn't support 64-bit, didn't support multi-core. It was a one-trick pony that ran XP and cashed in on that market before VMware. I don't respect Parallels for taking that easy route, for just cashing in like that instead of delivering a real that took advantage of what Intel Macs had to offer.

Two years later now Parallels has decided to play follower and merely catch up to where VMware was in late 2006. I hate Parallels! I wish they would just go away. But in the interests of fairness, here is a link to their latest propaganda: ( I think intelligent Macintosh users can realize that this is just a "me too" technology, and will know to steer clear of this and stick with VMware Fusion.

I particularly take issue with their claim: "Enhanced! Achieve maximum performance by leveraging Intel® Virtualization Technology". That is just plain misleading of them to say that. That claims goes in the face of everything I have been saying about hardware virtualization, my own posted benchmarks of Intel's hardware virtualization in NO EXECUTE Part 4, and even against VMware's paper and other published data about hardware VT. If it so "enhanced" where are the numbers? To the marketing folks at Parallels: stop blowing smoke and post actual real world data showing performance of Parallels now, Parallels before, and of VMware Fusion. Oh, wait, you did, here (bottom link on the page above in case my hyperlink doesn't work).

Here is how Parallels makes their claim, based on a review by Neil Ticktin in the Feb 2008 issue of MacTech magazine. If you click on the link and read the article, in big bold letters it states that Parallels is 5.2 to 6.0 times faster than VMware Fusion. "On average" even. That's quite the big bold statement to make. Really? 5 to 6 times faster than VMware? Read the rest of the article, and oops, the actual real world gains one is likely to experience at best are in the tens of percent. Quite a bit different to claim 500% faster in big bold letters, and then to have the rest of your data give numbers like 12%.

The review is also interesting because it ultimately does not support Parallels' claim. Neil concludes that the fastest product for running Window Vista is... VMware Fusion! And get this, right on the first page, Neil states that what products were compared were VMware Fusion 1.0 (from more than 6 months ago!) to the new Parallels 3.0. In his words, "opted not to upgrade". I'm sorry, what do paying readers of MacTech magazine except if not fair and accurate product reviews? VMware Fusion 1.1 with fixes of Leopard and performance issues came out months ago. VMware Fusion 1.1.1 came out after Macworld 2008 in January.

The review does not answer the question of how much did Parallels "enhance" in speed from earlier versions, as per their marketing claim. And it does not do an accurate apples-to-apples comparison of current virtualization products on the shelves in February 2008. Parallels should be ashamed of itself for using that review as its marketing tool, and I have to question MacTech magazine's movites. In the interests of full disclosure, I have conducted business with MacTech magazine and Neil Ticktin in particular as part of my marketing effort of my SoftMac product in the year 2001. Back then when I was exhibiting at Macworld Expo computer shows, I took out some advertising space in MacTech magazine, and in turn provided some retail SoftMac product which was sold through Developer Depot. In that case it was not a review, just ad space being bartered for. But given that practice, I have to wonder how fair a review one can expect from MacTech magazine. It would be fair and honest for Neil Ticktin and MacTech magazine to disclose any such financial deals they have with the companies they write about.

Building Bochs 101

Back to positive news. With the trace cache tweaks checked in last week, Bochs is now running faster than ever. On my Core 2 based Mac Pro, Windows XP is now booting up in 1 minute 45 seconds, just 5 seconds from the goal of booting XP in 100 seconds. Bochs today is still purely C++ based, still purely interpreted, without any use of hardware virtualization or jitting or inline assembly or any such hacks. Bochs is now actually beating QEMU at several benchmarks. For example, my little trivial C-compiler loop, T1FAST.EXE, runs in as little as 8 or 9 seconds on the Mac Pro and the latest Bochs, compared to 10 to 11 seconds for QEMU. Proving once again that a tight small memory-efficient interpreter can actually outperform a jitting engine. I proved it almost 10 years ago with SoftMac's 68040 interpreter trouncing some 68040 jitters, and now Bochs does it to QEMU. I love proving commonly accepted myths wrong!

A few people have emailed me with problems with regards to downloading and building Bochs, so I thought this week I would focus on a little beginner's class on how to build Bochs on a fresh clean system. I didn't even know how to do this 6 months ago but as with anything else, the instructions are documented and it is just a matter of following them. I thank Stanislav yet again for holding my hand and helping me with this for when I was too lazy to read the documentation myself ;-)

I will try to distill it down to the simplest steps possible. The five build scenarios I am going to discuss:

  • building a 32-bit Windows version of Bochs using Cygwin,
  • building a 32-bit Windows version of Bochs using Visual Studio 2003/2005/2008,
  • building a 64-bit Windows version of Bochs using Visual Studio 2005/2008,
  • building a 32-bit Mac OS X version of Bochs on Leopard, and,
  • building a 32-bit Linux version of Bochs.

The steps are very similar in all five scenarios, so once you get one working the other four are easy. I just made myself do this yesterday from scratch using the current sources from March 13 and again with today's sources, so trust me, even a ranting idiot like me can do it. The good news is that even if you do not own any commercial development package like Visual Studio 2008, you can still build and customize Bochs using freely available tools.

Step 1 - Downloading Bochs

1a - the current source snapshot via CVS

I'll start first with Linux, the one I am building on is Novell SUSE Linux 10.1 which I picked up at Fry's as part of the "Linux Starter Kit" for about 35 dollars. This is the Linux I have had installed on my Dell D800 laptop for many months now, and I like it because besides being the "Microsoft approved" distribution, the install pretty much gives you the world - FireFox web browser, the GCC 4.1 C/C++ compiler, the CVS (Concurrent Versions System) source control utility, OpenOffice 2.0, and very seamless hardware support for Dell laptop hardware, such as the built-in 802.11 wireless with WEP/WPA support. Unlike many other Linux distributions I've tried, this one "just works" on the first try. The Linux Starter Kit lives up to its name.

Much as with the Windows "Start" button, go to the lower left corner of the screen, click on "Applications", then "System", then "Terminal", then "Gnome Terminal". This is your command line prompt, and it will open in your default "home" directory. Since CVS is installed, you can directly download the entire Bochs source tree using the CVS command, which I have copied here directly from the Bochs documentation page:

    cvs -z3 co -P bochs

This will take a few seconds, but will download the entire Bochs source tree to a folder called, of course, "bochs".

On Mac OS X 10.5 ("Leopard"), the procedure is similar. From your original Leopard install DVD, you need to install the optional developer tools. This installs the GCC 4.0 C/C++ compiler. Now, from the dock, launch a Terminal window and type in the same CVS command as above. Apple Developers: if you an Apple Developer Connection subscriber, you can log in to your ADC account and download the GCC 4.2 developer preview for Mac OS X. This is actually the GCC version I build Bochs with on my Macs, as it seems to fix a code generation bug in GCC 4.0.

On Windows (whether 32-bit or 64-bit XP or Vista), you need to download the Cygwin tools ( This is a free set of tools that installs common Unix/Linux commands to your Windows machine, including the CVS utility, the "tar" utility, and the GCC 3.4 C/C++ compiler. Cygwin does not yet include GCC 4.x, possibly because of known code generation bugs. Click on the "Install or update now!" link on the Cygwin home page. This will run the setup utility which gives you a long list of options for applications and tools to install. You want to make sure that you select the cvs, gcc-core, and gcc-g++ options under Devel, then click Next to install them. This may take a few minutes. Allow Cygwin setup to either create a Cygwin icon on your desktop and/or on your Start menu. Click that icon once to set up the Cygwin environment for the first time. Now type gcc -v or cvs -v or tar to make sure those commands are each installed. Now you can take the above steps as described for Linux and Mac OS X to download Bochs sources using CVS.

1b - the official release

Official releases of Bochs source tree such as 2.3.5 and 2.3.6 can be found by clicking on the "See All Releases" link on the Bochs web site ( For version 2.3.6 for example, there are two releases - a platform independent release file called bochs-2.3.6.tar.gz, and a specific version for Windows to be build by Visual Studio called Download both for now. Use the "tar" or WinRAR utilities to extract these files on your particular system, as explained below.

1c - the daily tarball

If you simply want the current "tip of tree" source code tree of Bochs, you can go to this page ( and download the daily "tarball" at the botton of the page. A tarball is a compressed collection of files similar to a ZIP file. On Linux and Cygwin and Mac OS X you will be able to use the "tar" command directly to expand the tarball, while on plain Windows you will need to use a third party decompression tool such as WinRAR (at Download and decompress the tarball to your home directory using your browser and WinRAR, or use the "tar -xf" command followed by the tarball filename to decompress. For example, today's (March 14 2008) tarball file is about 3.8 megabytes in size and is named bochs-20080314.tar.tar.

For example, to uncompress (extract) today's tarball, the command in Cygwin, Mac OS X, or Linux would be:

   tar -xf bochs-20080314.tar.tar

The source tree will then be found in a newly create directory of the same name, in this case it will be named bochs-20080314.

Step 2 - Configuring Bochs

If you are a Visual Studio user and cheated by download the file, you can skip ahead to step 3 to build the default 32-bit Windows release of Bochs. The configuration has already been done for you. We'll worry about the 64-bit Windows build in Step 4.

For everybody else, you have to do the traditional open source configuration step. Since open source projects are portable, there is generally a configure shell script which sets up the actual makefiles and some header files for your system. There are some platform-dependent little script files which wrap the main configure script. The files we care about today are .conf.win32-cygwin, .conf.win32-msvc, .conf.macosx, and .conf.linux. You will find these files in the root directory of the Bochs source tree.

In your Cygwin command prompt or Linux / Mac OS X terminal window that you already have open, run the sh command followed by the appropriate configuration script. For example, in Cygwin, you would type:

   sh .conf.win32-cygwin

while on Mac OS X you would type:

   sh .conf.macosx

About a minute later you are ready to build. You can actually use the Cygwin window to also type:

   sh .conf.win32-msvc

which sets up the makefiles for Visual Studio. This is essentially the only difference between the release snapshots which are platform independent and specifically for Visual Studio.

Step 3 - Building Bochs

If you are a Visual Studio use, it's really simple. Click on the Start button, and find your "Microsoft Visual Studio Command Prompt" icon. Click it, and you should have a command line window open with the Visual Studio build environment set up. Find your way over to your Bochs source tree and type NMAKE. That's it. BOCHS.EXE should build about two minutes later.

For the other scenarios which are thus using GCC, from the command prompt or terminal window that you already had open to run the configuration scripts, just type make. That's it.

Step 4 - Tips

DOS and Windows users not familiar with Unix-style command shells might find Cygwin confusing at first. There is no DIR command, but rather the ls command. Whenever you are tempted to type in DIR, type "ls -last". The cd command to change directories works as before, but you may find the directory structure confusing. Your home directory in the Cygwin prompt actually corresponds to the C:\CYGWIN\HOME\username directory from Windows. And similarly, Windows driver letters map in to the Cygwin file system under the /cygdrive folder, so for example, your D: drive you get to by typing cd /cygdrive/d.

If you are dual booting or even dual booting between Windows, Mac OS X, and Linux on the same PC or Mac, you can download and extract the Bochs source tree to a common directory on a shared hard disk partition. I do this with a shared FAT32 partition on my Mac which I share between Windows Vista and Mac OS X. What I do then using that as a template, copy the whole tree to a Windows-specific NTFS partition or a Mac specific HFS partition. Unfortunately the Mac OS X build does not build correctly on a FAT32 partition, and although Windows Vista supports UDF hard disk partitions, the Mac OS X does not seem to "see" those partitions. NTFS of course is read-only in Mac OS X, so until Apple and Microsoft kiss and make good, there still isn't a decent way to share a common partition between Mac OS X and Windows on the same computer such that it supports read and write access from both, long file names from both, and file streams and file attributes from both. (Anyone have a better tip here?)

If you are using the GCC 4.2 preview on Mac OS X, you need to add these four lines to the .conf.macosx file:

export CC
export CXX

To build a 64-bit native Win64 version of Bochs for 64-bit Windows Vista, you have to run a 64-bit build environment. Visual Studio 2005 and Visual Studio 2008 have command prompts which are specifically labeled 64-bit. Open those, then similarly you type NMAKE to build a 64-bit Bochs. But wait, first there are some minor issues to fix. The Bochs configure script incorrectly sticks a redundant "/MACHINE:I386" linker command line option in each of the makefiles. Manually edit the MAKEFILE file in the root directory and each of the sub-directories, and remove that linker switch. This switch is not needed for either the 32-bit build or 64-bit builds actually, as the machine type is encoded into each .OBJ object file. You may also have to edit the gui\win32.cpp file and comment out some of the GUI code which contains function calls which are not supported in 64-bit mode. This is my tweaked version of the 64-bit compatible win32.cpp which builds with the 64-bit Visual Studio 2008.

To make minor configuration changes, such as changing the level of SSE that is simulated by Bochs, to turn off the trace cache or edit minor constants, you can edit the CONFIG.H file directly and then just retype NMAKE or make to rebuild. For example, as I don't use the built-in Bochs debugger or disassembler, I set the DISASM define to 0 (in today's sources that is config.h around line 652). If you wish to only have Bochs simulate a 32-bit Pentium 4 (sufficient for booting Windows XP) edit line 160 in config.h and set BX_SUPPORT_X86_64 to 0. I am not a fan of special casing relatively rare instructions with their own special code (it's the Jan's Razor principle), so to slightly reduce the code size of Bochs (and as I have measured, to actually speed it up a little), set the BX_SupportRepeatSpeedups value to 0 as well.

I also cheat, I clone Visual Studio's NMAKE.EXE to the file MAKE.EXE so that I can just type make everywhere in all environments.

You can also tweak the settings of the newly introduced instruction trace cache. On older microprocessors the trace cache can be too large for the host CPU's L2 cache, and I have found that by reducing the size of the instruction hash table and the maximum length of each trace, I can reduce the memory footprint of the trace cache significantly without hurting performance on Core 2 but while gaining some performance on older machines. In the file cpu\icache.h, edit the define of BX_MAX_TRACE_LENGTH down to the value 9, and similarly edit the BxICacheEntries define down to (8 * 1024). To compensate for the smaller hash table, I tweak the hash function a little so as to involve more bits of the original address, as follows: return (pAddr + (pAddr>>6)) & (BxICacheEntries-1);

These tweaks get my Windows XP boot time of Bochs down to about 105 to 110 seconds. The Mac OS X build built with the GCC 4.2 preview is a little slower at about 120 seconds, but works just as well. By building in 64-bit mode, and turning off 64-bit simulation, the boot time of XP can drop down to about 80 seconds now, equivalent to a simulation rate of 60 million x86 instructions per second! That's cheating, but not bad, as that is a full 3x faster than Bochs 2.3.5 ran on Core 2 just six months ago - 60 MIPS versus 20 MIPS simulation speed. As I mentioned this actually outperforms QEMU on some simple test programs.

Happy hacking! Anyone tried to build Bochs on a Sony Playstation 3 under Yellow Dog Linux yet? :-)

I would love to hear if my directions for downloading and building Bochs are helpful. Please keep those comments and ideas coming by emailing me at or simply click on one of the voting links below to send your comments.

Darek, awesome, this is better than watching Oprah cry!
Darek, shut up and go back to eating your sandwich, Atari man!

[Part 1]  [Part 2]  [Part 3]  [Part 4]  [Part 5]  [Part 6]  [Part 7]  [Part 8]  [Part 9]  [Part 10]  [Part 11]  [Part 12]  [Part 13]  [Part 14]  [Part 15]  [Part 16]  [Part 17]  [Next]   [Return to]