Summer Break

After an 18-month stint at chip maker Intel involving many a crazy 80-hour work week, I decided to hand in my resignation last month and return back to the software world. I had a lot of fun there, enjoyed presenting at the CGO conference on their behalf in Toronto this spring, shared my thoughts with a lot of smart hardware engineers, and hopefully some of my ideas about the x86 architecture rubbed off on them enough to appear in future Intel processors years down the road. Time will tell.

Since the Memorial Day long weekend I have been poking around with the latest releases of Bochs 2.4.5, QEMU 0.12.4, Visual Studio 2010, Fedora 13 Linux, VirtualBox 3.2.4, VMware Workstation 7.1, GCC 4.5.0, and revisiting some older emulation in the form of WinQEMU 0.10.5 and Microsoft's Virtual PC 7.02 for Mac. In other words, lots of emulation related fodder to discuss.

As fictional Professor Farnsworth on Futurama likes to say, "Good news everyone!". I started jotting down notes about what to write and I realized I easily have at least three postings worth of content to go over this summer. As it happens, brand new episodes of Futurama return to television next week more than 6 years after being cancelled by Fox. In the spirit of other things from 6 years ago being bought back to life, in future postings I will explain why I recently purchased not one but three PowerPC based Macintosh computers and will devote an entire posting to the PowerPC processor and the wonderful things it is capable of.

Also this summer I will also take a look again at the latest QEMU emulator to see how it fares against Bochs today. If you recall from my posts in 2008, not so well. But QEMU's success is rather important to all of us as you I will explain as I compared QEMU against VirtualBox, VMware, and Virtual PC.

Almost forgot! By popular demand (translation: a lot of nag emails) I have put together a Table Of Contents page for this blog series to make it easier to find specific topics I've discussed in the past. Apparently just listing 29 links labeled "Part 1" through "Part 29" wasn't descriptive enough. Heh, what happens when an engineer writes a blog! <grin>

First things first. Today I will show you how to build the latest Bochs 2.4.5 release, and give a brief overview of the POSIX subsystem, what to many of you is the hidden Unix lurking inside your Windows 7.

Building the latest Bochs sources and testing with ttylinux

A few weeks ago the latest official release of Bochs was made, version 2.4.5, at the usual Sourceforge site:

http://bochs.sourceforge.net/

Around the same time Fedora 13 (http://fedoraproject.org/) was also being released, VMware Workstation (http://www.vmware.com/products/workstation/) version 7.1 was in beta, and VirtualBox 3.2 (http://www.virtualbox.org/) came out claiming the ability to virtualize Mac OS X guests, I have spent much of the past month installing these various virtualization products and trying out both Linux and Mac OS X virtualization on top of Windows and Mac OS X. In doing so I also ended up enlisting in both Bochs and QEMU source trees and building them as a way to make sure that the virtualization was working correctly and that the performance was decent.

The astute reader will recall that I gave a tutorial on building Bochs back in Part 17. I suggest a quick read of that old posting to get familar with tarballs and CVS and extracting archives. I will cover much the same territory today and follow it through right to booting an actual emulated operating system, with screen shots, and show you how to build a Unix version right on your Windows. The development tools to do this are all freely available, so you my readers can follow along at home regardless of whether you are running a Snow Leopard Mac, a Linux machine, or a Windows 7 PC.

I suggest downloading the latest post-2.4.5 snapshot of Bochs which has some patches that Stanislav and I added for the Windows builds. For this posting I used last week's June 11 tarball of the Bochs sources, which I include here for your convenience: http://www.emulators.com/freefile/bochs-20100611.tar.gz

If you prefer a ZIP file, or will be building Bochs on Windows, grab my custom ZIP file which has some minor tweaks to build both native 32-bit and 64-bit Windows builds as well as the POSIX configure script: http://www.emulators.com/freefile/bochs-20100611-win32.zip. My ZIP file's sources also have Windows makefiles already pre-built and ready to compile.

Extract either the tarball or the ZIP file to a folder on your Windows machine, something like C:\BOCHS.

You will also need something to actually boot inside of Bochs, a CD-ROM or hard disk image containing some bootable operating system. For my testing, I have a whole collection of Windows 2000, Windows XP, Windows 7, and various Linux disk images which I have created over the years to use on Bochs and QEMU. If you do not have such an image, most any Linux "Live CD" image will work, such as the one for Fedora 13 of course, or a new cutesy Linux distribution called Peppermint Linux available at: http://peppermintos.com/

For quick and simple testing I prefer a very small and simple Linux distribution called ttylinux, available at: http://minimalinux.org/ttylinux/

ttylinux is a very minimal text-only Linux, which has a relatively tiny boot CD image on the order of 20 megabytes in size, much smaller than the usual 700-megabyte image of most live CDs. I recommend downloading the "i686" version of the ttylinux from this page: http://minimalinux.org/ttylinux/downloadX86.html

Copy your disk images to some other folder, for demonstration purposes I will use C:\IMAGES.

Bochs on Windows using Visual Studio 2010

Let's start with the common Windows build using the Visual Studio C++ compiler. Microsoft has for the past few years made the command line compiler tools freely available, so even if you do not own the full Visual Studio 2003/2005/2008/2010 suite, you can use the 32-bit compiler that was included with the Signularity download a couple of years ago, or download the latest Visual Studio 2010 Express Edition CD-ROM image here: http://www.microsoft.com/express/downloads/

You just need the basic 32-bit command line tools (CL.EXE, LINK.EXE, RC.EXE, Win32 headers and libraries, etc.) so don't worry that this is not the full IDE and other nonsense that's bundled into the full version. Open up a Visual Studio 32-bit x86 command prompt (found in Start menu \ Program \ Microsoft Visual Studio \ Visual Studio Tools). Type "cl" and hit Enter to verify that the compiler is in your path, as shown here:

If you have extracted my ZIP file to C:\BOCHS, at this point navigate to that folder, and type:

    nmake

to kick off the build using Visual Studio's C++ compiler. Or, to get a slightly more optimized version, use my wrapper for nmake which specifies additional compiler optimization flags, performs link-time optimization, and generates compiler listing files:

    m clean

About a minute later you should have a finished BOCHS.EXE file built. Verify by typing:

    dir  bochs.exe

as shown here:

Now, in your C:\IMAGES folder, you will need to create a BOCHSRC.TXT file, a settings file which tells Bochs which ROM BIOS image to use, which CD-ROM image to boot from, how much memory to emulate, which x86 processor features to emulate, etc. I've created a very simple bochsrc.txt file for booting the ttylinux ISO image, which looks like this:

# A super simple BOCHSRC.TXT file which boots TTYLINUX .ISO in Bochs 2.4.5+

log: bochsout.log

memory: guest=24, host=24

# Guest clock speed in terms of emulated instructions per virtual second
cpu: count=1, ips=50000000, reset_on_triple_fault=1

# Dynamic processor features.
cpuid: mmx=1, sep=1, sse=sse2, xapic=1, aes=0, movbe=0, xsave=0, cpuid_limit_winnt=0

romimage: file="../bochs/bios/BIOS-bochs-latest", address=0xfffe0000
vgaromimage: file="../bochs/bios/VGABIOS-lgpl-latest"
vga_update_interval: 40000
mouse: enabled=0, toggle=f12

ata0: enabled=1, ioaddr1=0x1f0, ioaddr2=0x3f0, irq=14
ata1: enabled=1, ioaddr1=0x170, ioaddr2=0x370, irq=15

# CD-ROM image
ata0-slave: type=cdrom, path="ttylinux-i686-11.1.iso", status=inserted
boot: cdrom
i440fxsupport: enabled=1

You can cut and paste the above test into Notepad and say as C:\IMAGES\BOCHSRC.TXT. The "memory:" line tells Bochs to simulate 24 megabytes of guest RAM. Very small yes, but ttylinux is a very tiny Linux distribution. The "cpu:" line says to treat every 50 million simulated instructions a one virtual second in the guest time. The "cpuid:" line is new setting in Bochs 2.4.5 and later, which allows you to specify specific x86 instruction set extensions such as MMX and SSE. For this demonstration, I have set it to simulate SSE2, comparable to emulate a Pentium 4 processor.

The "romimage:" and "vgaromimage:" settings specify ROM BIOS images from the C:\BOCHS folder. You could also copy those ROM image files to the C:\IMAGES folder and adjust the path accordingly. The "mouse:" line says to not capture the mouse, but that the F12 key be use to capture and uncapture the mouse into the emulated machine. This is a setting that is more useful when emulating a graphical guest such as Windows or one of the more mainstream Linux live CDs.

The next few lines tell Bochs to emulate an IDE bus and to mount the ttylinux CD-ROM ISO image file as an IDE slave drive. The "boot:" line tells the emulated BIOS to boot from the CD-ROM device, not a hard disk.

That's it! Once you have created the bochsrc.txt file, from your C:\IMAGES folder type:

    c:\bochs\bochs  -q

and ttylinux should start booting up as seen here:

If you have followed along so far, you now have a Windows compatible build of Bochs. Congratulations. If not, you can take a shortcut and just download my pre-built Visual Studio 2010 build of the current Bochs sources here: bochs-p4-245-win32.zip

However, cheated a little, in that the ZIP file I provided already contains pre-built "makefile" files for a Visual Studio build environment. What about Linux? Mac OS X? Unix? Cygwin?

Bochs on Windows using Cygwin

As I covered two years ago in Part 17, the makefiles are generated by running the configure script inside of a Unix style shell. On Windows we can use Cygwin, available from http://www.cygwin.com/. Go ahead and run the Cygwin installer as per my previous directions and make sure to include the gcc C and C++ compilers in the installation. From your Windows Start menu, click Program \ Cygwin \ Cygwin Bash Shell to open up a Cygwin command prompt, or "shell" in Unix speak. Navigate over to your C:\BOCHS folder using the "cd C:/BOCHS" command. Remember, in Linux and Unix, path names use forward slashes "/", while in DOS and Windows command prompts one uses backward slashes "\".

To make sure that you did install the gcc compilers, type:

    gcc --version

and:

    g++ --version

to verify that each compiler is installed, as shown in this screen shot:

Once that is confirmed, type:

    sh  .conf.win32-cygwin

This will now create the necessary Cygwin-compatile Makefile files as well as update the config.h header file which sets various compile flags for the Cygwin environment. Once the configure script it finished, type:

    make

and in about a minute you will have a Cygwin build of Bochs ready, which you can very by typing:

    ls  -las  bochs

to see that the executable file bochs is built.

Bochs on Windows using POSIX - the "hidden" UNIX inside of Windows 7

Cygwin is an emulation layer which maps Unix/Linux style system calls to the WIN32 API of Windows applications. In other words, WIN32 is the set of system calls that Windows applications such as Internet Explorer and Microsoft Office call to for example, draw dialog boxes or access files. So Cygwin is really just a gcc compiler port for the WIN32 API, it is not really a true Unix/Linux type environment. One still has to "port" an application to Windows even building it in Cygwin. Is there a better way?

A long long time, back in 1992 when I was still a fairly young and new Microsoft employee, I was working in the Visual Studio group on various 32-bit compilers for the Apple Macintosh as well as the new 32-bit Windows NT 3.1 operating system, or simple "NT" as it was called, the predecessor to today's Windows XP and Windows 7 operating systems. One of the cool things about NT was that it supported a multitude of system call interfaces. Not just WIN32, but also MS-DOS system calls, OS/2 system calls, and a set of system calls refered to as POSIX (http://en.wikipedia.org/wiki/POSIX). Back in the 1980's Microsoft itself used and even sold a version of Unix called XENIX/386. When I first started as an intern at Microsoft in 1987, software wasn't developed in MS-DOS or Windows or OS/2 (it hadn't been released yet), and certainly NT was years away, so my first day on the job I dived right in to XENIX. Years later when NT was being developed, it was of course natural that NT support a POSIX environment.

POSIX is a standard that defines a set of common interfaces for Unix and Linux type operating system calls. Just as WIN32 does not specify any particular release of Windows, POSIX does not refer to any particular release of Unix or Linux. But any Unix, or Linux, or Mac OS X for that matter, supports the POSIX standard. Most open source software out there is not written for Linux per se, it is written for any POSIX-compatible operating system.

Over the years as Microsoft made enemies with everyone it slowly phased out both the OS/2 and POSIX support, then even the DOS support itself. If you buy a new laptop which has a 64-bit Windows Vista pre-installed on it, there is NO built-in support for MS-DOS applications, OS/2 applications, or POSIX.

So I was very pleasantly surprised to see Microsoft playing nice again with Windows 7, and putting POSIX right back into the Windows 7 box. If you are running Windows 7, go into your Control Panel \ Programs \ Turn Windows Features On or Off applet, and you will see an option to install "System For Unix-based Applications".

Guess what? That feature installs and enables the POSIX subsystem! This is really great news, because this enables one to download all sorts of open source code which isn't necessarily ported to Windows yet and easily build and run it on Windows 7. This a bigger deal than one mere checkbox in the Control Panel would imply.

To my surprise, Microsoft even offers its own build of the gcc compilers and related utilities for POSIX. If you go to Microsoft's download site at http://www.microsoft.com/downloads/ and search for "unix subsystem" you will see a download that says "Utilities and SDK for System for UNIX-based Applications in Microsoft 7 and Windows Server 2008 R2". It is about a 250-megabyte download. Download it and run it. What that does is install the Windows version of the POSIX gcc compiler tools and header files and libraries. It gives you the ability to download arbitrary open source projects and built them. Such as for example, Bochs!

When you install the POSIX subsystem and the SDK, you will find a C:\WINDOWS\SUA folder. That folder is effectively the Unix file system within Windows. It has a /bin folder, an /etc folder, a /home folder, etc. Go to one more web site, the SUA Community site at http://suacommunity.com/SUA.aspx and click on the Tool Warehouse link. This allows you to install various such open source tools, both in source code form and as pre-built binaries. You will want to minimally install the bash shell and the CVS source control packages, which will less your dependence on Cygwin.

Now you are ready to build a POSIX release of Bochs on your Windows 7 machine. Open up either the C shell or the Bash shell from the Start \ Program \ Subsystem for Unix-based Applications menu. Use the command:

    cd  /dev/fs/C/bochs

to navigate to the Windows C:\BOCHS folder. As with Cygwin, verify that gcc C and C++ compilers are installed. You also need my custom .conf.posix file, which is a modified configuration script specifically for the Windows 7 POSIX. Namely, take the existing .conf.linux file, save it as .conf.posix, and make sure to add -D_ALL_SOURCE -DIFNAMSIZ=44 to the CCFLAGS string. This is my version of the .conf.posix file:

#!/bin/sh
#
# .conf.posix
#
# This script sets up a Windows Services for Unix Applications (SUA) build.
# Optimized for Pentium/Core architecture.
# Contact Darek Mihocka (darek@emulators.com) if you have questions.
#

CC="gcc"
CXX="g++"
CFLAGS="-Wall -g -m32 -O1 -march=pentium -msse -fomit-frame-pointer -momit-leaf-frame-pointer -save-temps -D_ALL_SOURCE -DIFNAMSIZ=44"
CXXFLAGS="$CFLAGS"

export CC
export CXX
export CFLAGS
export CXXFLAGS

#######################################################################
# configuration 1 for release binary RPMs
# Include a few standard options, speed optimizations, X11 only.
#######################################################################
./configure --enable-sb16 \
--enable-ne2000 \
--enable-all-optimizations \
--enable-cpu-level=6 \
--enable-x86-64=no \
--enable-pci \
--enable-acpi \
--enable-clgd54xx \
--enable-usb \
--enable-usb-ohci \
--enable-show-ips \
--enable-debugger=no \
--enable-debugger-gui=no \
--enable-disasm=no \
--enable-logging=no \
${CONFIGURE_ARGS}
;;
 

I have highlighted my edits. Notice the use of -m32 to specify building a 32-bit executable, -O1 to optimize for code size (which is actually also a performance win), the two defines, and the use of =no to disable some unneeded options like 64-bit support and logging.

Then, similar to the Cygwin scenario, type:

    sh  .conf.posix

let it create new Makefiles and config.h file, then type:

    gmake

to start a build. In about a minute, your POSIX build of Bochs will complete, and you can very using the ls command as before that the executable has built, as shown here:

If per change you get an error at the step that runs the ar command, delete the file C:\Windows\SUA\bin\ar or rename it, it is the wrong one and the correct one will be found in the path. You may also find the VI editor that comes with POSIX a bit outdated. I suggest deleting C:\Windows\SUA\bin\vi and installing VIM (http://www.vim.org/) which is a much better version of VI for Windows.

Once built, from the Windows command prompt, you can launch this POSIX executable just as you did the Visual Studio and Cygwin-built versions before. However, when you try this, nothing will appear to happen. This is because your POSIX built is not calling the WIN32 system calls. It's not going to render a Windows window. POSIX applications target the graphical system calls of X11, the same graphics API as used by Mac OS X, Unix, and Linux. You need to enable an X11 graphics server for anything to be output. To do that, make sure that your Cygwin setup included the xinit and xterm packages, then from the Start \ Programs \ Cygwin-X menu launch XWin Server. This enables X11 to work, and when you launch Bochs again, presto, your X11-rendered POSIX version of Bochs springs to life:

Notice, this is NOT the same output as the previous Windows builds. The window borders and status bars are different, the fonts are different. This is being rendered by X11, not by WIN32 directly. Bochs is not running use the #ifdef WIN32 code paths, it is running the same code paths as a Linux build. Even if Bochs had never been ported to Visual Studio or Cygwin, this would have built and run. That is the really cool power of the POSIX subsystem in Windows 7, it completely broadens the scope of open source code that one can download and run on Windows.

Bochs on Windows using Mac OS X and Linux

Mac OS X right from its release almost a decade ago has been Unix based. Mac OS X is POSIX compatible and so Mac users have had the ability to download arbitrary open source code and run many of the same applications as any Linux users. Mac OS X is an X11 server, so it is ridiculously easy to build and run Bochs in Mac OS X.

The other day I booted up my Macbook into Mac OS X 10.6.3 and built Bochs in much exactly the same way as I just described:

• download the June 11 tarball of Bochs sources and extract
• install the gcc C and C++ compilers, which are provided directly on any Mac OS X Leopard (10.5) or Snow Leopard (10.6) setup DVD
• specify the .conf.macosx configure script
• type make
• run the generaetd bochs executable using the same bochsrc.txt file I used for the other builds to boot ttylinux

Tada! Below is screen shot of my Bochs build window (on the right), the window that I launched the executable from, and of course, the X11-rendered Bochs output window.

On Linux, it is much the same story. Many Linux distributors, especially the developer oriented releases such as Fedora and CentOS, install the gcc compiler tools by default, or by selecting the "Development Tools" checkbox during setup. Use the .conf.linux configuration file to set up your Makefiles and config.h and the rest you are a pro at by now. Whether Linux, Mac OS X, or Windows POSIX, the steps are almost the same on every platform.

This is a screen shot taken on one my Atom nettop machines running Feroda Linux with Bochs booting ttylinux (click the thumbnail to enlarge):