mips64emul



mips64emul User Documentation


Contents:


Overview:

mips64emul is a free MIPS machine emulator. The goals are to emulate MIPS-like CPUs and enough surrounding hardware to let unmodified operating systems run as if they were running on a real machine, and to assist in experiments involving MIPS-like code in general.

It is important to keep in mind that devices and CPUs are not really emulated correctly, they are only "faked" well enough to make eg. NetBSD run. That said, the emulator could still be of interest for academic research and experiments, such as learning how to write an OS for a MIPS-like machine, or profiling SMP, memory usage, or system call patterns.

The emulator is written in C, depends on no external libraries (except X11, which is optional), and should compile and run on any Unix-like system. If it doesn't, that is to be considered a bug.

The emulator contains code which tries to emulate the workings of MIPS-like CPUs and surrounding hardware found in real machines, but it does not contain any actual MIPS ROM code or other form of MIPS program code. You will need some form of program (in binary form) to run in the emulator. For many emulation modes, PROM calls are handled by the emulator itself, so you do not need to use any PROM image at all.

You can use pre-compiled kernels (for example NetBSD kernels, or Linux), or other programs that are in MIPS binary format, and in some cases even actual ROM images. A couple of different file formats are supported (ELF, a.out, ECOFF, SREC, raw binaries).

You do not need any MIPS compiler toolchain to build or use mips64emul. If you need to compile MIPS binaries from sources, then of course you need such a toolchain, but that is completely separate from mips64emul. (There is also a regression testing framework, which requires that a GNU CC for mips64-unknown-elf or similar is available, but it is not required to build the emulator.)

(MIPS is a registered trademark of MIPS Technologies. mips64emul is developed as a spare-time hobby, and there is no affiliation with MIPS Technologies in any way whatsoever.)


Is mips64emul free software?

Yes. mips64emul is distributed under a free license. (For a definitions of the four freedoms associated with free software, please read http://www.gnu.org/philosophy/free-sw.html.)

The code I have written is released under a 3-clause BSD-style license (or "revised BSD-style" if one wants to use GNU jargon.) Apart from the code I have written, some files are copied from other sources such as NetBSD, for example header files containing symbolic names of bitfields in device registers. They are also covered by similar licenses, but with some additional clauses. If you plan to redistribute mips64emul (for example as a binary package), or reuse code from mips64emul, then you should check those files for their license terms.

(The licenses usually require that the original Copyright and license terms are included when you make a copy or modification. The "easiest way out" if you plan to redistribute code from mips64emul is to simply supply the source code. You should however check individual files for details.)


How to compile/build the emulator:

Uncompress the .tar.gz distribution file, and run
	$ ./configure
	$ make

This should work on any Unix-like system. If it doesn't, then please mail me a bug report.

(Note for Windows users: there is a small possibility that this will also work under Cygwin, but this is totally unsupported, as I have no such system to try it on.)

The emulator's performance is highly dependent on both runtime settings and on compiler settings, so you might want to experiment with different CC and CFLAGS environment variable values. For example, on a modern PC, you could try the following:

        $ CFLAGS="-mcpu=pentium4 -O3" ./configure
        $ make

A number of different configure options are available:

        --caches       enable cache emulation
        --delays       enable instruction latency/delay emulation
        --mips16       enable MIPS16 instruction support
        --nobintrans   configure without bintrans support, even if the host supports it
        --nox11        configure without X11 support
        --userland     enable userland emulation

--caches enables emulation of caches (R3000 only, so far). This causes an overall slowdown, so don't enable it unless you need it. Enabling caches does not by itself mean correct cache miss penalties and such (you need both --caches and --delays for that) but it is needed to fool some operating system kernels (eg Mach/pmax) that the cache works like it should.

--delays simulates instruction latencies. Memory accesses can have penalties. When combined with --caches, cache misses cause penalties as well. This causes an overall slowdown, so don't enable it unless you need it.

--mips16 enables MIPS16 instruction support. This doesn't really work yet, as only very few MIPS16 instructions are translated. This causes an overall slowdown, so don't enable it unless you need it.

--nobintrans disables dynamic binary translation completely.

--nox11 disables X11 support completely. (If the configure script cannot compile a small X11 test program, it will be disabled anyway.) This might improve performance slightly, but makes it impossible to run certain things in the emulator.

--userland enables userland emulation support. This is experimental, only a few syscalls are implemented, mostly as a proof-of-concept. This might cause a tiny slowdown, so don't enable it unless you need it.

The possible configure options might differ between different releases and snapshots, so this list may be incomplete.

Once you have built mips64emul, running it should be rather straight-forward. To exit the emulator, type CTRL-C to enter the single-step debugger, and then type quit. By typing CTRL-B instead, a CTRL-C is sent to the emulated program.


Which machines does mips64emul emulate?

A few different machine types are emulated. The machine types that are emulated best at the moment are:

There is code in mips64emul for emulation of many other machine types; the degree to which these work range from "almost" being able to run a complete OS, to almost completely unsupported (perhaps just enough support to output a few boot messages via serial console).

In addition to specific machine types, a "bare" machine can be emulated. A bare machine consists of one or more CPUs and a few experimental devices such as:

This mode is useful if you wish to experiment with software for MIPS-like systems, but do not with to target any specific real-world machine type, for example for educational purposes or generic MIPS related experiments.


Which CPU types does mips64emul emulate?

While other emulators aim to emulate only one specific CPU type, eg R3000, mips64emul does not. The ideal goal would be to emulate "all" MIPS-like CPUs.

Both 64-bit and 32-bit MIPS-like CPUs can be emulated. mips64emul is basically an instruction-level simulator; things that would happen in several steps within a physical CPU are not necessarily simulated correctly or at all (eg. pipe-line stages).

Emulation of R4000, which is a 64-bit CPU, was my initial goal. Right now, R2000/R3000-like CPUs are also emulated (32-bit), and emulation of R1x000 (at least the parts that are similar to R4000) is beginning to work as expected. Code targeted for MIPS32 and MIPS64 also happens to work, as long as no specific features of MIPS32/64 are used that didn't exist in R4000 etc.

I have written an experimental dynamic binary translation subsystem. This gives higher total performance than interpreting one instruction at a time and executing it. However, even with the binary translation activated, mips64emul is not nearly as fast as some of the other emulators out there that have speed as their number one priority. (If you wish to enable bintrans, add -b to the command line, but keep in mind that it is still experimental.)


Which guest OSes are possible to run?

This table sums up the guest OSes that run well enough to be considered working in the emulator. They can boot from a harddisk image and be interacted with similar to a real machine:


Guest OS:   Emulation mode:   Guest OS:   Emulation mode:
NetBSD/pmax DECstation NetBSD/arc ARC (Acer Pica)
OpenBSD/pmax DECstation OpenBSD/arc ARC (Acer Pica)
Ultrix/RISC DECstation Sprite DECstation
Redhat Linux* DECstation Debian GNU/Linux* DECstation
 
NetBSD/pmax 1.6.2 OpenBSD/pmax 2.8 Ultrix 4.5 Redhat Linux*
 
NetBSD/arc 1.6.2 OpenBSD/arc 2.3 Sprite Debian GNU/Linux*


* Although Linux runs under DECstation emulation, at the time of writing this Linux has no support for keyboards on the 5000/200 (the specific DECstation model being emulated), so when the login prompt is reached you cannot interact with the system. Some people are working on improving the situation. Running in text-mode (emulated serial console) works fine, though, but then you don't get to see the penguin in the corner.

It is non-trivial to get a specific operating system or OS kernel to run in the emulator, so don't expect the list above to grow too quickly.

There is no guarantee that anything specific will run in the emulator, but NetBSD is a good starting point for someone who wants to experiment.


How to install NetBSD/pmax in the emulator:

To run NetBSD/pmax in the emulator (on an emulated DECstation machine), you first need to install NetBSD onto a disk image. To NetBSD, the disk image looks and acts like a real SCSI disk, but it is just a normal file in the host's filesystem.

I've done most of the testing with NetBSD 1.6.2, but the following instructions should hopefully work for NetBSD 2.0 as well.

You need to do the following:

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Create an empty harddisk image, which will be the root disk that NetBSD installs itself onto:
    	$ dd if=/dev/zero of=netbsddisk.img bs=1 count=512 seek=1900000000
    
    

From this point, there are two separate ways to continue the installation. You can either download a CD-ROM iso image (and let the installation program copy files from the CD-ROM image to the harddisk image), or you can install via ftp. For an installation from a CD-ROM image, follow these steps:

  1. Download a NetBSD CD-ROM iso image:
    	ftp://ftp.netbsd.org/pub/NetBSD/iso/1.6.2/pmaxcd.iso
    	or
    	ftp://ftp.netbsd.org/pub/NetBSD/iso/2.0/pmaxcd.iso
    
    
  2. Start the emulator like this:
    	$ ./mips64emul -X -b -D2 -d netbsddisk.img -d bc:pmaxcd.iso
    

For an ftp install, substitute steps 3 and 4 above with these:

  1. Download a NetBSD pmax INSTALL kernel, and gunzip it:
    	ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-1.6.2/pmax/binary/kernel/netbsd-INSTALL.gz
    	or
    	ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-2.0/pmax/binary/kernel/netbsd-INSTALL.gz
    
    	$ gunzip netbsd-INSTALL.gz
    
    
  2. Start the emulator like this:
    	$ ./mips64emul -X -b -D2 -d netbsddisk.img -O netbsd-INSTALL
    

(If you don't want to use a graphical framebuffer during the install, you can remove -X from the command line, but then make sure you choose 'vt100' when prompted with which terminal type to use, and not 'rcons'.)

Then proceed like you would do if you were installing NetBSD on a real DECstation. Even if the emulator is running on a very fast host system, you should expect the installation to take quite some time. If you are installing from the network, then suitable networking parameters are as follows:

	Which device shall I use? [le0]: le0
	..
	Your DNS domain: mydomain.com
	Your host name: foo
	Your IPv4 number: 10.0.0.1
	IPv4 Netmask [0xff000000]: 0xff000000
	IPv4 gateway: 10.0.0.254
	IPv4 name server: 10.0.0.254

(If using 10.0.0.254 as the nameserver fails, then try entering the IP number of a real-world nameserver instead.)

When the installation is completed, the following command should start NetBSD from the harddisk image:

	$ ./mips64emul -X -b -M64 -D2 -d netbsddisk.img

Use startx to start X windows.

NOTE: For some reason, NetBSD 2.0 doesn't work with X out-of-the-box on pmax. It seems that this has to do with a switch to WSCONS. For now, if you want X, then try NetBSD 1.6.2.

If you want to run without the X framebuffer, use this instead:

	$ ./mips64emul -D2 -b -d netbsddisk.img

NetBSD/pmax seems to work fine with dynamic binary translation (enabled by the -b command line option) in R3000 mode. NetBSD also supports R4000-like CPUs on pmax, but the bintrans support for R4000 in the emulator is less stable than for R3000.


How to install NetBSD/arc in the emulator:

It is possible to run NetBSD/arc on an emulated Acer PICA-61 in the emulator.

To install NetBSD/arc from a CDROM image onto an emulated harddisk image, follow these instructions:

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Create an empty harddisk image, which will be the root disk that NetBSD installs itself onto:
    	$ dd if=/dev/zero of=nbsd_arc.img bs=1024 count=1 seek=900000
    
    
  3. Download a NetBSD/arc CDROM image from ftp:
    	ftp://ftp.netbsd.org/pub/NetBSD/iso/1.6.2/arccd.iso
    
    
  4. Download a NetBSD/arc installation (ramdisk) kernel, and gunzip it:
    	ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-1.6.2/arc/binary/kernel/netbsd.RAMDISK.gz
    
    
  5. Start the emulator using this command line:
    	$ ./mips64emul -A2 -b -d nbsd_arc.img -d arccd.iso netbsd.RAMDISK
    
    
  6. From now on, you have to use your imagination, as there is no automatic installation program for NetBSD/arc. Here are some tips and hints on how you can proceed with the install:
    	$ mount /dev/cd0a /mnt2
    	$ disklabel -i -I sd0    (for example 'a', '4.2BSD', '1c',
    	    '700M', 'b', 'swap', '701M', '$', 'P', 'W', 'y', and 'Q')
    	$ newfs /dev/sd0a
    	$ mount /dev/sd0a /mnt
    	$ cd /mnt
    	$ for a in /mnt2/arc/binary/sets/*.tgz; do tar xvzpf $a; done
    	$ cd dev; sh MAKEDEV all
    	$ cd ../etc; echo "rc_configured=YES" >> rc.conf
    	$ cat > /mnt/etc/fstab
    	    /dev/sd0a / ffs rw 1 1
    	    /dev/sd0b none swap sw 0 0
    	    (ctrl-d)
    	$ cd /; umount /mnt; umount /mnt2
    	$ halt
    
    
  7. Download a generic NetBSD/arc kernel, and gunzip it:
    	ftp://ftp.netbsd.org/pub/NetBSD/NetBSD-1.6.2/arc/binary/kernel/netbsd-GENERIC.gz
    
    

You can now use the generic NetBSD/arc kernel to boot from the harddisk image, using the following command:

	$ ./mips64emul -A2 -b -d nbsd_arc.img netbsd-GENERIC

Using -b (enabling dynamic binary translation), as in the instructions above, is not 100% stable. If you experiment weird crashes, try removing that option and try again.


How to install OpenBSD/pmax in the emulator:

Installing OpenBSD/pmax is a bit harder than installing NetBSD/pmax. You should first read the section above on how to install NetBSD, before continuing here. If you have never installed OpenBSD on any architecture, then you need a great deal of patience to do this. If, on the other hand you are used to installing OpenBSD, then this should be no problem for you.

OpenBSD/pmax died at release 2.8 of OpenBSD, so you should be aware of the fact that this will not give you an up-to-date OpenBSD system. The files are available at this URL:

	ftp://ftp.se.openbsd.org/pub/OpenBSD/2.8/pmax/

Following these instructions might work. If not, then use common sense and imagination to modify them as you see fit.

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Create an empty harddisk image, which will be the root disk that OpenBSD installs itself onto:
    	$ dd if=/dev/zero of=openbsddisk.img bs=1 count=512 seek=900000000
    
    
  3. Download the entire pmax directory from the ftp server: (approx. 99 MB)
    	$ wget -r ftp://ftp.se.openbsd.org/pub/OpenBSD/2.8/pmax/
    
    
  4. Execute the following commands:
    	$ mv ftp.se.openbsd.org/pub/OpenBSD/2.8/pmax/simpleroot28.fs.gz .
    	$ gunzip simpleroot28.fs.gz
    	$ chmod +w simpleroot28.fs		<--- make sure
    
    
  5. You now need to make an ISO image of the entire directory you downloaded. I recommend using mkisofs for that purpose. If you don't already have mkisofs installed on your system, you need to install that in order to do this.
    	$ mkisofs -o openbsd_pmax_2.8.iso ftp.se.openbsd.org/pub/OpenBSD/2.8/pmax
    
    
  6. Start the emulator with all three (!) disk images:
    	$ ./mips64emul -D2 -b -d openbsddisk.img -d b:simpleroot28.fs -j bsd -d c:openbsd_pmax_2.8.iso
    
    
    (If you add -X, you will run with the graphical framebuffer. This is REALLY slow because the console has to scroll a lot during the install. I don't recommend it.)

  7. Go on with the installation as you would do if you were installing on a real machine. If you are not used to the OpenBSD installer, then this will most likely be a very uncomfortable experience. Some important things to keep in mind are:

  8. For some unknown reason, the install script does not set the root password! The first time you boot up OpenBSD after the install, you need to go into single user mode and run passwd root to set the root password, or you will not be able to log in at all!
    	$ ./mips64emul -D2 -b -d openbsddisk.img -d 2c:openbsd_pmax_2.8.iso -j bsd -o '-s'
    
    While you are at it, you might want to extract the X11 install sets as well, as the installer seems to ignore them too. (Perhaps due to a bug in the installer, perhaps because of the way I used mkisofs.)

    Execute the following commands in the emulator:

    	# fsck /dev/rz0a
    	# mount /
    	# passwd root
    
    	# cd /
    	# mount -t cd9660 /dev/rz2c /mnt
    	# sh
    	# for a in /mnt/[xX]*; do tar zxvf $a; done
    	# ln -s /usr/X11R6/bin/Xcfbpmax /usr/X11R6/bin/X
    	# ln -s /dev/fb0 /dev/mouse
    	# echo /usr/X11R6/lib >> /etc/ld.so.conf
    	# ldconfig
    
    	# sync
    	# halt
    

NOTE: It is also possible to install via ftp instead of using a CDROM image. This is not much less awkward, you still need the simpleroot filesystem image, and you still have to manually add the X11 install sets and set the root password, and so on.

Once you have completed the installation procedure, the following command will let you boot from the new rootdisk image:

	$ ./mips64emul -D2 -b -X -M64 -o '-aN' -d openbsddisk.img -j bsd

OpenBSD/pmax seems to work fine with dynamic binary translation (enabled by the -b command line option).

(Normally, you would be asked about which root device to use (rz0), but using -o '-aN' supresses that.)

When asked for which terminal type to use, when logging in as root, enter rcons if you are using the graphical framebuffer, vt100 for text-mode.
Use startx to start X windows.


How to install OpenBSD/arc in the emulator:

It is possible to run OpenBSD/arc on an emulated Acer PICA-61 in the emulator.

(You should be aware of the fact that OpenBSD for the ARC platform died at release 2.3, so this will not give you an up-to-date OpenBSD system. See http://www.openbsd.org/arc.html for more information.)

To install OpenBSD/arc onto an emulated harddisk image, follow these instructions:

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Create an empty harddisk image, which will be the root disk that OpenBSD installs itself onto:
    	$ dd if=/dev/zero of=obsd_arc.img bs=1024 count=1 seek=700000
    
    
  3. Download the entire arc directory from the ftp server: (approx. 75 MB)
    	$ wget -np -l 0 -r ftp://ftp.se.openbsd.org/pub/OpenBSD/2.3/arc/
    
    
  4. You now need to make an ISO image of the entire directory you downloaded. I recommend using mkisofs for that purpose. If you don't already have mkisofs installed on your system, you need to install that in order to do this.
    	$ mkisofs -o openbsd_arc_2.3.iso ftp.se.openbsd.org/pub/OpenBSD/
    
    
  5. Start the emulator using this command line:
    	$ ./mips64emul -X -A2 -d obsd_arc.img -d openbsd_arc_2.3.iso ftp.se.openbsd.org/pub/OpenBSD/2.3/arc/bsd.rd.elf
    
    
    and proceed like you would do if you were installing OpenBSD on a real Acer PICA-61. (Answer 'no' when asked if you want to configure networking, and then install from CD-ROM.)

Once the install has finished, the following command should let you boot from the harddisk image:

	$ ./mips64emul -X -A2 -d obsd_arc.img ftp.se.openbsd.org/pub/OpenBSD/2.3/arc/bsd

You can experiment with adding -b to the command line, to enable dynamic binary translation. This is not 100% stable yet, but sometimes works.


How to install Ultrix/RISC in the emulator:

The following instructions should guide you through an install of Ultrix onto a disk image:
  1. Compile mips64emul:
            $ ./configure; make
       
    
  2. Create an empty harddisk image, which will be the root disk that Ultrix installs itself onto:
            $ dd if=/dev/zero of=rootdisk.img bs=1024 count=1 seek=800000
    
    
  3. Place your Ultrix installation media in your CDROM drive. (I'm assuming it is called /dev/cd0c here, replace that with the name of your CDROM drive, or the name of a .iso image file.) Then, start the emulator like this:
            $ ./mips64emul -X -b -M64 -D2 -d rootdisk.img -d bc:/dev/cd0c -j vmunix
    
    
    Once the first stage of the installation is done (restoring the root filesystem), you need to restart the emulator, booting from the new rootdisk, to continue the installation process. This is done by removing the bootflag ('b') from the second diskimage argument:
            $ ./mips64emul -X -b -M64 -D2 -d rootdisk.img -d c:/dev/cd0c -j vmunix
    
    

When the installation is completed, the following command should start Ultrix from the harddisk image:

        $ ./mips64emul -X -b -M64 -D2 -j vmunix -d rootdisk.img

Ultrix mostly seems to work with dynamic binary translation (enabled by the -b command line option). If you have a very fast host machine, and use bintrans, you might experience a weird timer related bug, which makes it impossible to logon to the system. It is triggered when the emulation goes faster than any real DECstation machine was capable of running. A temporary workaround is to add -I33000000 to fix the emulated clock speed to 33 million instructions per emulated second. (When using -CR4400, -I16000000 should be used instead.)

You can experiment with adding -Z2 (for emulating a dual-headed workstation) or even -Z3 (tripple-headed), and also the -Y2 option for scaling down the framebuffer windows by a factor 2x2. There is also a -z option for supplying names of X11 displays to use. The following example starts Ultrix on an emulated tripple-headed workstation, on three different displays (remote1:0.0, localhost:0.0, and remote2:0.0), using no scaledown:

        $ ./mips64emul -M64 -bND2 -jgenvmunix -d rootdisk.img -XZ3 \
            -z remote1:0.0 -z localhost -z remote2:0.0

The X11 displays may differ in bit depth and endianness. Unfortunately, there is no way yet to set the scaledown factor on a per-window basis, so the scaledown factor affects all windows.

(If you didn't use -Zn during the installation, and compiled your own /vmunix, then it will not contain support for multiple graphics cards. To overcome this problem, use the generic kernel, -j genvmunix, whenever you are running the emulator with a different setup than the one you used when Ultrix was installed.)

A note for the historically interested out there: OSF/1 for MIPS is quite similar to Ultrix, so that is possible to run as well. If you are unsuccessful in installing Ultrix or OSF/1 directly in the emulator, you can always install it on your real machine onto a real SCSI disk, and then copy the contents of that SCSI disk into a file (using dd(1)), and use that file as a disk image file in the emulator.


How to run Sprite for DECstation in the emulator:

Sprite was a research operating system at the University of Berkeley. The Unix Heritage Society (TUHS, www.tuhs.org) has made available a copy of a Sprite harddisk image for a DECstation 5000/200. If you want to find out more about Sprite in general, read http://www.cs.berkeley.edu/projects/sprite/retrospective.html.

The following instructions should let you run Sprite in the emulator:

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Download the Sprite harddisk image using ftp or http:
    	http://www.es.embnet.org/Services/ftp/misc/TUHS/sprite/ds5000.bt
    	or ftp://ftp.es.embnet.org/misc/TUHS/sprite/ds5000.bt
    	83973120 bytes, MD5 = ec84eeeb20fe77b758370d5e312e4a5e
    
    
  3. Start the emulator with the following command line:
    	$ ./mips64emul -X -b -D2 -M128 -d ds5000.bt -j vmsprite -o ''
    
    

The first time you boot up with the disk image, you will be asked a number of questions regarding network settings. If you feel like entering correct values, then you should use the following:

    Your machine's Ethernet address:    10:20:30:40:50:60
    Your machine's IP:                  10.0.0.1
    Subnet mask:                        0xff000000
    Gateway's Ethernet address:         60:50:40:30:20:10
    Gateway's IP:                       10.0.0.254

Unfortunately, at the end of http://www.es.embnet.org/Services/ftp/misc/TUHS/sprite/boot.txt, the following sad statement can be found:

    The bootable Sprite image is meant to be a demonstration of Sprite, not
    a robust Sprite system.  There are several missing things, such as 
    floating point and network support.

Once you are logged in as root, running xinit will start the X11 environment.

Sprite works fine with dynamic binary translation (enabled by adding -b to the command line options).


How to install Debian GNU/Linux for DECstation in the emulator:

NOTE: This is experimental, and extremely unstable. During my tests, even pressing the wrong key during the install (for example the wrong cursor key) can cause a kernel Oops. I think this has to do with interrupts from the serial controller. Hopefully using the -U command line option will reduce the risk for such crashes. (I haven't had time to come up with a clean solution to this yet; it feels like a buffer overflow in Linux' serial driver for the 5000/200, but it is also likely that it is a bug in mips64emul.)

The following steps should let you install Debian GNU/Linux for DECstation onto a harddisk image:

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Create an empty harddisk image, which will be the root disk that Debian installs itself onto:
    	$ dd if=/dev/zero of=debian.img bs=1024 count=1 seek=2000000
    
    
  3. Download the install kernel:
    	http://http.us.debian.org/debian/dists/testing/main/
    	    installer-mipsel/current/images/r3k-kn02/boot.img
    
    
  4. For a text-mode installation, start the emulator like this:
    	$ ./mips64emul -D2 -b -U -M64 -o 'console=ttyS3' -d debian.img -O boot.img
    
    

Debian GNU/Linux on DECstation in R3000 mode should work fine with dynamic binary translation, enabled by the -b command line option. (Without this option, the emulator is very slow.)

(If you want to, you can try -X instead of -o 'console=ttyS3' on the command line. This will cause Linux to use the graphical framebuffer. Unfortunately, Linux does not seem to have a driver for the DZ11 keyboard controller yet, so you cannot interact with the system. You will see the penguin in the upper lefthand corner while booting, and nicely rendered Unicode characters, but that's about it.)

You need to enter some values during the installation procedure, for example network settings. The following should work:

    DHCP:                       No, choose "Configure network manually"
    IP address:                 10.0.0.1
    Netmask:                    255.0.0.0
    Gateway:                    10.0.0.254
    Name server addresses:      10.0.0.254

Once the first phase of the install has finished, the following command should let you boot into Debian, and perform post-install configuration:

	$ ./mips64emul -D2 -U -b -M64 -o 'console=ttyS3' -d debian.img

It seems that there's a problem with getting a login prompt on serial console (at least when I've done test installs), so when the installation is finished and you're supposed to get a login prompt, you need to press CTRL-C and type quit, and then:

  1. Download a normal kernel (not a RAMDISK one).
    	http://http.us.debian.org/debian/dists/testing/main/
    	 installer-mipsel/current/images/cdrom/vmlinux-2.4.27-r3k-kn02
    
    
  2. Boot Debian using the following command line:
    	$ ./mips64emul -D2 -U -b -M64 -o 'console=ttyS3 root=/dev/sda1 rw
    		init=/bin/sh' -d debian.img vmlinux-2.4.27-r3k-kn02
    
    
    You'll enter single-user mode. You need to add a line to /etc/inittab, to enable logins via serial console.
    	sh-2.05b# echo 'T3:23:respawn:/sbin/getty -L ttyS3 9600 vt100' >> /etc/inittab
    	sh-2.05b# echo 'ttyS3' >> /etc/securetty
    	sh-2.05b# sync; umount /
    

The system should now be ready for everyday use.

Use this command to boot from the completely installed disk image:

	$ ./mips64emul -D2 -U -b -M64 -o 'console=ttyS3' -d debian.img


How to run Redhat Linux for DECstation in the emulator

NOTE: This is experimental, and extremely unstable. Read the note about '-U' in the section on how to install Debian.

The following steps should let you run Redhat Linux for DECstation in mips64emul:

  1. Compile mips64emul:
    	$ ./configure; make
    
    
  2. Download a kernel. This is a Debian kernel, but it works fine:
    	http://http.us.debian.org/debian/dists/testing/main/
    	 installer-mipsel/current/images/cdrom/vmlinux-2.4.27-r3k-kn02
    
    
  3. Download a root filesystem tree:
    	ftp://ftp.uni-wuppertal.de/pub/linux/mips/
    	 mipsel-linux/root/mipsel-root-20011216.tgz
    	19486676 bytes, md5 = 5bcb725c90209479cd7ead8ad0c4a414
    
    
  4. This is the tricky part: Create an ext2 filesystem image called redhat.img using the filesystem tree you just downloaded. The disk image should have a MS-DOS partition table (!), and then one or more ext2 partitions. (Use loopback mount, or similar. This is probably easiest to do on a Linux host.) However, in order to actually boot the system you need to modify /etc/fstab. Change
    	/dev/root               /               nfs     defaults        1 1
    	#/dev/sdc1              /               ext2    defaults        1 1
    	none                    /proc           proc    defaults        0 0
    	none                    /dev/pts        devpts  mode=0622       0 0
    
    
    to
    	#/dev/root              /               nfs     defaults        1 1
    	/dev/sda1               /               ext2    defaults        1 1
    	none                    /proc           proc    defaults        0 0
    	none                    /dev/pts        devpts  mode=0622       0 0
    
    
    (Note sda1 instead of sdc1.)

  5. To boot Linux, start the emulator like this:
    	$ ./mips64emul -D2 -U -b -M128 -o "console=ttyS3 root=/dev/sda1 ro" -d redhat.img vmlinux-2.4.27-r3k-kn02
    
    

If you need to boot into single user mode, try the following:

	$ ./mips64emul -D2 -U -b -o "console=ttyS3 root=/dev/sda1 rw init=/bin/sh" -d redhat.img vmlinux-2.4.27-r3k-kn02

Redhat Linux on DECstation in R3000 mode should work fine with dynamic binary translation (enabled by the -b command line option).

NOTE: You can add -XN and remove console=ttyS3 from the command line, if you want to use a graphical framebuffer. Unfortunately, Linux doesn't have support for the keyboard controller yet, so you cannot actually interact with the sytem. :-(


How to run Mach/PMAX in the emulator:

Read the following link if you want to know more about Mach in general: http://www-2.cs.cmu.edu/afs/cs/project/mach/public/www/mach.html

NOTE: Mach for DECstation requires some files (called 'startup' and 'emulator') which I haven't been able to find on the web. Without these, Mach will not get very far. These installation instructions are preliminary.

The following steps should let you experiment with running Mach for DECstation in the emulator:

  1. Compile mips64emul: (NOTE: --caches)
    	$ ./configure --caches; make
    
    
  2. Download the pmax binary distribution for Mach 3.0:
            http://lost-contact.mit.edu/afs/athena/user/d/a/daveg/Info/Links/Mach/src/release/pmax.tar.Z
    	7263343 bytes, md5 = f9d76c240a6e169921a1df99ad560cc0
    
    
  3. Extract the Mach kernel:
            $ tar xfvz pmax.tar.Z pmax_mach/special/mach.boot.MK83.STD+ANY
    
    
  4. Create an empty disk image:
    	$ dd if=/dev/zero of=disk.img bs=1 count=512 seek=400000000
    
    
  5. Load the contents of pmax.tar.Z onto the disk image. This is complicated, and should be described in more detail some time. For now, use your imagination. (For example using OpenBSD/pmax: disklabel -E rz1; newfs -O /dev/rz1a; mount /dev/rz1a /mnt; cd /mnt; download pmax.tar.Z using ftp; tar xzvf pmax.tar.Z; mv pmax_mach/* .; rmdir pmax_mach; mkdir mach_servers; cd mach_servers; cp ../etc/mach_init .; cp ../tests/test_service startup; dd if=/dev/zero of=paging_file bs=65536 count=400; cd /; sync; umount /mnt)

  6. Start the emulator with the following command:
    	$ ./mips64emul -D2 -X -d disk.img pmax_mach/special/mach.boot.MK83.STD+ANY
    
    


Experiments with other kernels and guest OSes:

Operating system kernels and other test programs can be downloaded from various places. Here are links to some of the kernels that I usually experiment with. (The degree to which these work in the emulator varies a lot.)


Changing the framebuffer resolution:

Running the emulator with the -X option turns on the X11 framebuffer. If the framebuffer device for the specific machine mode you are running is implemented well enough, then the framebuffer window will look exactly like the real machine's screen, pixel by pixel.

For many emulated graphics cards in the emulator (or emulation modes), the resolution is fixed. It is possible in some emulation modes, however, to change cards completely. The TURBOchannel cards on some of the emulated DECstation models support this. (The -D2 model is luckily one of these models.) This is not implemented as a command-line option yet, so you have to modify the source code to do this.

The turbochannel_default_gfx_card string defines the name of the graphics card. Look in src/machine.c for this line:

	char *turbochannel_default_gfx_card = "PMAG-BA";

Different cards have different resolution.

Card name:   Resolution:   Color depth:   Comments:  
PMAG-AA 1280 x 1024 B/W (perhaps problems with the cursor?)
PMAG-BA 1024 x 864 8-bit color Works fine in OpenBSD, NetBSD, Ultrix, Linux, Sprite
PMAGB-BA 1280 x 1024 8-bit color Works fine in OpenBSD, NetBSD, Linux, Sprite
PMAG-FA 1280 x 1024 8/24-bit Works fine in text-mode only with NetBSD and Ultrix,
(3D acceleration is not emulated)

If the framebuffer window is too large, is is possible to scale it down by adding -Y2 to the command line.


Networking:

It is possible to let the guest OS running inside the emulator get access to the Internet. If you are interested in the technical details, and the reasons why networking is implemented in the emulator the way it currently is implemented, you might want to read the networking section in the technical documentation.

The guest OS running inside the emulator uses a private IPv4 address, such as 10.0.0.1, and the emulator acts as a NAT-like gateway/firewall at IPv4 address 10.0.0.254. To the outside world it will seem like it is the host's OS that connects to other machines on the internet, not the guest OS.

NOTE: This is still experimental! As of 2004-07-21, ARP + ICMP + UDP + TCP are emulated well enough to let NetBSD and OpenBSD install via ftp, and use the network for many normal activities, but not everything works yet.


Hello world:

You might want to use the emulator to develop programs on your own, not just run precompiled kernels such as NetBSD. To get started, I'd recommend you do two things:

If the emulator is launched without specifying which machine to emulate, it will emulate only a CPU and very few devices. One of those devices is a console putchar() device, at a fixed address.

	/*  Hello world for mips64emul  */

	#define	PUTCHAR_ADDRESS		0xb0000000

	void printchar(char ch)
	{
		*((volatile unsigned char *) PUTCHAR_ADDRESS) = ch;
	}

	void printstr(char *s)
	{
		while (*s)
			printchar(*s++);
	}

	void f(void)
	{
		printstr("Hello world\n");
		for (;;)
			;
	}
This hello world program is available here as well: hello.c

I'd recommend you build a GCC cross compiler for the mips64-unknown-elf target, and install it. Other compilers might work too, but GCC is good because of its portability. Then try to compile the hello world program:

	$ mips64-unknown-elf-gcc hello.c -mips4 -mabi=64 -c
	$ mips64-unknown-elf-ld -Ttext 0x80030000 -e f hello.o -o hello --oformat=elf64-bigmips
	$ file hello
	hello: ELF 64-bit MSB mips-4 executable, MIPS R3000_BE, version 1 (SYSV), statically linked, not stripped
	$ ./mips64emul -q hello
	Hello world

	$ mips64-unknown-elf-gcc hello.c -c
	$ mips64-unknown-elf-ld -Ttext 0x80030000 -e f hello.o -o hello
	$ file hello
	hello: ELF 32-bit MSB mips-3 executable, MIPS R3000_BE, version 1 (SYSV), statically linked, not stripped
	$ ./mips64emul -q hello
	Hello world
As you can see above, a GCC configured for mips64-unknown-elf can produce both 64-bit and 32-bit binaries.

Hopefully this is enough to get you inspired. :-)


Porting operating systems to MIPS using mips64emul:

Is this a good idea? The answer is yes and no, depending on what you are trying to port to. If you are developing an operating system or operating system kernel of your own, and wish to target MIPS-like systems in general, then the answer might be yes, for experimental purposes.

However, if you think that you can port an operating system to, say, the Silicon Graphics machine mode of mips64emul and hope that your operating system will run on a real SGI machine, then you will most likely fail. mips64emul simply does not emulate things well enough for that to work. Another example would be specific CPU details; if your code depends on, say, R10000 specifics, chances are that mips64emul will not be sufficient.

In many cases, hardware devices in mips64emul are only implemented well enough to fool for example NetBSD that they are working correctly, while in fact they don't work very much at all. Please keep this in mind, if you plan to use mips64emul when porting your code to MIPS.


How to start the emulator with a disk image:

Add -d [prefixes:]diskimagefilename to the command line, where prefixes are one or more single-character options. Run mips64emul with no command line arguments to get a list of possible options.

Here are some examples. If you want to run a NetBSD/pmax kernel on an emulated DECstation machine, you would use a command line such as this:

	$ ./mips64emul -D2 -d pmax_diskimage.fs netbsd-pmax-INSTALL

NOTE: For some emulation modes, such as the DECstation mode, you do not have to specify the name of the kernel, if the disk image is bootable!

It is possible to have more than one disk. For each -d argument, a disk image is added; the first will be SCSI target 0, the second will be target 1, and so on, unless you specify explicitly which ID number the devices should have.

	$ ./mips64emul -D2 -d disk0.raw -d disk1.raw -d 5:disk2.raw netbsd-pmax-INSTALL
Note: In the example above, disk2.raw will get scsi id 5.

If a filename has a 'c' prefix, or ends with ".iso", then it is assumed to be a CDROM device (this can be overridden with a 'd' prefix, to force a read/write disk). For example, the following command would start the emulator with two CDROM images, and one harddisk image:

	$ ./mips64emul -D2 -d image.iso -d disk0.img -d C:second_cdrom.img netbsd-pmax-INSTALL
Usually, the device with the lowest id becomes the boot device. To override this, add a 'b' prefix to one of the devices:
	$ ./mips64emul -D2 -d rootdisk.img -d bc:install-cd.iso name_of_kernel
If you have a physical CD-ROM drive on the host machine, say /dev/cd0c, you can use it as a CD-ROM directly accessible from within the emulator:
	$ ./mips64emul -D2 -d rootdisk.img -d bc:/dev/cd0c name_of_kernel
It is probably possible to use harddisks as well this way, but I would not recommend it.

Using emulated tape drives is a bit more complicated than disks, because a tape can be made up of several "files" with space in between. The solution I have choosen is to have one file in the host's file system space for each tape file. The prefix for using tapes is 't', and the filename given is for the first file on that tape (number zero, implicitly). For files following file nr 0, a dot and the filenumber is appended to the filename.

As an example, starting the emulator with

	-d t4:mytape.img
will cause SCSI id 4 to be a tape device, using the following file number to name translation scheme:

File number: File name in the host's filesystem:
0 mytape.img
1 mytape.img.1
2 mytape.img.2
.. ..

If you already have a number of tape files, which should be placed on the same emulated tape, then you might not want to rename all those files. Use symbolic links instead (ln -s).

There is another advantage to using symbolic links for tape filenames: every time a tape is rewound, it is reopened using the filename given on the command line. By changing what the symbolic name points to, you can "switch tapes" without quiting and restarting the emulator.


How to extract large gzipped disk images:

Unix filesystems usually support large files with "holes". Holes are zero-filled blocks that don't actually exist on disk. This is very practical for emulated disk images, as it is possible to create a very large disk image without using up much space at all.

Using gzip and gunzip on disk images can be very slow, as these files can be multiple gigabytes large, but this is usually necessary for transfering disk images over the internet. If you receive a gzipped disk image, say disk.img.gz, and run a naive

	$ gunzip disk.img.gz

on it, you will not end up with an optimized file unless gunzip supports that. (In my experiments, it doesn't.) In plain English, if you type ls -l and the filesize is 9 GB, it will actually occupy 9 GB of disk space! This is often unacceptable.

Using a simple tool which only writes blocks that are non-zero, a lot of space can be saved. Compile the program cp_removeblocks in the experiments/ directory, and type:

	$ gunzip -c disk.img.gz | ./cp_removeblocks /dev/stdin disk.img

This will give you a disk.img which looks like it is 9 GB, and works like the real file, but the holes are not written out to the disk. (You can see this by running for example du disk.img to see the physical block count.)


Running userland binaries:

You can run (some) userland programs as well. This will not emulate any particular machine, but instead try to translate syscalls from for example NetBSD/pmax into the host's OS' syscalls. For this to work, you have to add --userland to the command line when running ./configure.

For example, running /bin/hostname or /bin/date and similarly trivial programs from the NetBSD/pmax distribution works:

	$ ./mips64emul -qu1 pmax_bin_hostname
	tab.csbnet.se
	$ ./mips64emul -qu1 pmax_bin_date
	Sun Jan 25 02:26:14 GMT 2004
	$ ./mips64emul -qu1 pmax_bin_sleep
	usage: pmax_bin_sleep seconds
	$ ./mips64emul -qu1 pmax_bin_sleep 5
	$ ./mips64emul -qu1 pmax_bin_sync
There's also an Ultrix4 emulation mode:
	$ ./mips64emul -qu2 ultrix4_bin_date
	UNIMPLEMENTED ultrix syscall 54
	UNIMPLEMENTED ultrix syscall 62
	Mon Feb  9 12:50:59 WET 2004
	$ ./mips64emul -qu2 ultrix4_bin_hostname
	tab.csbnet.se
NOTE: Userland syscall emulation is still in its beginning stages, so almost nothing works. It's mostly a proof-of-concept implementation, to show that such cross-platform userland emulation is possible.

IRIX userland emulation might theoretically also be possible to add.


Using a PROM image from a DECstation:

(The general ideas in this section applies to using ROM images from other machines as well.)

Raw PROM images from real machines can, in a few cases, be used. One case which I've tried is to use a DECstation 5000/125 PROM image. ROM images from other machines might work as well, but ROM code is usually much more sensitive to correctness of the emulator than operating system kernels or userland programs are.

The image first needs to be extracted from the machine. I presume that these PROMs are not legal to redistribute, so you really need to extract the PROM from a physical machine that you own. There are several ways to do this.

The easiest way is to hook up a serial console. The terminal must be able to capture output to a file.

These are approximately the commands that I used:

        >>cnfg                             Show machine configuration

        >>printenv                         Show environment variables

        >>setenv more 0                    This turns off the More messages

        >>e -x 0xbfc00000:0xbfffffff       Dump the PROM data

Remember that DECstation are little endian, so if the dump data looks like this:

        bfc00000:  0x0bf0007e
then the bytes in memory are actually 0x7e, 0x00, 0xf0, and 0x0b.

At 9600 bps, about 10KB can be dumped per minute, so it takes a while. Once enough of the PROM has been dumped, you can press CTRL-C to break out. Then, restore the more environment variable:

        >>setenv more 24

Now, convert the data you just saved (little-endian words -> bytes), and store in a file. Let's call this file DECstation5000_125_promdump.bin.

        $ ./decprom_dump_txt_to_bin DECstation5000_125_promdump.txt DECstation5000_125_promdump.bin
This binary image can now be used in the emulator:
	$ ./mips64emul -D3 -Q -M128 -q 0xbfc00000:DECstation5000_125_promdump.bin

	KN02-BA V5.7e   
	?TFL:  3/scc/access (1:Ln1 reg-12: actual=0x00 xpctd=0x01) [KN02-BA]
	?TFL:  3/scc/io (1:Ln0 tx bfr not empty. status=0X 0) [KN02-BA]
	...
	--More--?TFL: 3/scsi/cntl (CUX, cause= 1000002C)
	>>?
	 ? [cmd]
	 boot [[-z #] [-n] #/path [ARG...]]
	 cat SCRPT
	 cnfg [#]
	 d [-bhw] [-S #] RNG VAL
	 e [-bhwcdoux] [-S #] RNG
	 erl [-c]
	 go [ADR]
	 init [#] [-m] [ARG...]
	 ls [#]
	 passwd [-c] [-s]
	 printenv [EVN]
	 restart
	 script SCRPT
	 setenv EVN STR
	 sh [-belvS] [SCRPT] [ARG..]
	 t [-l] #/STR [ARG..]
	 unsetenv EVN
	>>cnfg
	 3: KN02-BA  DEC      V5.7e    TCF0  (128 MB)
	                                     (enet: 00-00-00-00-00-00)
	                                     (SCSI = 7)
	 0: PMAG-BA  DEC      V5.3a    TCF0
	>>printenv
	 boot=
	 testaction=q
	 haltaction=h
	 more=24
	 #=3
	 console=*
	 osconsole=3
	>>
(Note: at the moment, this doesn't work. I must have broken something when fixing something else, but this is what it looked like at the time.)

During bootup, the PROM complains a lot about hardware failures. That's because the emulator doesn't emulate the hardware well enough yet.

The command line options used are: -D3 for DECstation model 3 (5000/xxx), -Q to supress the emulator's own PROM call emulation, -M128 for 128MB RAM (because mips64emul doesn't correctly emulate memory detection well enough for the PROM to accept, so it will always believe there is 128MB ram anyway), and -q to supress debug messages. The 0xbfc00000 in front of the filename tells mips64emul that it is a raw binary file which should be loaded at a specific virtual address.


Feedback:

If you have found mips64emul useful, found a bug, or have other comments, then don't hesitate to mail me.

(Anders Gavare)