On November 29, 2014, I posted the following message to the Classic Computer Mailing List.

Folks,

For some reason I got it in my head that writing an AT&T 3B2
emulator might be a good idea. That idea has pretty much
been derailed by lack of documentation.

I have been unable to find any detailed technical description
of any 3B2 systems. Visual inspection of a 3B2 300 main board
reveals the following major components:

  - WE32100 CPU
  - WE32101 MMU
  - 4 x D2764A EPROM
  - TMS2797NL floppy disk controller
  - PD7261A hard disk controller
  - SCN2681A dual UART
  - AM9517A Multimode DMA Controller

How these are addressed is anybody's guess.

To even dream of doing an emulator, I at least need to know
the system memory map -- what physical addresses these devices
map to. Without that it's pretty pointless to get started.

If anyone has access to this kind of information, please drop
me a line. Otherwise, I'll just put this one on the far-back
burner!

-Seth

When I sent that message, I had no idea that it would lead to almost four years of effort, and perhaps if I had known I would have given up. But thankfully I didn’t, and today I’m happy to say that the effort was, at long last, successful. The 3B2/400 emulator works well enough that I have released it to the world and rolled it back into the parent SIMH source tree.

Because this project required so much reverse engineering, and because documentation about the 3B2 is still so scarce and hard to come by, I wanted to take the time to document how the emulator came about.

Background

In the mid 1980s, when big companies were entering the computer market left and right, AT&T made a bold and (in retrospect) unwise move by introducing not just one computer, but three.

AT&T entered the PC compatible market with the PC 6300, which was really just a re-branded Olivetti M24. Simultaneously, they entered the UNIX market with the AT&T UNIX PC (a personal, single-user UNIX system based on the 68010 microprocessor), and finally with the 3B2, which was a small computer designed as a workgroup server. The 3B2 could accommodate up to 50 simultaneous users connected by dumb terminal or modem, and could even support AT&T’s “Blit” bitmapped terminal. It probably helped that AT&T Bell Labs invented UNIX, so the company had some history behind it when it brought the 3B2 to market.

Marketing wise, none of these computers did blockbuster sales. The 3B2, though, had decent success due to its small size and modest cost—just $9,950 entry level.

The 3B2 was special largely because it was so perfectly designed for running UNIX. Its CPU, the WE 32000 series, was designed by Bell Labs and Western Electric based on earlier work building a CPU meant to map to the C programming language. The CPU was ideally suited to UNIX because it was ideally suited to C. In fact, the 3B2 became the base porting platform for AT&T UNIX System V Release 3 (SVR3).

When I learned about the 3B2, I quickly became interested in playing with one. Unfortunately, they’re pretty rare to come by nowadays. Maybe ten or twenty years ago they were fairly common on the surplus market, but not any more. I went looking for an emulator instead, but no emulators were to be found, so I did something regrettable: I decided to write my own.

Emulating the 3B2

The first step in any emulator is gathering up all the source material and documentation you can find about the system you want to recreate. This usually means schematics, maintenance documents, offline diagnostics, design memos, source code, firmware, anything that shows the internals of the system and how it was implemented. This is arguably the most important step, because without this you can’t even get started.

Well, bad news: For the 3B2, none of this was available. No schematics. No internals docs. No offline diagnostics. No firmware. Nothing. Despite my best effort, Google yielded very little but high-level user manuals and some floppy diskette images. The only goldmine I found was a WE32100 microprocessor manual

I immediately sent out a call online to see if anyone had anything more. Eventually, I was able to gather three critical sources: First, a copy of the 3B2/400 boot ROM, dumped by a friend with access to a real machine. Second, a high resolution picture of a 3B2/400 motherboard. And third, a copy of the AT&T UNIX SVR3 kernel source code.

At last, I had something to go on.

Figuring Out the Basics

Visual inspection of the 3B2 motherboard was a first critical step. I’ve annotated the photograph above. It revealed every major component in the system, and allowed me to look up data sheets for each part. Still, I had no idea how the whole system was actually connected. If you’re going to emulate a computer, you need to know how all the pieces fit together, and that can’t be gleaned from a picture. That’s when I turned to the SVR3 kernel source code. These sources are pretty revealing, but of course, they would have to be. The kernel needs to know the layout of the system in order to operate.

The first thing I wanted to know was, what does the memory map look like? What physical addresses does the system use to communicate with all of these devices? Thankfully a little poking around led me to the kernel linker table, showing the memory location for every component in the system. This block told me pretty much everything I needed to know to get started. Although there are no comments, the big clue here came from digging through the source code to find these symbol names and making some educated guesses about what exactly they corresponded to.

Decoding the Firmware

At around the same time that I was trying to understand the memory map and system components, I started to decode the ROM image so I could figure out how the 3B2 boots. I didn’t (and still don’t) have the firmware source code, so I had to disassemble the boot ROM, and unfortunately there was no disassembler. I had to write one.

Writing a basic disassembler is pretty straight-forward for the most part. You just read in some bytes, figure out what instructions and operands those bytes correspond to, and spit out the mnemonic instructions as text. I’d written disassemblers for the 6502, but of course the WE32100 is much more complex, so it took a little longer than I was planning.

Once decoded, reading the firmware was pretty useful. For one thing, it has a lot of CPU exercise tests, tests that would eventually verify that the emulated CPU was doing the right things. But more importantly it provides a slew of functions that can be called by the SVR3 boot code. These added to my understanding of the system architecture.

Armed with the knowledge gleaned from the SVR3 source code and the boot ROM, I had enough to get started writing something.

The Emulation Framework

I’ve written emulators for small computers before, but this time, for a system so complex, I knew I wanted to use a framework that already existed and had a lot of support. I turned to the SIMH framework (available on GitHub), which I consider to be more or less the gold standard for emulating minicomputers. SIMH gave me the ability to bootstrap things very, very quickly, because most of the hooks for emulating a minicomputer are already there. Console and serial IO, disk image attaching and detaching, debugging support, it’s all baked in. So I rolled up my sleeves, forked SIMH on GitHub, and started work.

The CPU

The first thing I worked on was a CPU emulation. I assumed (incorrectly) that this was going to be the most challenging part, and of course an emulator can’t do anything useful without it, so I got a head start as soon as I could.

The WE32100 is a complex microprocessor, and I’m grateful that I had an excellent document to follow, “Maxicomputing in Microspace”, the official WE 32100 Processor Information Manual. This manual covers in excruciating detail the inner workings of the WE32100 and each instruction. It also describes each interrupt and exception microsequence in such glorious detail that emulating them became… well, not easy. But certainly manageable.

Within a month or so, I had enough of the CPU emulated that it could pass the ROM diagnostics and try to print something to the console.

The Console

Console emulation was tricky. The 3B2 has two serial ports on board, CONSOLE and CONTTY. CONSOLE is, naturally, where the console terminal plugs in, and it is the port that receives all the boot messages.

CONSOLE and CONTTY are both controlled by a single SCN2681A dual-UART chip. The SCN2681A datasheet is pretty good, but not quite as detailed as I’d like, so a lot of educated guesses had to be made about how the SCN2681A works internally. The first pass was actually pretty brain-dead and didn’t behave properly at all, but did print boot messages well enough to verify that the CPU was trying to come up.

As the emulator progressed and I learned more about the SCN2681A, the CONSOLE and CONTTY emulation got better. SIMH provided almost all of the boilerplate necessary to handle keyboard input and terminal output. I will leave out the gory details, but eventually a full 3-character receive ring buffer, transmit buffer,  all control registers, and modem line control were accurately emulated.

By this time, I was able to interact fully with the on-board ROM monitor and its small set of built-in commands. The next challenge was to try to boot UNIX from a floppy image.

The Floppy Drive

When the CPU and console at least mostly working, I turned my attention to the floppy disk controller, a Western Digital 2797 compatible chip. Again, a datasheet was available, and described the chip well enough to get started emulating it.

The 3B2 floppy drive is an off the shelf CDC model that supports quad-density diskettes (96tpi, 80 track DSDD). SIMH once again provided all the boilerplate necessary to read and write a diskette image, so all I really needed to do was figure out how the 3B2 system talked to the controller, emulate enough of the controller to respond correctly to commands, and then read and write sectors on a mounted image.

With a skeleton controller written, I was finally able to try to boot the first UNIX installation diskette. Naturally, it didn’t work. (Nothing works the first time in an emulator)

The Hard Drive

I didn’t know it at the time, but it turns out that having a working hard drive controller is essential to getting UNIX to boot off of a floppy. The 3B2 has an NEC μPD7261A Winchester controller, and supports one or two MFM hard disks in 31MB, 43MB, 72MB, or 161MB capacities.

SIMH provides excellent hard disk image access routines. You really just need to use the SIMH disk library, and a few routines allow reading and writing sectors to the disk image. But, that was only part of the challenge. The larger problem was figuring out how the Winchester controller interfaced with the host and responded to commands. In some ways it was very similar to the floppy controller. In other ways, much different.

I soon discovered that a key problem was going to be interrupts (more on that later). Computers do not operate infinitely quickly: Disks are slow, it may take 250 milliseconds to do a disk operation, an extraordinarily long time when your CPU is operating at an instruction every microsecond. Normally the system will issue a command to the disk controller, and then go off to do other things while it waits for I/O to complete. When the disk controller has completed its work, it has to interrupt the system board and say “my work is done”.

Part of the job of emulating a disk controller, then, is to artificially delay certain actions and only interrupt after a certain number of microseconds or milliseconds has gone by. SIMH supports this through several routines that put jobs on an internal execution queue with a specified delay in either simulated machine steps, or microseconds through a calibrated clock.

It took ages, but eventually the hard disk controller emulation was working well enough to accurately read and write a virtual hard disk image.

System Timers

The 3B2 has a timer chip, the NEC μPD8253C, that provides three interval timers. This is a very complex chip, in that each independent timer can be configured a broad number of ways and perform a number of different functions. It was, in fact, one of the harder parts of the system to get working accurately.

The most important timer is Timer #1, configured by the boot ROM to deliver a 100 Hz square wave. Once every 10ms, the timer generates a Level 15 interrupt. This interrupt, it turns out, drives much of UNIX. Without it, UNIX can’t schedule or switch tasks. So it was essential to get right.

This is another area where SIMH shines. SIMH has support for fully calibrated system timers, meaning that you can tell SIMH to trigger an action once every 10 ms of wall clock time, and it will do so. This is how the 3B2 emulator’s 100Hz system timer works.

The MMU

By this time, I was actually getting very close to booting UNIX. In fact, I was able to get much of the way through the boot process, until the computer turned on virtual memory. Then everything came crashing down, because I needed something very important: A Memory Management Unit.

The 3B2 uses a WE32101 MMU, a custom part manufactured by Western Electric and part of the WE32000 chipset. The MMU provides translation between physical and virtual memory addresses, and it’s a complicated piece of engineering. In fact, it was tougher to get the MMU working correctly than it was to get the CPU working correctly.

At its heart, the MMU has a set of configuration registers that tell it where to find translation tables in main memory. Without going too much into detail here, the translation is controlled by a very complex state machine, and the MMU must cause an exception in the CPU whenever translation cannot happen. These exceptions are essential to get right, because the operating system relies on them to keep its process virtual memory tables up to date.

Frustratingly, the only document I had at the time that described the MMU was the WE32101 data sheet. The data sheet was good, but not great, and although it described the translation algorithm at a high level, it didn’t go into great detail.

The MMU took what felt like ages to get done, and little edge cases kept popping up as I went along. It wasn’t until very recently indeed that I felt like the MMU emulation was completely polished and (hopefully) bug-free.

… And The Rest

Finally, there were only a few bits and pieces of glue logic left to emulate. Among them were:

• The DMA controller for moving blocks of memory around between the system board and peripherals.
• The battery-backed Time-of-Day clock that provides the current date and time on power-up.
• A set of latches called the CSR that hold run-time and interrupt status.
• The Non-Volatile RAM (NVRAM), which holds system configuration between boots.

All of these were important to a working system, some more than others. The DMA, CSR, and NVRAM were particularly important, but were also thankfully trivial to get working.

Interrupts

But still, the 3B2 would not boot.

After all this work, it was still incredibly buggy. System timing felt impossible to get right. It all came down to interrupts.

The floppy controller, hard disk controller, timer, and UART all generate interrupts in the system. I was so sure I had all of the interrupts correct, but no matter how much I played with interrupt timing, the system would crash before getting to the OS installation screen. At several points I gave up, frustrated, and shelved the emulator for months at a time, convinced that I was missing some important architectural detail that lay hidden in internals documentation I would never see.

It wasn’t until late last year that I had the breakthrough necessary to lead to success. You see, I had been working under the assumption that interrupts are fired and handled (or not handled) immediately after firing. If the CPU is ready for an interrupt, it would handle it. If not, it would ignore it. But that’s not actually how interrupts in the 3B2 work.

What really happens is that interrupts are latched. When a peripheral generates an interrupt, the system latches the interrupt signal in the CSR. If the CPU isn’t ready to handle the interrupt yet, that’s fine, it’ll keep processing until it is. Once the CPU is ready, only then does it acknowledge and handle the latched interrupt, and clear the CSR latch.

This was the missing piece, the silver bullet. I spent an afternoon refactoring my interrupt code, and booted the installation floppy image again.

This time, it worked. I was greeted by this screen, and a did a little dance of joy:

At long last, I installed UNIX, and it was good.

Ongoing Work

An emulator like this never really feels done. Since I got the base system working, there have been several areas I continue to work on. Of course, every now and then I find a bug, and fixing bugs is priority number one. There have been several critical bugs with the MMU, with system timing, and with the UART. At the moment, I don’t know of any outstanding bugs, but of course that doesn’t mean there aren’t any yet to be discovered.

Outside of bugs, most of the work has been around adding support for expansion cards. The 3B2/400 has a backplane with 12 slots that can accept I/O expansion cards, many of which are based around a common architecture called “Common I/O”, or CIO for short. So far, I have implemented two expansion cards:

• The PORTS card, which supports four dial-up or remote terminal serial ports.
• The CTC card, which adds a removable 23MB Cipher QIC tape cartridge.

There are more cards I’d like to do, but reverse engineering these cards takes a lot of time and effort. PORTS and CTC were low-hanging fruit, and other cards will be much more challenging.

Conclusion

Four years is a long time. Of course I didn’t work on the emulator non-stop for four years. I have a day job, and a life that needs attending to. There were long periods when I didn’t work on the emulator at all.

But now that it’s in a state that I can share it with the rest of the world, it feels good. I’m glad I tackled this project, saw it through, and made something that works and can help preserve this piece of computing history.

Now it’s time to look for my next project.