What do you do when you have a SPARC-based project, and no easy access to a SPARC machine to develop and thoroughly test your programs? Your client may offer to set up a VPN to their servers, but this is only practical if you do user-space application development. If you’re writing system software, and they don’t have a spare development machine, you’re pretty much on your own, because all those servers in the data centers are production machines, right?
Another possible option is to buy (or rent?) a development server from Oracle (the new owner of Sun Microsystems) or Fujitsu. The big advantage is that you could use the newest SPARC T3, or the not-so-new but still great and open-sourced OpenSPARC T2. Sadly, even their cheapest servers‘ prices at currently $18,642 are way above what a single developer could afford on his or her own limited budget.
For small toy projects, emulators like QEMU for SPARC (e.g. qemu-system-sparc64) could be an alternative… but they’re far from mature and can’t replace real hardware at the moment. It’s a shame that we simply don’t have the equivalent of Bochs, VMWare, or VirtualBox for SPARC.
Getting Used UltraSPARCs from eBay
Enters the second hand market for used equipment. The latest UltraSPARC IIIi based desktop computers that Sun produced were the SunBlades and Ultras. Ultras are currently still pretty expensive, but SunBlades are very affordable nowadays. By the way, please don’t confuse the SunBlade workstations with the machines of the same name that are Blades to be put into a Blade Center. I’m talking about the workstations here.
So I went looking for SunBlade workstations, and was happy to find:
- SunBlade 1500 (Silver), with a single UltraSPARC IIIi processor with 1500 MHz, and 2 GB of ECC-RAM. [UP]
- SunBlade 2500 (Red), with two UltraSPARC IIIi processors (a real multi-processor board) with 1280 MHz each, and 4 GB of ECC-RAM. [MP]
Both used machines were in the range of 60 to 80 Euros each, plus shipping. All in all, a very decent deal.
The SunBlade 1500 (Silver) Single Processor
The SunBlade 1500 (Silver) is a nice little workstation, that looks pretty much like a regular PC… at least from the outside.
The Outside View of the SunBlade 1500 (Silver)
The outside is a regular case, with the characteristic SUN logo. Oh, it’s so sad that Sun Microsystems has been cannibalized by Oracle!
The front side is just as boring… but look at the nice little extra: a flash card reader at the top. At the bottom, there are USB 2.0 connectors, and the usual sound plugs. Have you also noticed that there’s no CD/DVD-ROM? Installing an operating system on this machine will require netbooting…
The back of the SunBlade looks a lot like a regular PC too. In the following photo, you can see from left to right two USB 1.0 connectors for the keyboard (any USB keyboard will do, you don’t need a Sun Keyboard!) and the mouse. They must be plugged there and not in USB 2.0 ports, since this is a limitation of the Sun firmware, a.k.a. the OpenBoot PROM (OBP). Next to the two USB ports are two RS-232C serial connectors. Connector #1 is a serial console that can be connected via a null-modem cable to another computer, so that we can perform a headless installation. There’s a parallel connector for printers, and Ethernet (bge) 100 MBit/s connector and sound connectors.
You’ll also notice the VGA connector on a graphics adapter (framebuffer in Sun jargon) that I had to replace (more on that in a moment), and finally three USB 2.0 and two Firewire connectors on a dedicated I/O card at the right of the picture (bottom of the case).
Inside View of the SunBlade 1500 (Silver)
So now is the time to open the cover and look inside. The cover itself is easy to remove. Just open both screws (no need for a screwdriver, they’re big enough to open with your fingers), and slide the cover open.
Interestingly, inside the cover, you can see schematics, just like in professional equipment. If you’re not used to real machines, you’ll be surprised at the high manufacturing quality. This is what the cover looks like from the inside. Nice, eh?
Okay, so you’re really curious by now! Don’t be disappointed, the inside looks a lot like off-the-shelf PCs.
As you can see, the motherboard is pretty standard, seen from the outside. Of course, the CPU isn’t: it’s an UltraSPARC IIIi with 1500 MHz, and the layout of the JBus interconnect is different from the PC architecture. Furthermore, the PCI expansion slots are standard (the white connectors run at 33 MHz, and the green one at 66 MHz), but you can’t plug in regular cards in there. The reason is that those cards must contain OBP firmware, that will be queried by the OBP firmware of the motherboard.
The IDE connectors are without surprises. You can throw at a Sun almost any IDE PATA disk or CD-ROM/DVD-ROM PATA device. As you see, the SunBlade I bought had already an HDD, and it was still operational.
Looking more closely at the motherboard, you can see how the RAM is being covered by a green piece of plastic. This is Sun’s way of getting enough air to flow from the right to the left fans to cool the RAMs.
On the second close up, you can see the PCI expansion boards. I had to replace the framebuffer (graphics adapter) with another one (see below), but I decided to put it into a PCI 33 MHz slot, leaving the green PCI 66 MHz slot empty for further uses. Perhaps not the wisest of choices, but we’ll see how far it takes me. The other expansion card is an I/O card with 3 USB 2.0 and 2 Firewire ports on the outside, and, should you need more ports, a couple of USB 2.0 ports on the inside as well.
Replacing the XVR-1200 of the SunBlade 1500
The frame buffer that came with the SunBlade 1500 was an XVR-1200, a nice high end accelerated graphics adapter.
The DVI-I connectors of the XVR-1200 (dual head display for two monitors) didn’t fit quite well the VGA connectors of my LCDs and CRTs, neither did the other connectors, so I had to get a DVI-I to VGA adapter.
Because Sun never released the specs of the XVR-1200, there’s no open source driver for them. In other words, Xorg doesn’t recognize them. Only the deprecated Xsun server in Solaris 10 9/10 does. In the console mode, OpenBSD/sparc64 is able to access it with the inextricable frame buffer ifb(4) (wildcatfb) driver, but with a lot of issues — X, a.k.a. Xenocara crashes on them. On FreeBSD/sparc64, the kernel doesn’t even boot if it sees this adapter (but it can use the serial console anyway…), and the Linux kernel hangs as soon as it tries to switch the console to graphics.
In other words, I had to replace the XVR-1200 with something better… or at least with a supported frame buffer. Now, you can’t simply put a usual PCI graphics adapter for PCs in a Sun, because the firmware OpenBoot Prom (OBP) has a mechanism to query the peripherals. Each peripheral, including the frame buffers, are expected to reply to OBP, and in the case of PCI adapters, they must usually contain a bit of OBP themselves. Since I don’t have the time and patience to flash a regular PCI graphics adapter with OBP (something that is not for the faint of heart, and definitely aking to black magic), I ordered a couple of used Sun PGX64 that feature the well-supported and understood ATI Rage XL chipset:
Oh, one nice touch is that the PGX64 features a VGA connector. No need to use a DVI-I to VGA adapter to connect my LCD and CRT.
The SunBlade 2500 (Red) Multi Processor
The SunBlade 2500 (Red) is bigger than the SunBlade 1500 (Silver). Not only because it is a multiprocessor, but its case is bigger too. It is also quite heavy to carry around. Be careful to wear good shoes if you decide to move this machine: you don’t want it to fall on your feet! I really, really mean it! :-)
Like the 1500, the front side of the 2500 features a flash card reader at the top. But since this workstation is meant to be more of a server than a client, there are no USB 2.0 connectors at the front. This machine came with a DVD-ROM… which I think is faulty and that I may replace later.
The back of the SunBlade 2500 is interesting. From left to right: 4 USB 1.0 connectors. You must connect the USB keyboard and USB mouse to one of those, and not to the USB 2.0 connectors at the bottom (right, on the picture), because of a limitation of the OBP firmware. Then, there’s an RS-232C DE-9M connector for the serial console, a parallel port for printers, an Ethernet connector for 100 MBit/s, and a SCSI connector for (additional) SCSI disks and -devices.
The first PCI card provides an additional RS-232C serial connector and a couple of music plugs. The next PCI card with the DVI-I connector is a Sun XVR-600, which is not well supported by open source software, so I’ll replace it too with a Sun PGX64 ATI Rage XT framebuffer.
The next three slots belong to a Bellerophon multicard: you can see an I/O port (RS232C serial and parallel), then a card with VGA, Ethernet, … and another one with additional connectors. More on the Bellerophon multicard (SunPCi-3) below.
The last card at the right of the photo (bottom of the case) is a regular I/O card with 3 USB 2.0 and 2 firewire connectors.
So, let’s open the case. Just as in the SunBlade 1500, there are schematics on the inside of the panel too. In the case of the SunBlade 2500, those schematics are actually really useful and highly appreciated!
Okay, now for the main photo of this blog entry: the insides of the SunBlade 2500 (Red) multiprocessor machine. The first noticeable thing are the two separate (!) UltraSPARC IIIi CPUs. This is a real multiprocessor, not the multicore variety of modern SMP machines (x86, sun4v alike). Each processor has access to the RAMs. As you can see, there are also 2 separate bridges to the JBus. Programming such a beast could be interesting!
There are two sets of 4x banks for ECC RAM. On this machine, 4 GB RAM (4 x 1 GB) out of a possible of 8 GB RAM are installed. Yes, UltraSPARC IIIi is a SPARC V9 64-bit architecture, it supports more than 4 GB of RAM.
Two kinds of PCI slots are present. The white slots are for PCI 33 MHz, and the green slots for PCI 66 MHz. I removed the Bellerophon multicard and XVR-600 framebuffer (more on this below), and installed an PGX64 ATI Rage XL framebuffer in one of the fast PCI slots. The other remaining card is the usual I/O card with the USB 2.0 and firewire ports.
As you can see, there are two (SCSI-) HDDs in this machine. I didn’t buy them extra: they were part of the eBay deal. Great! However, I suspect that the DVD-ROM is faulty, because I wasn’t able to install ANY OS from CDROM. I’ll replace it sometimes in the future. For now, netbooting this machine is the only option.
Replacing the XVR-600 of the SunBlade 2500
Just like the XVR-1200 of the SunBlade 1500, the XVR-600 that came with the SunBlade 2500 is only supported by Xsun (Solaris 10 9/10), and barely by OpenBSD in console mode… and not at all on other operating systems.
And just like the XVR-1200, it features a DVI-I connector that needs a DVI-I to VGA adapter… so it had to go, and be replaced by a Sun PGX64.
Removing the Bellerophon multicard from the SunBlade 2500
The first time I’ve tried to use this SunBlade 2500, I was pretty confused. I couldn’t get an image on my VGA-only LCD and CRT. The reason was that the only VGA connector belonged to the so called Bellerophon multicard. The other framebuffer connectors on the XVR-600 were DVI-I and others, and I didn’t get a chance to use them, because my monitors don’t have a DVI-I connector. At that time, I didn’t have a DVI-I to VGA adapter yet.
Anyway, there was an orange LED blinking on the card with the VGA adapter, but I had no way to get the monitors to display anything at all, when connected to this card. Except for the serial console (of the SunBlade 2500, not of the Bellerophon card!) which worked perfectly, it meant: no display for now, for unknown reasons.
After opening the case, and looking at this blinking LED card more closely, I was surprised to see that it occupied 3 adjacent slots, and the card itself consisted of a long main card, and an associated “daughter card”, plugged into two PCI connectors simultaneously.
Hmmm… pretty unusual. Actually, this Bellerophon multicard, a SunPCi-3, is a complete x86- (AMD-) based PC-on-a-card! This PC-card was supported by old versions of Solaris, up to Solaris 9 (I think, perhaps Solaris 10?), and was used to run x86-applications, typically Windows stuff, under Solaris/SPARC with the /opt/SUNWspci3/bin/sunpci utility, at native speed (i.e. no need for Bochs IA-32 emulation). Very neat actually. I wished some manufacturer would create a SPARC-T3-on-a-card that would fit into an x86 machine, and would provide OSS drivers for it!
However, this Bellerophon card combo is currently pretty useless without drivers on Solaris 10+ (?) and definitively pointless on Linux/SPARC, FreeBSD/SPARC, OpenBSD/SPARC etc… at the moment. To save myself some headaches, and to save some electricity as well, I removed that card for the SunBlade without ill effects.
This concludes the show of the SunBlade 1500 (Silver) UltraSPARC IIIi uniprocessor and SunBlade 2500 (Red) 2x UltraSPARC IIIi multiprocessor workstations.
In the next couple of postings, I’ll show how to install Solaris 10 10/09, OpenBSD/sparc64 4.9 and FreeBSD/sparc64 8.2 on them using netboot, i.e. over the net without any CDROM/DVD-ROM. I’ll also show how to install Linux Debian Squeeze 6.0.1a SPARC on the SunBlade 1500, after adding a DVD writer.