Cray J90 (computer)

The Cray J90 series was a minisupercomputer manufactured by Cray Research from 1994 - 1998. This learning resource documents the restoration of a model J916 that was donated to the Retro-Computing Society of Rhode Island (RCS/RI) historic computer collection.

These systems have multiple scalar/vector parallel processors. Unlike larger, more powerful, supercomputers that required liquid cooling, these used air cooling.

Index of Cray J90 Wikiversity subpages:

Hardware

System WorkStation (SWS)

SPARCstation 5 (for jumpers see: Sun SparcStation 5 / SparcServer 5)
- Node: hbar
  - STP1012PGA-85 microSPARC-II CPU
  - Two internal 4 GB SCSI drives
  - SBus
    1. SunFDDI
    2. quad fast Ethernet
    3. TCX graphics (uses AFX Bus slot, instead of SBus connector)
      - See: Sun 501-2337 S24 24-Bit Color Frame Buffer - X323A or X324A
    4. ~~10base5 or 10base2 Ethernet~~

IO Subsystem (IOS)

VMEbus

IOP-0 - Themis SPARC 2LC-8 D1 S26950023
- Ethernet: 00 80 B6 02 9E 40
- Host ID: FF050078
- Node: sn9109-ios0
- Fujitsu SPARC MB86903-40 CPU Processor IOSV BOOT F/W REV 1.4
- A/B serial
- AUI Ethernet
- SCSI
  - tape drive
  - CDROM
IOBB-64 - Y1 Channel (Connection to processor board)
EI-1 – System Ethernet
- Rockwell Int'l/CMC Network Products P/N 320057-06
DC-6S - SCSI Disk Controller
- Interphase H4220W-005 SCSI-2 Fast Wide High Voltage Differential controller
  - PE-30S disk tray - 2c x 2t x 9.11 GB (36.44 GB formatted) specs^[1] for each disk:
    - Seagate ST410800WD Elite 9
    - 10.8 GB unformatted capacity
    - 9.08 GB formatted capacity
    - 5,400 rpm
    - 7.2 MB/s peak transfer rate (formatted)
    - 4.2 – 6.2 MB/s sustained transfer rate (formatted)
    - 1.7 – 23.5 ms access time (11.5 ms average)
    - Aggregate transfer rate capacity of controller is unknown
    - Maximum number of drives per controller is unknown
  - Sun StorEdge D1000. (8 X Seagate ST150176LC disk array w/ 18 GB, 10,000 rpm, SE/LVD)
(empty)
(empty)
IOP-1 - Themis SPARC 2LC-8 D1 S26950078
- Ethernet: 00 80 B6 02 6B 40
- Host ID: FF050023
- Node: sn9109-ios1
- Fujitsu SPARC MB86903-40 CPU Processor IOSV BOOT F/W REV 1.4
- A/B serial
- AUI Ethernet
- SCSI
IOBB-64 - Y1 Channel (Connection to processor board)
DC-5I - Disk Controller (IPI)
- Xylogics SV7800 IPI-2 controller “The DC-5I disk controller is an intelligent and high-performance controller that can sustain the peak rates of four drives simultaneously to mainframe memory. You can attach up to four DD-5I drives to a DC-5I controller.”^[1]
  - PE-10I disk tray - 2c x 2t x 3.4 GB (13.6 GB unformatted) Specs^[1], For each DD-5I disk:
    - Seagate ST43200K Elite 3
    - 2.96 GB formatted
    - 3.4 GB unformatted
    - 5,400 rpm
    - 12.4 MB/s peak transfer rate (unformatted)
    - 9.5 MB/s peak transfer rate (formatted)
    - 6 - 8.5 MB/s sustained transfer rate (formatted)
    - 1.7 – 24 ms access time (11.5 average)
FI-1? system FDDI
- Interphase H04211-004
(empty)
(empty)
(empty)
(empty)
(empty)
(empty)
(empty)
(empty)
(empty)
(empty)

Allied Telesis CentreCOM 470 MAU with 4 AUI and 1 10bse2

For jumpers on VME boards see the hardware reference manual.^[2]

VME slots are labeled C1 – C20 in a 6-4-6-4 slot arrangement. Any of the four sections could be (but are not) jumpered to an adjacent section.

VME0 C1 – C6
VME1 C7 – C10
VME2 C11 – C16 (unused)
VME3 C17 – C20 (unused)

Note: the disk controller notation used here is [c]ontroller, SCSI [t]arget address, and [GB] capacity.

The IOS (IO Subsystem) contains two IOPs (IO Processors, each with its own VME backplane) running the VxWorks IOS-V operating system.

Need to check the MAC addresses on the Themis IOPs to see if they match our custom config file. Also, document IP address mappings for MACs. The IOPs use the 10/8 private subnet.

Central Control Unit (CCU)

On the Cray Y-MP EL and EL98 the LED panel batteries take 36 hours to charge and last for 72 hours. The J90 uses four Eveready CH50 cells; these are standard D size Ni-Cd cells at 1.2 V and 1.8 Ah. These will be replaced with EBL Ni-MH cells at 1.2 V and 10.0 Ah. With these new batteries it takes about 10 hours to fully charge discharged batteries with a standard charger. There is a switch on the back of the CCU to disable the batteries to prevent them from discharging while the system is off.

Mainframe

Serial number: 9109. Node: boson

MEM0
MEM1
CPU0 with two Y1 channels
CPU1
(empty / disabled)
(empty / disabled)
(empty / disabled)
(empty / disabled)

Our specific model is J916/8-1024 (J90 series with a backplane that has space for eight modules. The backplane is only wired for four modules. There are two boards with a total of eight CPUs and two memory boards with a total of 1 GB RAM total. (We need to verify RAM size.) Based on the IOP JTAG boundary scan results, all of the eight processors are enabled.
J90 Series: “The allowable backplane types are 1x1, 2x2, 4x4, and 8x8. There can be up to 8 processor modules with each module containing 4 CPUs. There can be up to 8 memory modules with a combined range of 0.25 to 4 Gbytes.”^[3] It is not clear if Cray ever manufactured or sold a 1x1 J916 backplane.
J90se series: “The Cray J90se mainframe runs the UNICOS operating system. It allows backplane types of 2x2, 4x4, or 8x8 processor modules. A Cray J98 system has up to 2 processor modules for a total of 8 CPUs. A Cray J916 system has up to 4 processor modules for a total of 16 CPUs. A Cray J932 system has up to 8 processor modules for a total of 32 CPUs. The combined memory capacity of these configurations ranges from 0.50 to 32 Gbytes.”^[3] (J90se is “scaler enhanced; the scaler processors are upgraded from 100 to 200 MHz, but the vector processors are still 100 MHz.)
"Memory has a peak bandwidth of 32 words per clock period (CP) (25.6 Gbytes/s) for a 4 X 4 backplane (J916) configuration and 16 words per CP (12.8 Gbytes/s) for a 2 X 2 backplane (J98) configuration."^[4]
"Data travels from a peripheral device, across a data channel to the device controller and then from the device controller, across the VMEbus to the I/O buffer board (IOBB). From the IOBB, data travels to the mainframe memory through the 50-Mbyte/s data channel."^[4]

Installed software

CDROM install media

CrayDocs for UNICOS 8.0.3 March 1994
J90 Console Install v 1.3 3/14/95
UNICOS 10.0.0.5 Install May 1999

{Note: the CrayDocs and Console Install are seriously incompatible with UNICOS v. 10.}

Support System and IOS-E Installation Guide SG-560A
Cray J90 (unknown version SWS software and IOS software)
UNICOS 10.0.0.2 May 1998

CrayDoc Documentation Library 3.0 (UNICOS 10.0.1.2, SWS 6.2, NQE 3.3,)
UNICOS 10.0.1.2 (May not support J90 "Classic")
SWS 6.2
NQE 3.3.0.15 Modules 2.2.2.3 CAL 10.1.0.6

Software versions

SWS
- Solaris 7 / SunOS 5.7 / November 1998
- Cray console software

IOS
- IOS-V Kernel 3.0.0.5 97/10/16 15:44:46 (installed)

Mainframe
- UNICOS

Installation

“If you need to power-cycle the machine, you must press the CPU reset button first followed by the VME reset button on the control panel. Failure to press the reset buttons in this order will cause the power-up diagnostic tests to fail.”^[3] This is an important note that I missed.

Release contents:

IOS tar file
Install tar file
Generic UNICOS file system
Generic system files
UNICOS binaries

Read in the files from the install CD:

Usage of the /src partition is decreasing; the /opt partition is used to store the installation and IOS-related files
The install script is ./setup and it asks for the four digit serial number. This can be found on a plate on the back of the mainframe cabinet. The EL series serial numbers are 5nnn. Serial numbers 9nnn are J916 backplane; serial numbers 95nn are J932 backplane. "In 1996 350 Cray J90 systems where shipped the large part of the total of 415 J90 systems. Some J90 systems are being converted to SV1 chassis just to keep the records complicated."^[5] Serial numbers 3nnn are SV-1.^[5]
There is a crayadm account and an ios group account
“Loads the opt. tar file from the CD into /opt/install, /opt/local, and /opt/packages”
“Establishes the J90 Console script (jcon) script for the master lOS”
“Sets up the BOOTPD daemon”
“Updates the following Solaris network files in /etc: inetd.conf, services, hostname.le1, netmasks, hosts, nsswitch.conf”
Reboot
Log in with the crayadm account using the password of initial0.

Cray Load Optional Async Product Relocatables. Versions of UNICOS 9.0 and later automatically load this optional software.

User Exits
Tape Daemon
Ultra
Kerberos / Enigma
Secure - Id
NQS
Accounting user - exits

Use fold -80 logfile | more to view /opt/install/log/xxxx, where xxxx is the serial number. Otherwise, vi and other editors will truncate the long lines of text making it unreadable.

Right mouse click on the OpenWindows root X window will show menu options for J90 Console and J90 Install Menu.

“If you are performing an initial install starting from CD-ROM, after running the Load Binaries procedure, you must quit the J90 Install Utility and restart it before continuing the installation. This avoids an lOS reset problem between the CD-ROM version of Load Binaries and the J90 UNICOS 9.0.2 version.”^[3] Another important note that I missed.

Configuration files containing the ASICs chip information.

/sys/pm0.cfg	# Processor Module configuration
/sys/mem0.cfg	# Memory Module Configuration

The UNICOS root password is initial. Run mkfs /core and mkdump.

After installation there are two disk partitions roota/usra/srca and rootb/usrb/srcb for both a live boot and an alternate root used for upgrade. We need to install double the original disk space to accommodate the archive of the original disk arrays and a fresh install.


Recommended minimum partition sizes
Partition	4k blocks	MB
root	110,000	440
usr	190,000	760
src	120,000	480
opt	150,000	600
total	570,000	2,280

Use CONTROL-A to toggle between the IOS-V and UNICOS consoles.

Administration

“Device recommendations: To avoid contention, you should configure the /usr file system on a different controller, disk, and lOS than the one on which the root (/) file system resides.”^[1]

“On baseline systems however, only swap is recommended as a striped disk. Striping is best used only for large I/O moves, such as swapping.”^[1]

“Device recommendations: If two or more lOSs are present, to avoid contention, you should configure /tmp and /home on a different controller, disk, and lOS than the one on which the frequently accessed system file systems and logical devices reside. This file system is best handled by allocating slices from several different disks to compose the logical file system. This disk allocation strategy is called banding.”^[1]

Banding is striping a bunch of disks to create a logical disk. Unlike striping, the banded disks can vary in size. Striping requires disks that are closely identical in raw capacity. I’ve seen no indication that the cray can do other levels of RAID.

Banding partitions / file systems:

/usr/src
/tmp

Startup

Note: turn on the battery backup on the CCU before starting.

“

Power Up CRAY J916 System

Reconnect the mainframe cabinet AC power plug to its source.
Using the right mouse button, click on any open working space. The Workspace menu will appear.
Select the J90 Console menu item.
Move the circuit breaker on the back of the mainframe cabinet to the ON position first, and then move the circuit breaker on the back of the I/O cabinet to the ON position.
Press the Alarm Acknowledge button on the CCU.
Press the CPU RESET button on the CCU.
Press the VME RESET button on the CCU.
Observe any errors on the console screen.
Ensure that the DC enable indicators for the memory and processor modules are green.
Verify that the SYSTEM READY light on the control panel illuminates.
Close the rear door of the cabinet by swinging the door shut and turning the two door-locking fasteners.
Replace the backplane cover and the cover below it and tighten the retaining screws.
Install and close the front door of the cabinet by reconnecting the ground wire and swinging the door shut, ensuring that the door latches are connected.

”

— CRAY J916 2 X 2 to 4 X 4 Backplane Upgrade Procedure, June 1995.^[6]

References

↑ ^1.0 ^1.1 ^1.2 ^1.3 ^1.4 ^1.5 UNICOS Basic Administration Guide for CRAY J90 and CRAY EL Series. Mendota Heights, MN: Cray Research, Inc.. February 1995 [1994]. SG-2416 8.0.3.2. https://bitsavers.org/pdf/cray/J90/SG-2416_UNICOS_Basic_Administration_Guide_for_CRAY_J90_and_CRAY_EL_Series_8.0.3.2_Feb95.pdf. Retrieved 24 March 2025.
↑ Cray J90 I/O Cabinet Hardware Reference Book. Chippewa Falls, WI: Cray Research, Inc.. November 1995. HMQ-261-0. https://cray.modularcircuits.com/cray_docs/hw/j90/HMQ-261-0-CRAY_J90_Series_IO_Cabinet_Hardware_Reference_Booklet-November_1995.pdf. Retrieved 9 June 2025.
↑ ^3.0 ^3.1 ^3.2 ^3.3 UNICOS Installation Guide for Cray J90 Series. Mendota Heights, MN: Cray Research, Inc.. April 1996 [1995]. SG-5271 9.0.2. http://bitsavers.org/pdf/cray/J90/SG-5271_UNICOS_Installation_Guide_for_CRAY_J90_Series_9.0.2_Apr96.pdf. Retrieved 24 May 2025.
↑ ^4.0 ^4.1 CRAY J98 and CRAY J916 Systems Hardware Overview. Cray Research / Silicon Graphics. April 1998 [1995]. HMM-094-B. https://cray.modularcircuits.com/cray_docs/hw/j90/HMM-094-A-Hardware_Overview_for_CRAY_J916_System-April_1998.pdf. Retrieved 24 May 2025.
↑ ^5.0 ^5.1 "Cray Research and Cray computers FAQ Part 3". Cray Supercomputer FAQ and other documents. December 2003. Retrieved 28 May 2025.
↑ CRAY J916 2 X 2 to 4 X 4 Backplane Upgrade Procedure. Chippewa Falls, WI: Cray Research, Inc.. June 1995. HMU-200-0. https://cray.modularcircuits.com/cray_docs/hw/j90/HMU-200-0-CRAY_J916_2X2_to_4X4_Backplane_Upgrade_Procedure-June_1995.pdf. Retrieved 13 June 2025.