The orange cluster is intended for high performance computing applications and in particular for large memory and tightly coupled parallel processes.  The hardware for the 90 compute nodes are described here.

This document supplements the LSF Basics and Running MPI programs, also on this website.  You'll want to consult those documents and the official SCCS LSF and MPI documents.

To use the orange cluster, you will need to be a member of the LSF group "kipac-ibgrp".  Contact Stuart Marshall to access to the cluster batch system.

If your login shell is tcsh or csh you will want to put a section like this in your .cshrc file:

if ( $HOSTNAME =~ "orange*" ) then
   #- intel compilers
   source /afs/slac.stanford.edu/package/intel_tools/compiler9.0/@sys/cc/bin/iccvars.csh
   source /afs/slac.stanford.edu/package/intel_tools/compiler9.0/@sys/fc/bin/ifortvars.csh
   #- package/OpenMPI
   set path = (/afs/slac/package/OpenMPI/bin $path )
   if !($?LD_LIBRARY_PATH) then
      setenv LD_LIBRARY_PATH /afs/slac/package/OpenMPI/lib
   else
      setenv LD_LIBRARY_PATH /afs/slac/package/OpenMPI/lib:${LD_LIBRARY_PATH}
   endif
endif

The important points of this set up are:

• To use the intel compiler suite, source the files above in /afs/slac.stanford.edu/package/intel_tools/compiler9.0/ to set up your environment.
• Put  /afs/slac/package/OpenMPI/bin in your path.
• Put /afs/slac/package/OpenMPI/lib in your LD_LIBRARY_PATH

The head node, orange, is accessible to anyone with a SLAC account for developing, compiling and testing codes and for LSF job submission.  The compute nodes, orange001..090 are only for batch jobs submitted via LSF.  To submit a job, you need to use the command:

bsub -q kipac-ibq -a openmpi {other bsub args} <command line for your job>

It is a good idea to verify that your path and environment work on the compute nodes (where you can't log in).

You can verify like this:

bsub -n 1 -q kipac-ibq -a openmpi /usr/bin/which mpirun                 #- verify OpenMPI/bin in your path
bsub -n 1 -q kipac-ibq -a openmpi /usr/bin/ldd <path to your program>   #- verify all shared libs are found

Until the cluster parallel file system is ready for regular use, we only have NFS storage available.  The are /nfs/slac/g/ki/ki01 is probably the most suitable place to output large collections of simulation results.  As usual contact Stuart Marshall regarding space on ki01.

Finally, a note about logging into orange.  This host is running the newer version of sshd which does not forward the user's AFS "ticket" for file system access.  The appropriate way to use the newer ssh client is: (1) type "kinit -f" on your local computer and follow the prompts, and then (2) type "/usr/local/bin/sshnew/ssh orange".