For a long time, people in MRSEC used public desktops machines like joule, fresnel, alder, and even some more ancient and forgotten machines like romulan, for long-term computational jobs and simulations. Then we setup a small server called pondermatic (which had initially been setup earlier on as an actual cluster) for that purpose, but it was inadequate.
That era is drawing to a close. MRSEC has purchased a new computational system called superabacus (named by Jin Wang in 2011), which will replace joule as JFI/MRSEC's server to login, get a UNIX shell environment, handle mail and home directory files, and most importantly, to do computational/simulation work. However, there are some things to know about superabacus before you get started.
Anyone with a JFI account. If you don't have a JFI account, please get one -- see here for information on how to get one and what benefits it offers. Please note that this is not the same thing as the CNet account that NSIT gave you when you came to the University.
The short answer is, whatever you used to do on joule, except better. Superabacus offers access to your JFI home directory, JFI email (pine, mutt, procmail, etc.), automated twice daily backups from our backup server "penrose", and a full UNIX environment with compilers and scripting tools. In some ways, this is like the machine that NSIT used to offer the campus that was known as harper.
However, unlike joule and harper, superabacus is a cluster, which means that when you run computational jobs, rather than just run them on the machine itself, you submit your jobs through a queueing system that hands it off to another machine in the background, or a whole farm of machines. The machine you login on is actually not the work node for most types of computations (there are exceptions though, see below). The queueing system that does this is called SGE, and it has its own commands that you will need to learn if you're not already familiar with it.
Much like on joule and fresnel, you can also access your daily backups from superabacus via the NFS mounts at /nfs/penrose. See the discussion of backups in JFI for more information.
Superabacus has a login node (or "head node") that faces the network, which is where you will be when you login. This machine is brand new, but it only has 8 processor cores, so we try to keep computation on that system to a minimum. Behind it, there is a work node with 48 processor cores and 64GB of memory, which should only be used by submitting jobs to it through the queueing system, which is called SGE. If I deem that computation on the login machine is getting excessive or is slowing things down for other users, I WILL KILL JOBS/SIMULATIONS until usability is restored. Please always use the queueing system if you can.
To submit a job, use the qsub command, which has a long manual that's worth reading over briefly (type "man qsub" to see it). Please note! SGE jobs are never binary executables (like C or Fortran programs). They are always shell scripts! That means that if you've coded a simulation program that runs as a C or Fortran program, you will still need to create a script that runs that program from the script. SGE jobs are never executables!
The reason is because SGE jobs are controlled by "active comments" in your scripts, which are commented out lines that contain settings for SGE. They are interpreted as code by SGE even though they are commented out. Since these settings are needed, your jobs must always be in the form of a shell script.
There are many guides on the Internet that discuss how you setup your script environment for an SGE job, but one of the best I've seen is this one from UCLA, which is a pretty good starter guide. Note that SGE has many more commands than these, and they all have man pages for full information.
Also, if you need help, please let us know. We administer SGE clusters for three different research groups here at the JFI, so this is something we deal with a lot.
Please be aware that since superabacus currently is a "cluster" consisting of one node with 48 processor cores, that all of your job slots will show up in qstat under that one node. You will not see a listing of 48 actual nodes.
New Jersey Institute of Technology
University of California at San Diego
The Message Of The Day ("MOTD") bulletin that you see when you login may mention locally installed or commercially bought applications that are available on superabacus. Most of these are simple to launch and run, but here are some advisories that may apply where needed:
The Intel compilers need to have a script run in your environment to work correctly. If you're a bash user, you can put this into your ~/.bashrc file (creating it if you don't have one):
# For Intel Fortran/C/C++ Compiler Support:
source /opt/intel/composerxe-2011.0.084/bin/compilervars.sh intel64
If you're a tcsh user, use this in your ~/.cshrc instead (creating the file if you don't have one). Note the conditional test below is needed by tcsh to make this run only if you're on an interactive session. If you already have a test block like this in your .cshrc file, just put the source line inside it:
# For Intel Fortran/C/C++ Compiler Support:
if ($?prompt) then
source /opt/intel/composerxe-2011.0.084/bin/compilervars.sh intel64
endif
Some of the programs that we may have installed on superabacus, such as Matlab and Mathematica, may use a graphical user interface (rather than command line shell), and require you to connect to the system with a command like 'ssh -Y superabacus' to allow graphics windows from superabacus to be opened on your desktop. These types of programs can also be troublesome to run on the worker machine(s) behind the login node, especially if your program relies on this type of graphics intrinsically. I will therefore allow a certain amount of computation on the login node for programs like this, under Matlab, Mathematica, etc. I will also allow light computation of other kinds as long as it does not use up all 8 of the login node's processor cores or slow down the system for other users. Please use the SGE queueing system if you can.
Currently, we have only one 48-core system behind the login/head node for actual work. However it is possible for us to add more, and they do not all necessarily even have to be the same type of machine. It is likely that at some point we will put whatever spare computational resources we have available behind the front machine and make them also available as worker nodes. SGE's commands allow users to specify which types of machines they would like their jobs to run on, for case where users may have a preference about this.
As I'm sure you've noticed, GCIS is getting extremely crowded. As the old RI building comes down, more and more people are needing office space. Some are going to Searle. Others are moving in with people from other research groups besides their own, in cases where such space has been found.
With these office space shortages, we can no longer afford space for public walkup computer rooms such as the old GCIS E124 "fishbowl" area, which has now already been converted to secretarial office space. While such walkups have been helpful, they are increasingly an anachronism in a time when virtually everyone on campus has a laptop, and computational facilities for simulations really ought to come in the form of a robust server/cluster that is designed for such, and not ordinary desktop computers like joule.
Therefore, it seems likely that joule and fresnel will soon be decommissioned (at time of this writing), and alder already has been. If you have been using joule as a shell server, logging into it remotely to get various things done after the fashion that people used to use harper, please be encouraged now to use superabacus for that role. If you would like to have a Linux (or other UNIX) desktop for use in your area, managed by JFI Computing, please let us know, and also consult with us if you're considering buying a new machine intended to run Linux or some other UNIX flavor for compatibility advice.
In short, public walkups like joule are probably going away soon. We just don't have the physical space for them anymore.