SQS - Simple Queueing System ---------------------------- Version 2.8 June, 2008 ---------------------- This file only describes the installation if you want to install and use SQS on a single machine. Read INSTALL if you have a cluster or want to inspect queues on remote machines. INSTALLATION ------------ 1. Download the file sqs-2.8.tar.gz and unzip and tar it to give the directory sqs-{version} where {version} is the version number, e.g. 2.8. 2. You can install sqs as root or locally. If you install locally, other users can still use sqs. If you do not want that, install locally and then alter the permissions on the scripts in $bindir and on the directory $sqsdir. 3. Edit sqs.conf. There are two ways to do this. The first. of course, is the direct use of an editor. The second uses a script called "config-sqs". The latter method is particularly usefull for making small changes, once you are familiar with the system. With both methods, the first time is the most difficult. The "config-sqs" script generates a sqs.conf file with none of the comments in the original. It is therefore a good idea to save a copy of the original sws.conf as, say "sqs.conf.orig", but not "sqs.conf.save" as that file is created by "config-sqs". This is done in the release. If you are going to use "config-sqs", you must have the Term::ReadKey module, and you should check that the first line correctly identifies where your perl is. Edit sqs.conf or run config-sqs. You must change each variable or confirm they are already correct:- '$noofqueues' Set to the number of queues. Just ignore the section on the $clustercommand command. For each queue, you have to define the appropriate element of the following five arrays:- '@queue' The queue name. '@maxqu' For each queue, set to the maximum number of jobs that can run at any one time. '@maxperuser' For each queue, set to the maximum number of jobs any one user can run at once in the given queue. This is useful on dual processor machines where setting $maxqu = 2 and $maxperuser = 1 allows each user to run jobs that depend on the output from the previous job. '@quhost' Set to the hosts, as returned by `hostname -s`, for the machines to which the queues are linked. '@web' Set to '1' if queue is a web use queue, and to '0' otherwise. Ignore the section on remote machines use, or if running "config-sqs" reply "n" to the question whether you are going to run on remote machines. The nest section defines $wait, $psexe and $psflag. You probably do not need to change these, but you should check the value given for $psflag. On linux $psexe is always 'ps'. On Cygwin, it should be changed to 'procps' and $psflag should be set to "-f -U". Ignore the section on using SQS for submitting jobs from web scripts, or reply "n" to the question in "config-sqs".. You then have to define the following directories:- '$sqsdir'. This points to where the 'sqs.users', 'sqs.id', pid files queue files and 'sqs.log' files will go. This directory will be created if it does not exist. Check sqs.conf using the instructions there. It has to be world executable. For a local install, edit sqs.conf as directed in that file. $HOME/.sqs is a good choice for $sqsdir, but note you must expand $HOME so it is an absolute path, otherwise other users look in their own home directories, not yours. '$bindir'. This points to where 'install.pl' will put the scripts. For a root install, this should probably be /usr/local/bin, but it can be different. The same applies as for $sqsdir. This is set in sqs.conf. Read that file carefully as for $sqsdir. Note you must use a full path for $bindir, not $HOME, to allow other users to use SQS. $bindir is created if it does not exist. Note that '$bindir' must be on the path of all users of sqs. '$mandir'. This points to where the man pages will be installed, typically /usr/local/man for a root install and $HOME/man for a local install. The man pages are put into $mandir/man1. Again you must expand out $HOME in sqs.conf. If this is a local install, each user must add $mandir to their $MANPATH. If you define $mandir = '' the man pages will never be installed. Do not edit sqs.inc. 4. Edit 'install.pl' so that the first line is correct for where Perl is on your system. 5. Run:- ./install.pl For a root install, you must be root to do this. This will edit the scripts and install them in $bindir. It then installs 'sqs.id', 'sqs.log', the queue files, README files and the man pages. 6. A call:- ./install.pl 'script name' will install just a single script. Calls:- ./install.pl perl ./install.pl README ./install.pl man installs just the perl scripts (useful if you want to just update the scripts and leave the queues intact, although take care - an update may have a different format for the queue files if a new feature has been added), 'sqs.conf', the README file and all the man pages respectively. A call ./install.pl queue {queue-name} will install the queue file for queue {queue-name}. This is necessary if you alter sqs.conf to add new queues and install sqs.conf as above with './install.pl sqs.conf' 7. You can call install.pl as './install.pl -u' to uninstall everything. 8. Web pages giving some information on SQS and web access to the README files, this INSTALL file, INSTALL.single and the ChangeLog file are available in the htdocs directory and are installed on http://sqs.sourceforge.com. The three text files are updated in the htdocs directory when you run a full install. 9. You may now want to write some scripts that simplify calling 'qsub' for specific types of jobs. These are generally bash scripts that are much more complex than the simple Perl scripts 'qexample' and 'qexample.simple' discussed in the README file. Our script for running the computational chemistry code, Gaussian98, is called 'sub_g98' and is:- #!/bin/bash cat > $1.job << END cd $PWD rm -f core g98 $1 END chmod 0775 $1.job qsub -d -q molecule $PWD/$1.job > /dev/null If you are running Gaussian98 on a cluster machine with SQS, you have to add a few lines, that are normally in the .bash_profile file. These ensure that the Gaussian variables are set. This version also has the queue as the 2nd parameter, and uses the queue name to define the scratch directory. We have modified g98.profile, so that it does not add the Gaussian directories to the path if they are already there. This prevents the path from steadily getting longer. #!/bin/bash cat > $1.job << END cd $PWD export g98root="/home/qchem" . \$g98root/g98/bsd/g98.profile export GAUSS_SCRDIR="/scr/$2" rm -f core g98 $1 END chmod 0755 $1.job qsub -d -q $2 $PWD/$1.job > /dev/null Our script for running the computational chemistry code, Gamess, is called 'sub_gam' and is:- #!/bin/bash cat > $1.job << END cd $PWD rm -f $1.dat rm -f $1.irc rungms $1 > $1.log END chmod 0775 $1.job qsub -d -q molecule $PWD/$1.job > /dev/null These scripts are available in the scripts sub-directory. There are other scripts including 'sub_g98_full' which is the one I am now using. This allows you to add '-h' and/or '-p priority' as arguments of sub_g98 after the job and queue. Others relate to remote and cluster jobs. ----------------------------------------------------------------------------- Brian Salter-Duke b_duke@bigpond.net.au June, 2008.