Technical Support Information
Last update: 26 September 2006

Using Gaussian 03 with Linda

This document describes the process of installing the Linda software you have purchased through Gaussian, Inc. and   building a distributed-memory parallel version of Gaussian. It assumes that you have already built and tested the regular version of the program. It also assumes that you have read the normal installation instructions and also that you have access to the Gaussian 03 User's Reference .

Linda Parallel Methods

HF, CIS=Direct, and DFT calculations on molecules are Linda parallel, including energies, optimizations and frequencies. TDDFT energies and MP2 energies and gradients are also Linda parallel. PBC calculations are not Linda parallel.

The default for molecules larger than 65 atoms is to use the linear scaling algorithms (FMM), which is not Linda parallel. This value should be increased to about 300 for jobs using Linda (e.g., on a cluster) to obtain the full benefit of parallelization. This is accomplished via the Int=FMMNAtoms= n keyword, where n is the number of atoms. This may be set in the route section or in the Default.Route file.

Installing the Linda Software and Compiling G03/Linda

If you have purchased Gaussian binaries, Linda is distributed on the same CD as the Gaussian binaries and no additional installation is necessary. Follow the directions in the file README file on the distribution CD.

If you have purchased Gaussian source code, then Linda is distributed on a separate CD. Follow the directions in the file README.source file on the Gaussian source distribution CD to install Linda and compile Gaussian to use Linda.

In either case, you must run the command bsd/install as detailed in the README files.

Running Gaussian with Linda

The Linda parallel programming model involves a master process, which runs on the current processor, and a number of worker processes which can run on other nodes of the network.   So a Gaussian 03 /Linda run must specify the number of processors to use, the list of processors where the jobs should be run, and occasionally other job parameters. An environment variable is generally the easiest way to specify this information (as we will see).

Each of these nodes needs to have some access to the Gaussian 03 directory tree. The recommended configuration is to have G03 on each system that will be used for the parallel job. Note that the Linda binaries need to have the same path on each machine. If this is not feasible, the G03 tree can be made accessible via an NFS-mounted disk which is mounted in an identical location on all nodes.  

For MP2 calculations, each node must also have some local disk where Gaussian 03 can put temporary files. This is defined as usual via the GAUSS_SCRDIR environment variable, which should be set in the .cshrc or .profile for your account on each node.

Configuring Gaussian 03 for Linda

Gaussian 03 gets configuration information from three primary sources

  • The Gaussian input file via %Link0 commands.
  • The Default.Route file.
  • The environment variable GAUSS_LFLAGS .

Details about %Link0 commands and the Default.Route file can also be found in the Gaussian 03 User's Reference manual. Entries specific to Gaussian 03 /Linda are described below.

Specifying the Number of Processors

The easiest way to specify the number of processors is to add the %NProcLinda Link 0 command at the top of your input file, e.g. to use 4 Linda processes add

%NProcLinda=4

before the route section. This will cause the job to be executed on the master and three workers.

You may also specify a default number of Linda workers in the Default.Route file using -L- . For example, the following line sets a default of four Linda workers:

-L- 4

The %NProcLinda command in the input file will override the -L- value in the Default.Route file.

Except under Mac OS X, distributed memory parallelization can be combined with shared memory parallelization for systems with multiprocessor nodes. For example, in order to run a job on 8 CPUs located on four networked dual-processor computers, use the following Link 0 settings:

%NProcShared=2                             Not valid for Mac OS X
%NProcLinda=4

See below for a method using multiprocessor nodes in a Mac OS X Linda calculation.

Specifying the Amount of Memory For a Parallel Calculation

Memory is specified using the %Mem Link0 command, just as for serial calculations.   See Chapter 4 of the Gaussian 03 User's Reference for important information about the memory requirements for parallel jobs.

Defining the Node List

The environment variable GAUSS_LFLAGS can be used to define the list of nodes on which to run. To define a list of nodes when using the C shell, execute a command like this one before running g03l :

% setenv GAUSS_LFLAGS '-nodelist "hamlet ophelia laertes lear duncan"'

or, for the Bourne/BASH shell:

$ GAUSS_LFLAGS='-nodelist "hamlet ophelia laertes lear duncan"'
$ export GAUSS_LFLAGS

Pay attention to the proper use of quotation marks illustrated in these examples.

Users should note that some systems (e.g., the IBM SP3), define separate hostnames for each CPU corresponding to different IP interconnection interfaces (such as Ethernet, fibre, high performance switch, etc). Use only one of   the hostnames for each CPU, and, where possible, use the hostname corresponding to the fastest IP-compatible interface.

Specifying Other Linda Options

A few other options that are sometime useful are:

-v                                    Display verbose messages
-vv                                  Display very verbose messages

For example, one could turn on very verbose Linda output using:

% setenv GAUSS_LFLAGS -vv

To combine the option with a node list, use a command like this one:

% setenv GAUSS_LFLAGS '-vv -nodelist "hamlet ophelia laertes lear duncan"'

Specifying Workers Per Node under Mac OS X

The -mp n option can be used to run Gaussian with Linda across multiple Mac OS X multiprocessor systems. It specifies the maximum number of Linda processes to be scheduled per node. Set it to 2 when the individual nodes are dual processors.

There are many other Linda options but most of them are not used by Gaussian.   Check the Linda manual on the Internet at www.lindaspaces.com/downloads/lindamanual.pdf . Note that Gaussian 03 /Linda does not use the native Linda resources minworker and maxworker.

Using ssh Instead of rsh

By default, Linda uses rsh to communicate between nodes. To override this default, you need to create a configuration file in your home directory on the master node named .tsnet.config which contains the following line:

Tsnet.Node.lindarsharg: ssh

This will cause ssh to be used instead. Note that passwordless ssh logins must already be configured from the master to all worker nodes.

Starting Gaussian 03 Jobs

The g03l (lowercase L) command is used to initiate a distributed memory parallel Gaussian 03 job (instead of g03 ). This command assumes the following conditions are true:

  • You have already executed the appropriate Gaussian 03 initialization file ($g03root/g03/bsd/g03.login or $g03root/g03/bsd/g03.profile ). Test this by running a serial Gaussian 03 calculation on the master node.

  • The directory $g03root/g03 is accessible on all nodes.  

  • The LD_LIBRARY_PATH variable is set (see the G03 install notes) to locate the Linda shared libraries.

  • Local scratch space is available on each node if needed (via GAUSS_SCRDIR ).

  • All nodes on which the program will run are trusted by the current host. You should be able to login   remotely with the rlogin or ssh command without having to give a password to each of them. Contact your system administrator about configuring security for the nodes in the network.

Calculations can then be started as you would for a serial calculations but using g03l :

% g03l input &

and Gaussian 03 will start the master and worker processes as needed.

Monitoring the Calculation

You will see processes started on the worker nodes for those links which have been parallelized, e.g. they will have a *.exel entry in the main G03 directory. Using the top or other commands on worker nodes will let you see lxxx.exel when it starts.                                    

The relevant measure of performance for a parallel calculation is the elapsed or wall clock time.   The easiest way to check this is to use an external monitor like time , times , or timex , e.g.

% times g03l input &

which will report elapsed, CPU and system times.   Note that the last two are only on the master node and similar amounts of CPU and system   time are expended on each node.   So the speedup is the ratio of the elapsed time running serial g03 and the elapsed time for g03l .

Pitfalls to Avoid with Linda Jobs

  • If there are more nodes listed than are specified with %NProcLinda , then a process may still be started on each listed node, but only the specified number of them will actually do any work. A corollary of this is that if you forget to include %NProcLinda within the Gaussian 03 input file or -L- in the Default.Route file, then the job will not run in parallel, although there may be an idle Linda process on each of the nodes in the node list.

  • Running g03 instead of g03l when you have specified a parallel job will produce an error message.

  • Running Linda calculations for methods which are not Linda parallelized.

  • Running Linda calculations on large molecules without increasing the value of Int=FMMNAtoms.