home

Archive for the '(pc-)gamess-us' Category

Compiling And Running GAMESS-US (1 May 2013(R1)) On 64-bit Ubuntu 12.X/13.X In SMP Mode

Saturday, April 5th, 2014

Author’s Note 1: It is my standard policy to put too much info into guides so that those who are searching for specific problems they come across will find the offending text in their searches. With luck, your “build error” search sent you here.

Author’s Note 2: It’s not as bad as it looks (I’ve included lots of output and error messages for easy searching)!

Author’s Note 3: I won’t be much help for you in diagnosing your errors, but am happy to tweak the text below if something is unclear.

Conventions: I include both the commands you type in your Terminal and some of the output from these commands, the output being where most of the errors appear that I work on in the discussion.

Input is formatted as below:

username – your username (check your prompt)
machinename – your hostname (type hostname or check your prompt)

Text you put in at the (also shown, so you see the directory structure) prompt (copy + paste should be fine)

Text you get out (for checking results and reproducing errors)

Having just recently downloaded the newest version of GAMESS-US (R1 2013), my first few passes at using it under Linux (specifically, Ubuntu 12.04) ran into a few walls that required some straightforward modifications and a little bit of system prep planning. As my first few passes before successful execution are likely the same exact problems you might have run into in your attempts to get GAMESS-US to run (after a successful compilation and linking), I’m posting my problems and solutions here.

Qualifier 1 – My concern at the moment has been to get GAMESS-US to run under 64-bit Ubuntu 12.04 on a multi-core board (ye olde symmetric multiprocessing (which I always called single multi-processor, or SMP)). While some answers may follow in what’s below, this post doesn’t cover MPI-specific builds (nothing through a router, that is). SMP is the only concern (which is to say, I likely won’t have good answers if you send along an MPI-specific question). Also, although I’m VERY interested in trying it, I’ve not yet attempted to build a GPU-capable version (but plan to in the near future).

Qualifier 2 – It is my standard policy to install apps into /opt, and my steps below will reflect that (specifically because there’s a permission issue that needs to be addressed when you first try to build components). You can default to whatever you like, but keep in mind my tweaks when you try to build your local copy.

So, with the qualifiers in mind…

1. Prepping The System (apt-get)

There are few things better than being able to apt-get everything you need to prep your machine for an install, and I’m pleased to report that the (current) process for putting the important files onto Ubuntu 12.X/13.X is easy. Assuming you’re not going the Intel / PGI / MKL route, you can do everything by installing gfortran (compiler, presently installing 4.4) and the blas and atlas math libraries.

username@machinename:~$ sudo apt-get install gfortran libblas-dev libatlas-base-dev

Note: your atlas libraries will be installed in /usr/lib64/atlas/ – this will matter when you run config.

After these finish, run the following to determine your installed gfortran version (will be asked for by the new GAMESS config)

username@machinename:~$ gfortran -dumpversion

GNU Fortran (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3 Copyright (C) 2010 Free Software Foundation, Inc. GNU Fortran comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU Fortran under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING

4.4 And you’re ready for GAMESS.

2. Downloading GAMESS-US, Placing Into /opt, And Changing Permissions

First, obviously, get the GAMESS source (click on the red text).

After downloading, copy/move gamess-current.tar.gz into /opt

username@machinename:~$ cd ~/Downloads
username@machinename:~/Downloads$ sudo cp gamess-current.tar.gz /opt
username@machinename:~/Downloads$ cd /opt
username@machinename:/opt$ sudo gunzip gamess-cuerent.tar.gz
username@machinename:/opt$ sudo tar xvd gamess-current.tar

gamess/ gamess/gms-files.csh gamess/tools/ ... gamess/misc/count.code gamess/misc/vbdum.src gamess/Makefile.in

At this point, if you go through the config process and get to the point of building ddikick.x, you will get an error when you first try to run ./compddi

username@machinename:/opt/gamess/ddi$ sudo ./compddi >& compddi.log &
[1] 4622 -bash: compddi.log: Permission denied

The problem is with the permission of the entire gamess folder:

drwxr-xr-x  4 root        root              4096 2014-04-04 21:43 . drwxr-xr-x 22 root        root              4096 2013-12-27 16:17 .. drwxr-xr-x 14 1300 504              4096 2014-04-04 21:43 gamess -rw-r--r-- 1 root        root         198481920 2014-04-04 21:42 gamess-current.tar

Which you remedy before running into this error by changing the permissions:

username@machinename:/opt$ sudo chown -R username gamess

The next step is recommended when you run config, so I’m performing the step here to get it out of the way. With the atlas libraries installed, generate two symbolic links.

username@machinename:/opt$ cd /usr/lib64/atlas
username@machinename:/usr/lib64/atlas$ sudo ln -s libf77blas.so.3.0 libf77blas.so
username@machinename:/usr/lib64/atlas$ sudo ln -s libatlas.so.3.0 libatlas.so

And, at this point, you’re ready to run the new (well, new to me) config script that preps your system install.

3. Building GAMESS-US

Back to the GAMESS-US folder.

username@machinename:/usr/lib64/atlas$ cd /opt/gamess
username@machinename:/opt/gamess$ sudo ./config
This script asks a few questions, depending on your computer system, to set up compiler names, libraries, message passing libraries, and so forth. You can quit at any time by pressing control-C, and then . Please open a second window by logging into your target machine, in case this script asks you to 'type' a command to learn something about your system software situation. All such extra questions will use the word 'type' to indicate it is a command for the other window. After the new window is open, please hit to go on.

You can open that second window or blindly assume that what I include below is all you need.

[enter]

GAMESS can compile on the following 32 bit or 64 bit machines: axp64 - Alpha chip, native compiler, running Tru64 or Linux cray-xt - Cray's massively parallel system, running CNL hpux32 - HP PA-RISC chips (old models only), running HP-UX hpux64 - HP Intel or PA-RISC chips, running HP-UX ibm32 - IBM (old models only), running AIX ibm64 - IBM, Power3 chip or newer, running AIX or Linux ibm64-sp - IBM SP parallel system, running AIX ibm-bg - IBM Blue Gene (P or L model), these are 32 bit systems linux32 - Linux (any 32 bit distribution), for x86 (old systems only) linux64 - Linux (any 64 bit distribution), for x86_64 or ia64 chips AMD/Intel chip Linux machines are sold by many companies mac32 - Apple Mac, any chip, running OS X 10.4 or older mac64 - Apple Mac, any chip, running OS X 10.5 or newer sgi32 - Silicon Graphics Inc., MIPS chip only, running Irix sgi64 - Silicon Graphics Inc., MIPS chip only, running Irix sun32 - Sun ultraSPARC chips (old models only), running Solaris sun64 - Sun ultraSPARC or Opteron chips, running Solaris win32 - Windows 32-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition) win64 - Windows 64-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition) winazure - Windows Azure Cloud Platform running Windows 64-bit type 'uname -a' to partially clarify your computer's flavor. please enter your target machine name:

We’re doing a linux64 build, so type the following at the prompt:

linux64
Where is the GAMESS software on your system? A typical response might be /u1/mike/gamess, most probably the correct answer is /opt/gamess GAMESS directory? [/opt/gamess]

Who is this mike and where is my folder u1? We’ll get to that in rungms. For now, I’m installing in /opt, so the default directory is fine:

[enter]

Setting up GAMESS compile and link for GMS_TARGET=linux64 GAMESS software is located at GMS_PATH=/opt/gamess Please provide the name of the build locaation. This may be the same location as the GAMESS directory. GAMESS build directory? [/opt/gamess]

Fine as selected.

[enter]

Please provide a version number for the GAMESS executable. This will be used as the middle part of the binary's name, for example: gamess.00.x Version? [00]

Is this important? Maybe, if you plan on building multiple versions of GAMESS-US (you might want a GPU-friendly version, one with a different compiler, one with MPI, etc.). Number as you wish and remember the number when it comes to rungms. That said, the actual linking step seems to really want to produce a 01 version (we’ll get to that). Meantime, default value is fine.

[enter]

Linux offers many choices for FORTRAN compilers, including the GNU compiler set ('g77' in old versions of Linux, or 'gfortran' in current versions), which are included for free in Unix distributions. There are also commercial compilers, namely Intel's 'ifort', Portland Group's 'pgfortran', and Pathscale's 'pathf90'. The last two are not common, and aren't as well tested as the others. type 'rpm -aq | grep gcc' to check on all GNU compilers, including gcc type 'which gfortran' to look for GNU's gfortran (a very good choice), type 'which g77' to look for GNU's g77, type 'which ifort' to look for Intel's compiler, type 'which pgfortran' to look for Portland Group's compiler, type 'which pathf90' to look for Pathscale's compiler. Please enter your choice of FORTRAN:

We’re using gfortran (currently 4.4.3):

gfortran

gfortran is very robust, so this is a wise choice. Please type 'gfortran -dumpversion' or else 'gfortran -v' to detect the version number of your gfortran. This reply should be a string with at least two decimal points, such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12. The reply may be labeled as a 'gcc' version, but it is really your gfortran version. Please enter only the first decimal place, such as 4.1 or 4.6:
4.4

Alas, your version of gfortran does not support REAL*16, so relativistic integrals cannot use quadruple precision. Other than this, everything will work properly. hit to continue to the math library setup.

If this was my biggest concern I’d be a happy quantum chemist. Obviously you can try to install other flavors of gfortran and, possibly, by the time you need the procedure I’m following, a newer version of gfortran will be apt-gotten.

[enter]

Linux distributions do not include a standard math library. There are several reasonable add-on library choices, MKL from Intel for 32 or 64 bit Linux (very fast) ACML from AMD for 32 or 64 bit Linux (free) ATLAS from www.rpmfind.net for 32 or 64 bit Linux (free) and one very unreasonable option, namely 'none', which will use some slow FORTRAN routines supplied with GAMESS. Choosing 'none' will run MP2 jobs 2x slower, or CCSD(T) jobs 5x slower. Some typical places (but not the only ones) to find math libraries are Type 'ls /opt/intel/mkl' to look for MKL Type 'ls /opt/intel/Compiler/mkl' to look for MKL Type 'ls /opt/intel/composerxe/mkl' to look for MKL Type 'ls -d /opt/acml*' to look for ACML Type 'ls -d /usr/local/acml*' to look for ACML Type 'ls /usr/lib64/atlas' to look for Atlas Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none':
atlas

Where is your Atlas math library installed? A likely place is /usr/lib64/atlas Please enter the Atlas subdirectory on your system:

Our location is, in fact, /usr/lib64/atlas, so we type it in accordingly.

NOTE: If you don’t type anything but [enter] below, the script closes (/usr/lib64/atlas is listed as the expected location, but it is not defaulted by the script. You need to type it in.

/usr/lib64/atlas
 
The linking step in GAMESS assumes that a softlink exists within the system's /usr/lib64/atlas from libatlas.so to a specific file like libatlas.so.3.0 from libf77blas.so to a specific file like libf77blas.so.3.0 config can carry on for the moment, but the 'root' user should chdir /usr/lib64/atlas ln -s libf77blas.so.3.0 libf77blas.so ln -s libatlas.so.3.0 libatlas.so prior to the linking of GAMESS to a binary executable. Math library 'atlas' will be taken from /usr/lib64/atlas please hit to compile the GAMESS source code activator

The symbolic linking was performed before the GAMESS steps.

[enter]

gfortran -o /home/username/gamess/tools/actvte.x actvte.f unset echo Source code activator was successfully compiled. please hit to set up your network for Linux clusters.
[enter]

If you have a slow network, like Gigabit Ethernet (GE), or if you have so few nodes you won't run extensively in parallel, or if you have no MPI library installed, or if you want a fail-safe compile/link and easy execution, choose 'sockets' to use good old reliable standard TCP/IP networking. If you have an expensive but fast network like Infiniband (IB), and if you have an MPI library correctly installed, choose 'mpi'. communication library ('sockets' or 'mpi')?

Again, I’m not building an mpi-friendly version, so am using sockets.

sockets

64 bit Linux builds can attach a special LIBCCHEM code for fast MP2 and CCSD(T) runs. The LIBCCHEM code can utilize nVIDIA GPUs, through the CUDA libraries, if GPUs are available. Usage of LIBCCHEM requires installation of HDF5 I/O software as well. GAMESS+LIBCCHEM binaries are unable to run most of GAMESS computations, and are a bit harder to create due to the additional CUDA/HDF5 software. Therefore, the first time you run 'config', the best answer is 'no'! If you decide to try LIBCCHEM later, just run this 'config' again. Do you want to try LIBCCHEM? (yes/no):
no

Your configuration for GAMESS compilation is now in /home/username/gamess/install.info Now, please follow the directions in /home/username/gamess/machines/readme.unix username@machinename:~/gamess$

At this stage, you’re ready to build ddikick.x and continue with the compiling.

4. Build ddikick.x

username@machinename:/opt/gamess$ cd ddi
username@machinename:/opt/gamess/ddi$ sudo ./compddi >& compddi.log &

Will dump output into compddi.log (which will now work with the correct permissions).

username@machinename:/opt/gamess/ddi$ sudo mv ddikick.x ..
username@machinename:/opt/gamess/ddi$ cd ..
username@machinename:/opt/gamess$ sudo ./compall >& compall.log &

Feel free to follow along as compall.log dumps results. You’re also welcome to follow the readme.unix advice:

This takes a while, so go for coffee, or check the SF Giants web page.

Upon completion, the last step is to link the executable.

Now, it used to be the case that you specified the version number in the lked step. So, if you wanted to stick with the 00 version from the config file, you’d type

username@machinename:/opt/gamess$ sudo ./lked gamess 00 >& lked.log &

When you do that at present, you get

[1] 7626 username@machinename:/opt/gamess$ [1]+ Stopped sudo ./lked gamess 00 &>lked.log

This then leads you to use the lked call from the readme.unix file.

username@machinename:/opt/gamess$ sudo ./lked gamess 01 >& lked.log &

Which then produces lked.log and gamess.01.x.

Now, if you run with 00 again, you get a successful linking of gamess.00.x . Not sure why this happens, but the version number isn’t important so long as you specify the right one when you use rungms (so I’ve not diagnosed it further).

At this point, you have a gamess.00.x and/or gamess.01.x executable in your /opt/gamess folder:

30828747 2014-04-04 22:41 gamess.01.x

I’m going to ignore the 00 issue out of the config file and use the gamess.01.x executable.

We’re ready to run calculations and work through the next set of errors you’ll receive if you don’t properly modify files.

5. PATH Setting

First, we copy rungms to our home folder, then add /opt/gamess to the PATH:

username@machinename:/opt/gamess$ cp rungms ~/
username@machinename:/opt/gamess$ cd ~/
username@machinename:~$ nano .bashrc

Add the following to the bottom of .bashrc (or extend your PATH)

PATH=$PATH:/opt/gamess

Quit nano and source.

username@machinename:~$ source .bashrc
[OPTIONAL] username@machinename:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:.../opt/gamess:

6. rungms (Probably Why You’re Here)

If you just go blindly into a run, you’ll get the following error:

username@machinename:~$ ./rungms test.inp

----- GAMESS execution script 'rungms' ----- This job is running on host machinename under operating system Linux at Fri Apr 4 22:47:55 EDT 2014 Available scratch disk space (Kbyte units) at beginning of the job is df: `/scr/username': No such file or directory df: no file systems processed GAMESS temporary binary files will be written to /scr/username GAMESS supplementary output files will be written to /home/username/scr Copying input file test.inp to your run's scratch directory... cp test.inp /scr/username/test.F05 cp: cannot create regular file `/scr/username/test.F05': No such file or directory unset echo /u1/mike/gamess/gms-files.csh: No such file or directory.

As is obvious, rungms needs some modifying.

username@machinename:~$ nano rungms

Scroll down until you see the following:

set TARGET=sockets set SCR=/scr/$USER set USERSCR=~$USER/scr set GMSPATH=/u1/mike/gamess

Given that it’s just me on the machine, I tend to simplify this by making SCR and USERSCR the same directory, and I make them both /tmp. If you intend on keeping all of the files, you’ll need to make rungms specific for each run case. My only concerns are .dat and .log, so /tmp dumping is fine. Furthermore, we must change GMSPATH from how the ever-helpful Mike Schmidt (he got me through some early issues when I started my GAMESS-US adventure 15ish years ago. Won’t complain about his continued default-ed presence in the scripts) has it set up at Iowa to how we want it on our own machines (in my case, /opt/gamess)

set TARGET=sockets set SCR=/tmp set USERSCR=/tmp set GMSPATH=/opt/gamess

With these modifications, your next run will be a bit more successful:

username@machinename:~$ ./rungms test.inp

----- GAMESS execution script 'rungms' ----- This job is running on host machinename under operating system Linux at Fri Apr 4 22:51:35 EDT 2014 Available scratch disk space (Kbyte units) at beginning of the job is Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 1905222596 249225412 1559217460 14% / GAMESS temporary binary files will be written to /tmp GAMESS supplementary output files will be written to /tmp Copying input file test.inp to your run's scratch directory... cp test.inp /tmp/test.F05 unset echo /opt/gamess/ddikick.x /opt/gamess/gamess.00.x test -ddi 1 1 machinename -scr /tmp Distributed Data Interface kickoff program. Initiating 1 compute processes on 1 nodes to run the following command: /opt/gamess/gamess.00.x test ****************************************************** * GAMESS VERSION = 1 MAY 2013 (R1) * * FROM IOWA STATE UNIVERSITY * * M.W.SCHMIDT, K.K.BALDRIDGE, J.A.BOATZ, S.T.ELBERT, * * M.S.GORDON, J.H.JENSEN, S.KOSEKI, N.MATSUNAGA, * * K.A.NGUYEN, S.J.SU, T.L.WINDUS, * * TOGETHER WITH M.DUPUIS, J.A.MONTGOMERY * * J.COMPUT.CHEM. 14, 1347-1363(1993) * **************** 64 BIT LINUX VERSION **************** ... INPUT CARD> DDI Process 0: shmget returned an error. Error EINVAL: Attempting to create 160525768 bytes of shared memory. Check system limits on the size of SysV shared memory segments. The file ~/gamess/ddi/readme.ddi contains information on how to display the current SystemV memory settings, and how to increase their sizes. Increasing the setting requires the root password, and usually a sytem reboot. DDI Process 0: error code 911 ddikick.x: application process 0 quit unexpectedly. ddikick.x: Fatal error detected. The error is most likely to be in the application, so check for input errors, disk space, memory needs, application bugs, etc. ddikick.x will now clean up all processes, and exit... ddikick.x: Sending kill signal to DDI processes. ddikick.x: Execution terminated due to error(s). unset echo ----- accounting info ----- Files used on the master node machinename were: -rw-r--r-- 1 username username 0 2014-04-04 22:51 /tmp/test.dat -rw-r--r-- 1 username username 1341 2014-04-04 22:51 /tmp/test.F05 ls: No match. ls: No match. ls: No match. Fri Apr 4 22:51:36 EDT 2014 0.0u 0.0s 0:01.08 9.2% 0+0k 0+8io 0pf+0w

Things worked, but with a memory error. This issue is discussed at the Baldridge Group wiki: ocikbapps.uzh.ch/kbwiki/gamess_troubleshooting.html

From the wiki:

If you are sure you are not asking for too much memory in the input file, check that your kernel parameters are not allowing enough memory to be requested. You might have to increase the SHMALL & SHMAX kernel memory values to allow GAMESS to run. (See http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/ for a better explanation.)
For example, on a machine with 4GB of memory, you might add these to /etc/sysctl.conf:
# cat /etc/sysctl.conf | grep shm
kernel.shmmax = 3064372224
kernel.shmall = 748137
Then set the new settings like so:
# sysctl -p
Since they are in /etc/sysctl.conf, they will automatically be set each time the system is booted.

In our case, we modify sysctl.conf with the recommendations from the wiki:

username@machinename:~$ sudo nano /etc/sysctl.conf

Add the following to the bottom of the file:

kernel.shmmax = 3064372224 kernel.shmall = 748137

Save and exit.

username@machinename:~$ sudo sysctl -p

net.ipv4.ip_forward = 1 kernel.shmmax = 3064372224 kernel.shmall = 748137

These memory values will change depending on your system.

Now we empty the /tmp and rerun.

username@machinename:~$ rm /tmp/*
username@machinename:~$ ./rungms test.inp

If your input file is worth it’s salt, you’ll have successfully run your file on a single processor (single core, that is). If you run into additional memory errors, increase kernel.shmmax and kernel.shmall.

Now, onto the SMP part. My first attempt to run games in parallel (on 4 cores using version 00) produced the following error:

username@machinename:~$ rm /tmp/*
username@machinename:~$ ./rungms test.inp 00 4

----- GAMESS execution script 'rungms' ----- This job is running on host machinename under operating system Linux at Fri Apr 4 22:52:52 EDT 2014 Available scratch disk space (Kbyte units) at beginning of the job is Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 1905222596 249225416 1559217456 14% / GAMESS temporary binary files will be written to /tmp GAMESS supplementary output files will be written to /tmp Copying input file test.inp to your run's scratch directory... cp test.inp /tmp/test.F05 unset echo I do not know how to run this node in parallel.

I tried a number of stupid things to get the run to work, finally settling on modifying the rungms file properly. To make gamess know how to run the node in parallel, we need only make the following changes to our rungms file.

username@machinename:~$ nano rungms

Scroll down until you find the section below:

# 2. This is an example of how to run on a multi-core SMP enclosure, # where all CPUs (aka COREs) are inside a -single- NODE. # At other locations, you may wish to consider some of the examples # that follow below, after commenting out this ISU specific part. if ($NCPUS > 1) then switch (`hostname`) case se.msg.chem.iastate.edu: case sb.msg.chem.iastate.edu: if ($NCPUS > 2) set NCPUS=4 set NNODES=1

The change is simple. We remove the cases for $NCPUS > 1 in the file and add the hostname of our linux box (and if you don’t know this or it’s not in your prompt, simply type hostname at the prompt first). We’ll disable the two cases listed and add our hostname to the case list.

# 2. This is an example of how to run on a multi-core SMP enclosure, # where all CPUs (aka COREs) are inside a -single- NODE. # At other locations, you may wish to consider some of the examples # that follow below, after commenting out this ISU specific part. if ($NCPUS > 1) then switch (`hostname`) case machinename: # case se.msg.chem.iastate.edu: # case sb.msg.chem.iastate.edu: if ($NCPUS > 2) set NCPUS=4 set NNODES=1

This gives you parallel functionality, but it’s still not using the machine resources (cores) correctly when I ask for anything more than 2 cores (always using only 2 cores).

[minor complaint]
Admittedly, I don’t immediately get the logic of this section as currently coded, as one cannot get more than 2 cores to work in this case given how the if statements are written (so far as I can see now. I will assume I am the one missing something but have not decided to ask about it, instead changing the rungms text to the following). You can check this yourself by running top in another window. This is the most simple modification, and assumes you want to run N number of cores each time. Clearly, you can make this more elegant than it is (my modification, that is). Meantime, I want to run 4 cores on this machine, so I change the section to reflect a 4-core board (and commented out much of this section).
[/complaint]

# 2. This is an example of how to run on a multi-core SMP enclosure, # where all CPUs (aka COREs) are inside a -single- NODE. # At other locations, you may wish to consider some of the examples # that follow below, after commenting out this ISU specific part. if ($NCPUS > 1) then switch (`hostname`) case machinename # case se.msg.chem.iastate.edu: # case sb.msg.chem.iastate.edu: # if ($NCPUS > 2) set NCPUS=2 # set NNODES=1 # set HOSTLIST=(`hostname`:cpus=$NCPUS) # breaksw # case machinename # case br.msg.chem.iastate.edu: if ($NCPUS >= 4) set NCPUS=4 set NNODES=1 set HOSTLIST=(`hostname`:cpus=$NCPUS) breaksw case machinename # case cd.msg.chem.iastate.edu: # case zn.msg.chem.iastate.edu: # case ni.msg.chem.iastate.edu: # case co.msg.chem.iastate.edu: # case pb.msg.chem.iastate.edu: # case bi.msg.chem.iastate.edu: # case po.msg.chem.iastate.edu: # case at.msg.chem.iastate.edu: # case sc.msg.chem.iastate.edu: # if ($NCPUS > 4) set NCPUS=4 # set NNODES=1 # set HOSTLIST=(`hostname`:cpus=$NCPUS) # breaksw # case ga.msg.chem.iastate.edu: # case ge.msg.chem.iastate.edu: # case gd.msg.chem.iastate.edu: # if ($NCPUS > 6) set NCPUS=6 # set NNODES=1 # set HOSTLIST=(`hostname`:cpus=$NCPUS) # breaksw default: echo I do not know how to run this node in parallel. exit 20 endsw endif #

And, with this set of changes, I’m using all 4 cores on the board (but have some significant memory issues when running MP2 calks. But that’s for another post).

The typical user will never be able to do what the GAMESS group has done in making an excellent program that also happens to be free. That said, the need to make changes to the rungms file is something that would be greatly simplified by having N number of rungms scripts for each case instead of a monolithic file that is mostly useless text to users not using one of the system types. This, for instance, would make rungms modification much easier. If I streamline rungms for my specific system, I may post a new file accordingly.

Bash Script For Generating 3D Potential Energy Surface Scan Input Files

Monday, October 9th, 2006

INPUTFILETEMPLATE.gjf
1_PESscan.bash
2_bcf.bash
3_gaussian_bcf_top4lines.txt

The above scripts were made to overcome some of the potential energy surface (PES) scan limitations (well, design issues I’d rather not design around) in GAMESS and Gaussian. The intent of the scripts are to:

a) take a molecular input file from some quantum chemical code

b) align the molecule such that the surface you want to perform the PES scan across is aligned in the XY, YZ, or XZ planes

c) define the atom/molecule you want to scan with (such as a cation) in variable form in the input file template

d) define the grid size you want to perform the scan with (how fine a mesh)

e) generate the input files

and finally,

f) write out a command line script in a format that leaves for little post-processing for both the calculation and the analysis.

The demo case will be the (nominally, depending on the level of theory) C2v molecular anion (-1) diphenylmethanide in Gaussian, which will also hit a couple of technical points to speed up the PES scan. The intent was to take a cation (K+, Rb+) and determine the binding energies of the cation/anion pair over the (here, YZ) plane of the molecule to look at preferred binding positions and, more interestingly, the range in binding energies at different positions (see figure below).

pes grid

A few setup point for the script (general):

1. The molecule should be aligned along a plane (assuming it is planar. If not, you’ll have to tweak the code a little to taste, which isn’t too bad) because the script is going to sweep along Y and Z with the assumption that X is a constant vertical value above the plane of the molecule. The goal is to make a 2D map of a surface at one height, then change the height and redo the scan to look at the energy differences in sheets.

2. A molecule optimized in GAMESS and Gaussian should, provided symmetry is employed, orient the molecule such that a molecular plane is aligned along a Cartesian plane. In Gaussian, using the keyword symm(loose,follow) will both nudge a molecule into a higher symmetry and maintain that higher symmetry in the optimization. For the sake of the surface scan, this restriction of the molecule to a higher point group saves significant time, makes the analysis of the script results easier, and will not significantly alter the energies of a molecule whose minimum lies close to a higher symmetry form anyway (the differences in energy between the C2v and Cs diphenylmethanide structures, for instance, is on the order of a few kJ/mol by most levels of theory when the level of theory does NOT predict the C2v form to be the minimum).

3. Negative values are OK in the script, although some shells may not like having the “-” sign in the name of the files. If this is a problem on your machine, take the oriented molecule from Step 1 and add some constant value to all of the X, Y, Z coordinates to make the corner of the scan you start the X,Y,Z analysis from be (X,0,0). I use “X” here because the molecule may best be aligned to the X = 0 plane, which is easiest for some of the later post-processing.

4. If your molecule has two mirror planes (such as diphenylmethanide), set up the scan such that you’re only looking at HALF the structure. That should make perfect sense.

The scripts are provided in the links below. Their uses and results are as follows:

Input File 0. INPUTFILETEMPLATE – this file contains the parameters and coordinates for the PES scan. Make sure that this isn’t just a cut-and-paste of the optimization file (change the keywords so you run only SINGLE POINT CALCULATIONS!). More on this file below.

Input File 1. 1_PESscan.bash – the bash script that takes a template input file and generates (a) all of the single-point energy input files for the scan and (b) a batch file for submitting all of the files.

Input File 2. 2_bcf.bash – this is Gaussian-specific for the standard Gaussian PC interface, where the script for running multiple jobs is defined in the batch control file (bcf). This script gets the formatting right. Not needed for GAMESS, etc., calculations.

Input File 3. 3_gaussian_bcf_top4lines.txt – this file could be embedded into the 2_bcf.bash file if you like. Part of the 2_bcf.bash script catenates (with cat) the contents of this file and the contents of name files in the right bcf format. This will make perfect sense after the first run.

Output File 4. 4_batchscript.bat – this file (with the numerous input files) is generated from the 1_PESscan.bash script and contains all of the input files and execution parameters for the prompt. Will be Windows- or UNIX-friendly if you did it right.

Output File 5. 5_bcf_for_gaussian.bcf – this is the batch control file for Windows-based Gaussian calculations generated from 2_bcf.bash.

Specifics of each file are provided below:

Input File 0. INPUTFILETEMPLATE

The input template file should look like the following…

%mem=500MB
%nproc=2
# scf=tight b3lyp/6-31+g(d,p) integral(grid=ultrafine)

diphenylmethanide surface scan

0 1
K             REPLACEX    REPLACEY    REPLACEZ
C             0.000000    0.000000    0.940353
C             0.000000    1.308620    0.380671
...

Note that the molecule is aligned along X=0 and that I replaced opt with scf from the molecular optimization. You don’t want to disappear for a weekend only to find that you’ve done 4 structure optimizations when you could have done 100 single-point calculations. The cation we’re sliding with is defined with its X,Y,Z coordinates as REPLACEX,REPLACEY,REPLACEZ. These variables will be replaced by values along the PES grid by running 1_PESscan.bash.

Input File 1. 1_PESscan.bash

The usage of this bash script is:

./1_PESscan.bash $1 $2 $3 $4 $5 $6

or, for instance,

./1_PESscan.bash Kdiphenylmethanide gjf g03 100 25 25

Here are what the variables are.

$1 = name of the input file template (no extension)
$2 = name of the file extension (gjf for Gaussian, inp for GAMESS, etc.)
$3 = command line executable (g03 for Gaussian, gms… for GAMESS
$4 = decimal increments for the X axis (100 = unit, no decimals)
$5 = decimal increments for the Y axis (100 = unit, no decimals)
$6 = decimal increments for the Z axis (100 = unit, no decimals)

You’re limited to nine variables in the command line, so there’s one modification you need to make to the 1_PESscan.bash file itself. In the first section, MINX through MAXZ need to be defined. These are the integer steps taken by the script to generate the surfaces. These will depend on how you oriented your molecule. The decimal increments will not (which is why they’re called in the command line).

Input File 2. 2_bcf.bash

There’s nothing to edit here. It will take all of the Gaussian input (.gjf) files in a directory and make the corresponding .bcf file. Major time-saver.

Input File 3. 3_gaussian_bcf_top4lines.txt

This file contains the following…

!
!user created batch file list
!start=1
!

Which is just the typical 4 top lines in a bcf file (start referring to which molecule in the series gets run first).
And that’s the worst of it. Definitely practice the script(s) a few times before beginning a calculation, expect some fuss (depending on how you’ve pathed your executables) initially, and start with very COARSE grid sizes before wasting a lot of time on generating and checking files (decimal increments of 100 or 50, for instance).

A brief word on the input/output files. The format for the file names generated from 1_PESscan.bash are as follows.

INPUTFILETEMPLATE_X_2.000000_Y_0.000000_Z_0.000000.gjf
INPUTFILETEMPLATE_X_2.000000_Y_0.000000_Z_0.500000.gjf
INPUTFILETEMPLATE_X_2.000000_Y_0.000000_Z_1.000000.gjf
INPUTFILETEMPLATE_X_2.000000_Y_0.000000_Z_1.500000.gjf
...

The format (with six decimal places) is used here so that a user can simply use “ls *.gjf > file.txt” to generate a text file with all of the scanned coordinates, which can then be opened in Excel or something to make the position columns. Further, of course, is that a grep for the final energies in these files will make another file with the coordinates and energies together, which is then easily tweaked and plotted.

That’s the worst of it, short of plotting a few surfaces. Any questions or better ideas, drop a line.

www.msg.ameslab.gov/GAMESS/
www.gaussian.com

Synthetic, Structural And Theoretical Investigations Of Alkali Metal Germanium Hydrides – Contact Molecules And Separated Ions

Thursday, August 31st, 2006

In press, available from Chemistry – A European Journal. This is a paper a year or so in the making that, had I started it a year from now, would have taken a very different route. Much of the work I’ve done in neutron and terahertz spectroscopy has demonstrated that the inclusion of the crystal environment in quantum chemical treatments of solid-state systems is the key to interpreting the data (makes sense). This paper examines the unusual orientation of the [GeH3]- anion in two crown ether complexes with potassium (K+) and rubidium (Rb+) cations. The crystal cells of these two complexes are far larger than computational resources would handle now (and definitely when the project started), but they’d be easily handled on better equipment (such as an 8-processor box with a terabyte or so of scratch space). The isolated molecule calculations (Dr. Alex Granovsky’s PC-GAMESS version with basis sets from EMSL) demonstrate that the potential energy surfaces corresponding to anion orientations in the vicinity of only the solvated cations is shallow (at best) and that any moderate collection of electrostatic interactions (such as those in the crystal cell) may be enough to stabilize the unexpected anion orientation. It is also of interest to note that the [GeH3]- anion prefers to bind to the K/Rb cations by the hydrogens (which we refer to as “inverted”) and NOT the germanium anionic lone pair (the traditional, van’t Hoff arrangement, all of those calculations being performed at B3LYP/6-311G(d,p) and MP2/6-311G(d,p) levels of theory with LANL2DZ ECPs for the K and Rb). This “oddity” was considered previously by the great Paul v. R. Schleyer and coworkers for a similar Na-SiH3 system some time back (Angew. Chem. 1994, 106, 221-223). This project will hopefully be revisited with solid-state density functional theory to see just how the crystal interactions combine to impose the non-traditional [GeH3]- binding orientation.

karin geh3

W. Teng, D. G. Allis and K. Ruhlandt-Senge

Abstract: The preparation of a series of crown ether-ligated alkali metal (M = K, Rb, Cs) germyl derivatives M(crown ether)GeH3 via hydrolysis of the respective tris(trimethylsilyl)germanides is reported. Depending on the alkali metal and the crown ether diameter, the hydrides display either contact molecules or separated ions in the solid state, providing a unique structural insight into the geometry of the obscure GeH3- anion.

Germyl derivatives displaying M-Ge bonds in the solid state are of the general formula M([18]crown-6)(thf)GeH3 with M = K, 1; M = Rb, 4. Interestingly, the lone pair at germanium is not pointed towards the alkali metal, rather two of the three hydrides are approaching the alkali metal center to display M-H interactions.

Separated ions display alkali metal cations bound to two crown ethers in a sandwich-type arrangement and non-coordinated GeH3- anions to afford complexes of the type [M(crown ether)2][GeH3] with M = K, crown ether = [15]crown-5, 2; M = K, crown ether = [12]crown-4, 3 and M = Cs, crown ether = [18]crown-6, 5.

The highly reactive germyl derivatives were characterized using X-ray crystallography, 1H and 13C NMR, and IR spectroscopy. Density functional theory (DFT) and Second-Order Moeller-Plesset perturbation theory (MP2) calculations were performed to analyze the geometry of the GeH3- anion in the contact molecules 1 and 4.

www3.interscience.wiley.com/cgi-bin/jhome/26293?CRETRY=1&SRETRY=0
www3.interscience.wiley.com/cgi-bin/abstract/112234636/ABSTRACT
en.wikipedia.org/wiki/Jacobus_Henricus_van_’t_Hoff
www.emsl.pnl.gov/forms/basisform.html
chemistry.syr.edu/faculty/ruhlandt.html
classic.chem.msu.su/gran/gamess
www.chem.uga.edu/schleyer

Extension Of The Single Amino Acid Chelate Concept (SAAC) To Bifunctional Biotin Analogues For Complexation Of The M(CO)3+1 Core (M = Tc And Re): Syntheses, Characterization, Biotinidase Stability And Avidin Binding

Thursday, March 30th, 2006

In press, available from the journal Bioconjugate Chemistry. The modeling study for the avidin-biotin structure and the biotin derivatives were completed with the molecular dynamics program NAMD on a Dual G4/450 loaned to me from Apple for development work, for which I am grateful (I’ve performed molecular dynamics simulations with the Walrus). I did manage to smoke the motherboard during this experience, for which I apologize. Given the state of the machine after the autopsy, I’m hoping no one (especially Eric Zelman!) asks for it back, even when I’m 64.

I made mention of the reasons for some of this work in an interview I did for nanotech.biz, completely unrelate to the other content, in case anyone wants some background.

Shelly James, Kevin P. Maresca, Damian G. Allis, John F. Valliant, William Eckelman, John W. Babich, and Jon Zubieta

Abstract: Biotin and avidin form one of the most stable complexes known (KD = 10-15M-1) making this pairing attractive for a variety of biomedical applications including targeted radiotherapy. In this application one of the pair is attached to a targeting molecule while the other is subsequently used to deliver a radionuclide for imaging and/or therapeutic applications. Recently we reported a new single amino acid chelate (SAAC) capable of forming robust complexes with Tc(CO)3 or Re(CO)3 cores. We describe here the application of SAAC analogs for the development of a series of novel radiolabeled biotin derivatives capable of forming robust complexes with both Tc and Re. Compounds were prepared through varying modification of the free carboxylic acid group of biotin. Each 99mTc complex of SAAC-biotin was studied for their ability to bind avidin, susceptibility to biotinidase and specificity for avidin in an in vivo avidin-containing tumor model. The radiochemical stability of the 99mTc(CO)3 complexes was also investigated by challenging each 99mTc-complex with large molar excesses of cysteine and histidine at elevated temperature. All compounds were radiochemically stable for greater than 24 hours at elevated temperature in the presence of histidine and cysteine. Both [99mTc(CO)3(L6)]+1 [TcL6; L6 = biotinyl- amido- propyl- N,N- (dipicolyl)- amine] and [99mTc(CO)3(L12a)]+1 (TcL12; L12 = N,N-(dipicolyl)- biotin- amido- Boc- lysine; TcL12a; L12a = N,N- (dipicolyl)- biotin- amide- lysine) readily bound to avidin whereas [99mTc(CO)3(L9)]+1 [TcL9; L9 = N,N- (dipicolyl)- biotin- amine] demonstrated minimal specific binding. TcL6 and TcL9 were resistant to biotinidase cleavage while TcL12a, which contains a lysine linkage, was rapidly cleaved. The highest uptake in an in vivo avidin tumor model was exhibited by TcL6, followed by TcL9 and TcL12a, respectively. This is likely the result of both intact binding to avidin and resistance to circulating biotinidase. Ligand L6 is the first SAAC analogue of biotin to demonstrate potential as a radiolabeled targeting vector of biotin capable of forming robust radiochemical complexes with both 99mTc and rhenium radionuclides.Computational simulations were performed to assess biotin-derivative accommodation within the binding site of the avidin. These calculations demonstrate that deformation of the surface domain of the binding pocket can occur to accommodate the transition metal-biotin derivatives with negligible changes to the inner-β-barrel, the region most responsible for binding and retaining biotin and its derivatives.

P.S. This publication is also of some use for explaining the series of images on the current departmental brochure for the Syracuse U. Chemistry Department. Steric interactions affect the local geometries of protein binding pockets. And a good thing, too.


Click on the image for a larger version.

www.apple.com
www.applecorps.com
chemistry.syr.edu/faculty/zubieta.html
www.molecularinsight.com
www.chemistry.mcmaster.ca/people/faculty/valliant/index.html
pubs.acs.org/journals/bcches/index.html
www.ks.uiuc.edu/Research/namd/

Everything You Need To Set Up PC-GAMESS On A(n) SMP System

Wednesday, January 4th, 2006

The following is a step-by-step guide to getting PC-GAMESS 6.4 (perhaps prior. I’m not inclined to check, but would appreciate a confirmation) running on a(n) SMP (that’s Symmetric MultiProcessing (useless trivia), a single motherboard with multiple processors) system. All of the websites describing what’s below are less… obvious than I would like, especially concerning the infamous .pg file for SMP machines. What’s below is specifically geared for a dual-core/dual-board (four processor) system, but is easily changed to other SMP cases.

1. Obtain the most recent (6.4 as of this posting) version of PC-GAMESS from the good Dr. Alex Granovsky.

2. After extracting the contents of the downloaded .exe file, you’ll notice the wmpi1_3.exe file. Double click to install this program, blindly saying yes to everything it asks.

3. In your c:\ directory (dig into My Computer), create the sub-directories pg1, pg2, pg3, and pg4 (note: the names are completely arbitrary). When that’s done, your folders should look like the following (PEXECUTE.BAT and PRUNGAMESS.BAT are separate batch files you can download below):

4. Into c:\pg1, place all of the extracted PC-GAMESS files (whatever came out of the .exe run). The folder should look like this (with three input files added by myself for testing):

5. Into the folders c:\pg2, c:\pg3, and c:\pg4, copy and paste (from c:\pg1) the files FASTDIAG.DLL, PCGP2P.DLL, and PGAMESS.EXE. These directories will look like the following:

6. Back in c:\pg1, create or place pgamess.pg, the wmpi config file you can download HERE (if you download, remove the .txt extension before use). The file is, simply,

local 0
localhost 1 c:\pg2\pgamess.exe
localhost 1 c:\pg3\pgamess.exe
localhost 1 c:\pg4\pgamess.exe

No surprises, but I’ve never found the SMP .pg file listed anywhere. This file is just for SMP runs. local 0 refers to the “base” copy of PGAMESS (in c:\pg1, the PC-GAMESS home directory). The localhost 1 lines call each of the other three directories and PGAMESS.EXE for the three remaining processors (in the quad box). For a dual core or dual CPU box, you’d remove c:\pg3 and c:\pg4 and delete the third and fourth lines in pgamess.pg.

7. The command line will look like the following (note: the input file must be named “input“):

c:\pg1> PGAMESS.EXE c:\pg1 c:\pg2 c:\pg3 c:\pg4 -p4local > filename.out

All should, in theory, work without hitch. For a batch-type system (you can’t add new files, but you can run existing jobs in the same directory automatically and sequentially), download the files PEXECUTE.BAT and PRUNGAMESS.BAT and place them into c:\pg1 (make sure to remove the .txt extension before use if you download these). To run this script, simply double-click on PRUNGAMESS.BAT.

NOTE1: Of course, what’s shown is not the most efficient way to run PC-GAMESS. For maximum speed-up, you’ll want a single hard drive dedicated to EACH processor (so each temp file is being written to a different disk). Your folders c:\pg1, c:\pg2, c:\pg3, and c:\pg4 would then be c:\pg1, d:\pg2, e:\pg3, and f:\pg4.

NOTE2: A number of my calculations fail randomly with MPI memory allocation/write errors on dual core/dual cpu AMD Opteron machines running Windows XP. These errors are actually hard drive write problems and not RAM problems. You can get around this problem (errors and OK buttons will pop up) by running the calculations in-RAM (DIRSCF=.TRUE.) and ramping up the amount of used RAM (with the MEMORY= keyword).

Obligatory

  • CNYO

  • Sol. Sys. Amb.

  • Ubuntu 4 Nano

  • NMT Review

  • N-Fact. Collab.

  • T R P Nanosys

  • Nano Gallery

  • nano gallery
  • Aerial Photos

    More @ flickr.com

    Syracuse Scenes

    More @ flickr.com