Tag: gamess-us

Posted on

Design Of A Molecular Field Effect Transistor (mFET) – Links And Commentary

Above: Multiple views of the proposed mFET (7694 atoms), with colors at left used to distinguish all components in the design. The total volume of the mFET is 46.52 nm3 based on van der Waals radii.

Update: 15 May 2025 – The most recent version of the article is now up on arXiv (h/t Tad Hogg) – see https://arxiv.org/abs/2505.05693.

Ralph Merkle, Robert A. Freitas Jr., and I had worked on a design for a molecular field effect transistor a few years ago, then all sorts of distractions got in the way of its completion (IYKYK). A recent return to it led to quick turnarounds of several significant iterations and improvements and, now, recent addition to the Reports section of the Institute for Molecular Manufacturing (IMM) website (www.imm.org) Like all such reports, it's free and open to anyone interested – I direct you to www.imm.org/Reports/rep056.pdf for the latest version.

Above: Give it a spin (and zoom) courtesy of 3DMol.

Below, I'm including some additional notes on the overall process for anyone wanting to take a stab at redesigns, considering similar design efforts, or just have questions about aspects of the analysis. There is a lot someone could do in any future optimization and assessment process with readily available theoretical tools. This first pass is just to, as such designs have always been for, get the conversation started.

DFTB

And I can't stress this enough. I'm now approaching 25 years of "thinking small" about nanotechnology (N), molecular nanotechnology (MNT), molecular manufacturing (MM), advanced molecular manufacturing (AMM), and now atomically-precise manufacturing (APM). Don't look at me, I was just doing theory.

One of the great challenges throughout the entirety of this effort has been trying to get more done with a sub-par coverage of theoretical tools for the size range of structures and behaviors being considered. Interfacing theoretical approaches to bridge the gap between molecular mechanics (MM) and density functional theory (DFT) – the two workhorses for all of this effort to date – have historically been too approximate or too slow (sometimes both). The MM-to-DFT divide is a gigantic (no pun intended) bridge to span, and it left the entirety of this field in the 200N's divided into (1) quantum chemical analyses of mechanosynthetic operations (dimers, ex: 1, 2, 3) and (2) molecular dynamics simulations of diamondoid behavior (bearings, gears, sorting rotors – see what Bryan Bishop has kept spinning at github.com/kanzure/nanoengineer).


From the paper: Figure 7. The model channel-insulator structure produced for assessing the performance of the DFTB PBC-0-3 parameter set against full DFT calculations (see text). The overall structure and components were optimized under C2v symmetry constraints. Click for a larger view.

I made a tutorial about bearings and nanotubes many years ago (A Low-Friction Molecular Bearing Assembly Tutorial, v1) and lamented it suffering mightily from being a molecular mechanics-heavy assessment. At the time, however, theoretical options were limited. The calculations were all done in CAChe (I can't even find links!) on a G4 Mac graciously donated to me by an Apple Rep (after my driving two hours to a seminar he was giving on Apple and Scientific Computing. I think everyone else in attendance was from within the same building). GAMESS-US was the only free program one could get their hands on to do DFT on a Mac, but virtually nothing about that tutorial or those structures was quantum-friendly.

Density functional tight binding (DFTB) methods are fast becoming that bridge, with optimizations of large components in this mFET structure taking only a "reasonable" bit longer than molecular mechanics would have taken 25 years ago. That's a tremendous improvement in the state of theoretical methods when one can work through an entire design and analysis process with calculations that, thanks to considerable parameterization efforts, are in very good agreement with much more involved theoretical calculations (in the mFET study, some time is spent comparing DFTB and DFT just to show the reasonable agreements).


From the paper: Figure 13. Modifications that can be made to the proposed insulator design to approach more “vacuum-like” behavior by simple modification to the Lonsdaleite framework. Click for a larger view.

The spirit of this paper is the proposal of a molecular FET, not an exhaustive comparison of theoretical methods on model systems to show where good agreement could be obtained for selections of calculable properties. That comparative study is something we are looking at in the mid-term.

That said, and this has come up in conversation on X in the not-too-distant past, the universe described by DFTB and a given parameter set is not our reality. The same is true for DFT. The same is still true for CCSD(T) (despite it being damn-close if you're a molecule!). In the Venn diagram of DFT, DFTB, and reality, there may be designs and devices that are equally well-described by DFT and DFTB, and that description might be very close to the behavior observed for an actual device. If you believe that to be the case, and you continue to accept that we're a looooong way away from making something like this, then a plausible path to production is to design exhaustively in the DFTB space and use that assessment as a "first gate" for feeding into DFT-level work. You, of course, risk missing a much better design that would have come about had you started with DFT calculations, but you, at least, improve your chances through DFTB (for instance) of narrowing the total design space to a clunky, slow, sub-par, smoke-billowing device THAT DOES WHAT YOU WANT. After that, you improve the design – and the basic science gets handed off to the engineers.


From the paper: Figure 8. Two views of the (20,20) CNT gate (1774 atoms) from the optimized mFET. At right, the connectivity of the (4,4) CNT to the (20,20) CNT. Click for a larger view.

There are also some sharp divides between the quantum chemistry of the components and the formulaic assessments of performance and behavior in the mFET paper. The bridging of that gap is ever-ongoing by the entire community and we await determination of where the gaps in both reside at the nanoscale.

Programs and Parameters

Gaussian 16 was the DFT workhorse for this work. DFTB+ has become the structure optimization go-to for virtually every part of design processes I find myself doing because it is fast and, for the properties of interest, accurate. PBC-0-3 might not be the best of the available parameter sets for C/H-only systems, but it has shown itself here to be a completely reasonable set for optimizations and some property predictions (again, the goal of this report was not a method survey).


From the paper: Figure 4. Three views of the proposed Lonsdaleite channel, including identification of the single repeat unit and representative six-unit supercell. Blue dot at lower-right is the channel axis. Click for a larger view.

The one code added late in the game was XTB (which also sees extensive use in active work for, primarily, configuration sampling with Crest) for the specific prediction of electronic coupling between the channel and Lonsdaleite insulator through the calculation of orbital overlap integrals. The issue it was used to provide some first-pass address of – the potential for insulator electrons tunneling into channel orbitals due to the uncomfortably-ordered frontier orbital energies of both – is a very nontrivial one to deal with at the design level, one that has definite methodological dependencies, and is even worse when you account for thermal behavior. The brute-force solution is in Figure 13 (above), where the insulator can be carved out to reduce close contacts (the breakdown issues associated with that are discussed in the paper).


From the paper: Figure 14. Assessment of orbital overlap between the channel (C) and insulator (I) in our model test system, employing the DIPRO method within XTB to obtain JAB,eff for two variants of the C/I model and a benzene-nitrobenzene complex (shown enlarged and to scale with the C/I models) [27]. Click for a larger view.

The fact that you can get an estimate (at all) in 15 seconds and cite a paper from which the XTB-generated values are not too dissimilar from wB97X-D4/TZ2P calculations across the HAB79 dataset is another testament to the quality of DFTB methods.

Bonus pics from left: The highest-lying channel-localized orbital, one of the few frontier orbitals with any contribution from the channel, and the insulator-localized HOMO (from CAM-B3LYP-GD3(BJ)/6-31G(d,p) calculations).

GUI

Ralph did the first pass in HyperChem. I, since 2007, continue to use NanoEngineer-1 almost daily. I am encouraged by the possibilities within MSEP for doing this work and find some of the build approaches to be, like NE-1, less chemical and more CAD-like – which is great for this type of design work (all the better that it's new and supported on modern OSs with no issues). SAMSON remains gorgeous, opens and assigns all of the atoms in this structure on first try from an .xyz file, and is very fast on the optimizations.


From the paper: Figure 3. Top: Londsaleite channel with attached (4,4) CNT leads (502 atoms). Bottom: close up of the channel in van der Waals (left) and ball-and-stick (right) representations. Click for a larger view.

Highly Irreversible Computing

"You can't un-toast the toast." – Ruff Ruffman, PBS Kids

Ignoring the capabilities of a sufficiently advanced technology, the above statement is quite correct. After a little bit of math, the full-speed assessment of the proposed mFET is summarized as follows:

This hypothetical sugar-cube-sized computer switching at 10 THz would generate the energy of the Hiroshima blast every six seconds. Cooling would be challenging.

That's the more immediate concern to the proposed design, with solutions proposed that bring the exothermal nature of the operation down to reasonable levels. As an homage to some previous work addressing much improved overall efficiency and thermal behavior by way of mechanical computing, a brief discussion that ends with the following snippet follows the above:

This switching energy per operation is below the Landauer limit, meaning that a conventional (irreversible) logic device would be thermodynamically impossible to operate at this power level. This highlights the need to use reversible computing to achieve such densities.

LLMs – Use Them Before They Get Too Garbage-ified By Lousy Science In Predatory Open Access Journals

Ralph threw draft V2.6 into o3-mini-high to see what it thought of our work, then asked for a technical assessment. I'm 54 peer-reviewed papers into this whole "science thing" and I can count on one hand the number of times I've gotten reviewer comments back that were as thorough and informative as what we got back from o3-mini-high. We were all very pleasantly surprised.

It also helps that LLMs are so good at making you feel like a knowledgeable authority who knows what you're talking about. Luxuriating in the constant praise of o3-mini-high for one's "thorough consideration" and "astute insight" into some topic while it tells you that you missed an exponent or spelled "FET" wrong is certainly better than Reviewer #2 telling you that the paper should never have been submitted in the first place.


From the paper: Figure 5. Three views of the mFET Lonsdaleite insulator (5418 atoms). Click for a larger view.

A long-short LLM take ca. March, 2025 – when it's good, it's terrific. Just like wikipedia, within which tabloid-esque updates of famous people can be fought over all day long, but no respectable contributor wants to be the dummy who wrote "F = ma" wrong in a physics page. Cited references from o3-mini-high were, however, barely in the ballpark. That said, there's A LOT to train on concerning field effect transistors, basic physics, and computational chemistry. For this particular topic, the review was excellent and I'm sold on including a research-grade LLM assessment into any future technical writings (having been pleased with the physics, we'll see how good the chemistry is in a future paper).

Design Of A Molecular Field Effect Transistor (mFET)

Version 2.16, 2025-03-21
Ralph C. Merkle, Robert A. Freitas Jr. and Damian G. Allis
Institute for Molecular Manufacturing

Abstract

Field Effect Transistors (FETs) are ubiquitous in electronics. As we scale FETs to ever smaller sizes, it becomes natural to ask how small a practical FET might be. We propose and analyze an atomically precise molecular FET (herein referred to as an “mFET”) with 7,694 atoms made only of hydrogen and carbon atoms. It uses metallic (4,4) carbon nanotubes as the conductive leads, a linear segment of Lonsdaleite (hexagonal diamond) as the channel, Lonsdaleite as the insulating layer between the channel and the gate, and a (20,20) metallic carbon nanotube as the surrounding gate. The (4,4) nanotube leads are bonded to the channel using a mix of 5- and 6-membered rings, and to the gate using 5-,6- and 7-membered rings. Issues of component design assessment and optimization using quantum chemical methods are discussed throughout. A 10 watt sugar-cube-sized computer made with 1018 such mFETs could deliver ~1025 switching operations per second.

Posted on

GROMACS 5.0.1, nVidia CUDA Toolkit, And FFTW3 Under Ubuntu 14.04 LTS (64-bit); The Virtues Of VirtualBox

Summarized below are the catches and fixes from a recent effort to build GROMACS 5.0.1 with FFTW3 (single- and double-precision) and GPU support (so, single-precision). Also, a trick I've been doing with great success lately, using a virtual machine to keep my real machine as clean as possible.

0. The Virtues Of VirtualBox

Open source means never having to say you're sorry.

I've made the above proclamation to anyone who'd listen lately who has any interest in using Linux software (because, regardless of what anyone says on the matter, it ain't there yet as an operating system for general scientific users with general computing know-how). You will very likely find yourself stuck at a configure or make step in one or more prerequisite codes to some final build you're trying to do, leaving yourself to google error messages to try to come up with some kind of solution. Invariably, you'll try something that seems to work, only to find it doesn't, potentially leaving a trail of orphaned files, version-breaking changes, and random downgrading only to find something else stupid (or not) fixed your build problems.

I've an install I'm quite happy with that has all of the working code I want on it working – and I've no interest in having to perform re-installs to get back to a working state again.

My solution, which I've used to great success with GAMESS-US, GROMACS, NWChem, and Amber (so far), is to break a virtual instance in VirtualBox first. For those who don't know (and briefly), VirtualBox lets you install a fully-working OS inside of your own OS that simply sits as a file in a Virtual VM folder in your user directory. My procedure has been to install a 60 GB VirtualBox instance of (currently) Ubuntu 14.04 (which I will refer to here as PROTOTYPE), fully update it to the current state of my RealBox (updates, upgrades, program installs, etc.), then copy PROTOTYPE somewhere else on the machine. The only limitation of this approach is that VirtualBox doesn't give you access to the GPU if you're testing CUDA-specific calculations. That said, it does let you install the CUDA Development Toolkit and compile code just fine, so you can at least work your way through a full build to make sure you don't run into problems.

When you're done trashing your VirtualBox after a particularly heinous build, just delete PROTOTYPE from Virtual VM and re-copy your copy back into Virtual VM – voila! You're ready for another build operation (or to make sure your "final" build actually works flawlessly before committing the build to your RealBox.

That's all I have to say on the matter. Consider it as your default procedure (at this point, I won't touch my RealBox with new installs until I know it's safe in VirtualBox).

1. The State Of My Machine Pre-GROMACS And All Other apt-get's Used Below

What follows below is pretty straightforward. Errors you might get that don't appear below might be related to the lack of certain installs on your machine that I installed on VirtualBox. That is, my standard PROTOTYPE comes standard with Intel's Fortran and C Compilers (for code optimization). Those installs required a few installs above the base Ubuntu install. These are (and are pretty standard anyway, so I say install them anyway): 

sudo apt-get install build-essential gcc-multilib rpm openjdk-7-jre-headless

I could have just installed a fresh version of 14.04 onto a machine to try this myself, but I'm not that motivated. Also, note this list does not include the all-important cmake. We'll get to that.

And for the rest of GROMACS (at least for older versions), there were lots of mesa/gnuplot/motif-specific dependencies in older versions of GROMACS to build all of the files included in the GROMACS package. Regardless of GPU builds or not, I tend to default to install all the packages below just to have them (which all, for 14.04, currently apt-get properly).

sudo apt-get install openmpi-bin openmpi-common gfortran csh grace menu x11proto-print-dev motif-clients freeglut3-dev libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev libgl1-mesa-dri libcurl-ocaml-dev libcurl4-gnutls-dev gnuplot

If you don't install the libblas3gf libblas-doc libblas-dev liblapack3gf liblapack-doc liblapack-dev series, you'll see the following note from your cmake steps in GROMACS.

— A library with BLAS API not found. Please specify library location.
— Using GROMACS built-in BLAS.
— LAPACK requires BLAS
— A library with LAPACK API not found. Please specify library location.
— Using GROMACS built-in LAPACK.

My own preference is to use the (assumedly newer) Ubuntu-specific libraries from apt-get.

sudo apt-get install libblas3gf libblas-doc libblas-dev liblapack3gf liblapack-doc liblapack-dev

GPU-Specific? One More apt-get

My first passes at proper GPU compilation involved several steps for the nVidia Developer Toolkit install. That's now taken care of with apt-get, so perform the final apt-get to complete the component/dependency installations.

sudo apt-get install nvidia-cuda-dev nvidia-cuda-toolkit

With luck, your first attempt at a GPU-based installation will look like the following:

[0%] Building NVCC (Device) object src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_copyrite_gpu.cu.o
…
[100%] Building CXX object src/programs/CMakeFiles/gmx.dir/legacymodules.cpp.o
Linking CXX executable https://www.somewhereville.com/bin/gmx
[100%] Built target gmx

2. Nothing Happens Without cmake

Install cmake! Reproducing the output below to make sure you're using the same versions for everything (in the event something breaks in the future).

sudo apt-get install cmake

Reading package lists… Done
Building dependency tree
Reading state information… Done
The following packages were automatically installed and are no longer required:
linux-headers-3.13.0-32 linux-headers-3.13.0-32-generic
linux-image-3.13.0-32-generic linux-image-extra-3.13.0-32-generic
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
cmake-data
Suggested packages:
codeblocks eclipse
The following NEW packages will be installed:
cmake cmake-data
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 3,294 kB of archives.
After this operation, 16.6 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Get:1 http://us.archive.ubuntu.com/ubuntu/ trusty/main cmake-data all 2.8.12.2-0ubuntu3 [676 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ trusty/main cmake amd64 2.8.12.2-0ubuntu3 [2,618 kB]
Fetched 3,294 kB in 30s (106 kB/s)
Selecting previously unselected package cmake-data.
(Reading database … 258157 files and directories currently installed.)
Preparing to unpack …/cmake-data_2.8.12.2-0ubuntu3_all.deb …
Unpacking cmake-data (2.8.12.2-0ubuntu3) …
Selecting previously unselected package cmake.
Preparing to unpack …/cmake_2.8.12.2-0ubuntu3_amd64.deb …
Unpacking cmake (2.8.12.2-0ubuntu3) …
Processing triggers for man-db (2.6.7.1-1) …
Setting up cmake-data (2.8.12.2-0ubuntu3) …
Setting up cmake (2.8.12.2-0ubuntu3) …

3. First Pass At GROMACS

The make install step will place GROMACS where you want it on your machine, so you're just as good building in $HOME/Downloads as you are anywhere else. I will be performing all operations from $HOME/Downloads unless otherwise stated.

According to the GROMACS Installation Manual, your quick-and-dirty install need only involve the following:

$ tar xvfz gromacs-src.tar.gz
$ ls
gromacs-src
$ mkdir build
$ cd build
$ cmake ../gromacs-src
$ make

This allows you build "out-of-source" as they put it. Frankly, I just dive right into the GROMACS folder and have at it.

CMake Error: The source directory "/home/user/Downloads/gromacs-5.0.1/build" does not appear to contain CMakeLists.txt.
Specify –help for usage, or press the help button on the CMake GUI.

And did you see the above error? If so, you read the GROMACS quick-and-dirty procedure backwards. I'm not running it this way, so doesn't matter to what follows.

My first attempt at building GROMACS produced the following output from PROTOTYPE (reproducing all the text below).

user@PROTOTYPE:~$ cd Downloads/
user@PROTOTYPE:~/Downloads$ gunzip gromacs-5.0.1.tar.gz 
user@PROTOTYPE:~/Downloads$ tar xvf gromacs-5.0.1.tar

gromacs-5.0.1/README
gromacs-5.0.1/INSTALL
…
gromacs-5.0.1/tests/CppCheck.cmake
gromacs-5.0.1/tests/CMakeLists.txt 

user@PROTOTYPE:~/Downloads$ cd gromacs-5.0.1/
user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=OFF

NOTE: If you just run cmake, you'll get the following…

cmake version 2.8.12.2
Usage

cmake [options] cmake [options] …

… which is to say, cmake requires at least one option be specified. Above, I'm just using -DGMX_GPU=OFF to start the process.

The C compiler identification is GNU 4.8.2
— The CXX compiler identification is GNU 4.8.2
— Check for working C compiler: /usr/bin/cc
— Check for working C compiler: /usr/bin/cc — works
— Detecting C compiler ABI info
— Detecting C compiler ABI info – done
— Check for working CXX compiler: /usr/bin/c++
— Check for working CXX compiler: /usr/bin/c++ — works
— Detecting CXX compiler ABI info
— Detecting CXX compiler ABI info – done
— Checking for GCC x86 inline asm
— Checking for GCC x86 inline asm – supported
— Detecting best SIMD instructions for this CPU
— Detected best SIMD instructions for this CPU – SSE2
— Try OpenMP C flag = [-fopenmp]
— Performing Test OpenMP_FLAG_DETECTED
— Performing Test OpenMP_FLAG_DETECTED – Success
— Try OpenMP CXX flag = [-fopenmp]
— Performing Test OpenMP_FLAG_DETECTED
— Performing Test OpenMP_FLAG_DETECTED – Success
— Found OpenMP: -fopenmp
— Performing Test CFLAGS_WARN
— Performing Test CFLAGS_WARN – Success
— Performing Test CFLAGS_WARN_EXTRA
— Performing Test CFLAGS_WARN_EXTRA – Success
— Performing Test CFLAGS_WARN_REL
— Performing Test CFLAGS_WARN_REL – Success
— Performing Test CFLAGS_WARN_UNINIT
— Performing Test CFLAGS_WARN_UNINIT – Success
— Performing Test CFLAGS_EXCESS_PREC
— Performing Test CFLAGS_EXCESS_PREC – Success
— Performing Test CFLAGS_COPT
— Performing Test CFLAGS_COPT – Success
— Performing Test CFLAGS_NOINLINE
— Performing Test CFLAGS_NOINLINE – Success
— Performing Test CXXFLAGS_WARN
— Performing Test CXXFLAGS_WARN – Success
— Performing Test CXXFLAGS_WARN_EXTRA
— Performing Test CXXFLAGS_WARN_EXTRA – Success
— Performing Test CXXFLAGS_WARN_REL
— Performing Test CXXFLAGS_WARN_REL – Success
— Performing Test CXXFLAGS_EXCESS_PREC
— Performing Test CXXFLAGS_EXCESS_PREC – Success
— Performing Test CXXFLAGS_COPT
— Performing Test CXXFLAGS_COPT – Success
— Performing Test CXXFLAGS_NOINLINE
— Performing Test CXXFLAGS_NOINLINE – Success
— Looking for include file unistd.h
— Looking for include file unistd.h – found
— Looking for include file pwd.h
— Looking for include file pwd.h – found
— Looking for include file dirent.h
— Looking for include file dirent.h – found
— Looking for include file time.h
— Looking for include file time.h – found
— Looking for include file sys/time.h
— Looking for include file sys/time.h – found
— Looking for include file io.h
— Looking for include file io.h – not found
— Looking for include file sched.h
— Looking for include file sched.h – found
— Looking for include file regex.h
— Looking for include file regex.h – found
— Looking for C++ include regex
— Looking for C++ include regex – not found
— Looking for posix_memalign
— Looking for posix_memalign – found
— Looking for memalign
— Looking for memalign – found
— Looking for _aligned_malloc
— Looking for _aligned_malloc – not found
— Looking for gettimeofday
— Looking for gettimeofday – found
— Looking for fsync
— Looking for fsync – found
— Looking for _fileno
— Looking for _fileno – not found
— Looking for fileno
— Looking for fileno – found
— Looking for _commit
— Looking for _commit – not found
— Looking for sigaction
— Looking for sigaction – found
— Looking for sysconf
— Looking for sysconf – found
— Looking for rsqrt
— Looking for rsqrt – not found
— Looking for rsqrtf
— Looking for rsqrtf – not found
— Looking for sqrtf
— Looking for sqrtf – not found
— Looking for sqrt in m
— Looking for sqrt in m – found
— Looking for clock_gettime in rt
— Looking for clock_gettime in rt – found
— Checking for sched.h GNU affinity API
— Performing Test sched_affinity_compile
— Performing Test sched_affinity_compile – Success
— Check if the system is big endian
— Searching 16 bit integer
— Looking for sys/types.h
— Looking for sys/types.h – found
— Looking for stdint.h
— Looking for stdint.h – found
— Looking for stddef.h
— Looking for stddef.h – found
— Check size of unsigned short
— Check size of unsigned short – done
— Using unsigned short
— Check if the system is big endian – little endian
— Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version "2.9.1")
— Looking for xmlTextWriterEndAttribute in /usr/lib/x86_64-linux-gnu/libxml2.so
— Looking for xmlTextWriterEndAttribute in /usr/lib/x86_64-linux-gnu/libxml2.so – found
— Looking for include file libxml/parser.h
— Looking for include file libxml/parser.h – found
— Looking for include file pthread.h
— Looking for include file pthread.h – found
— Looking for pthread_create
— Looking for pthread_create – not found
— Looking for pthread_create in pthreads
— Looking for pthread_create in pthreads – not found
— Looking for pthread_create in pthread
— Looking for pthread_create in pthread – found
— Found Threads: TRUE
— Looking for include file pthread.h
— Looking for include file pthread.h – found
— Atomic operations found
— Performing Test PTHREAD_SETAFFINITY
— Performing Test PTHREAD_SETAFFINITY – Success
— Could NOT find Boost
Boost >= 1.44 not found. Using minimal internal version. This may cause trouble if you plan on compiling/linking other software that uses Boost against Gromacs.
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so – found
— Setting build user/date/host/cpu information
— Setting build user & time – OK
— Checking floating point format
— Checking floating point format – IEEE754 (LE byte, LE word)
— Checking for 64-bit off_t
— Checking for 64-bit off_t – present
— Checking for fseeko/ftello
— Checking for fseeko/ftello – present
— Checking for SIGUSR1
— Checking for SIGUSR1 – found
— Checking for pipe support
— Checking for isfinite
— Performing Test isfinite_compile_ok
— Performing Test isfinite_compile_ok – Success
— Checking for isfinite – yes
— Checking for _isfinite
— Performing Test _isfinite_compile_ok
— Performing Test _isfinite_compile_ok – Failed
— Checking for _isfinite – no
— Checking for _finite
— Performing Test _finite_compile_ok
— Performing Test _finite_compile_ok – Failed
— Checking for _finite – no
— Performing Test CXXFLAG_STD_CXX0X
— Performing Test CXXFLAG_STD_CXX0X – Success
— Performing Test GMX_CXX11_SUPPORTED
— Performing Test GMX_CXX11_SUPPORTED – Success
— Checking for system XDR support
— Checking for system XDR support – present
— Try C compiler SSE2 flag = [-msse2]
— Performing Test C_FLAG_msse2
— Performing Test C_FLAG_msse2 – Success
— Performing Test C_SIMD_COMPILES_FLAG_msse2
— Performing Test C_SIMD_COMPILES_FLAG_msse2 – Success
— Try C++ compiler SSE2 flag = [-msse2]
— Performing Test CXX_FLAG_msse2
— Performing Test CXX_FLAG_msse2 – Success
— Performing Test CXX_SIMD_COMPILES_FLAG_msse2
— Performing Test CXX_SIMD_COMPILES_FLAG_msse2 – Success
— Enabling SSE2 SIMD instructions
— Performing Test _callconv___vectorcall
— Performing Test _callconv___vectorcall – Failed
— Performing Test _callconv___regcall
— Performing Test _callconv___regcall – Failed
— Performing Test _callconv_
— Performing Test _callconv_ – Success
— checking for module 'fftw3f'
— package 'fftw3f' not found
— pkg-config could not detect fftw3f, trying generic detection
Could not find fftw3f library named libfftw3f, please specify its location in CMAKE_PREFIX_PATH or FFTWF_LIBRARY by hand (e.g. -DFFTWF_LIBRARY='/path/to/libfftw3f.so')
CMake Error at cmake/gmxManageFFTLibraries.cmake:76 (MESSAGE):
Cannot find FFTW 3 (with correct precision – libfftw3f for mixed-precision
GROMACS or libfftw3 for double-precision GROMACS). Either choose the right
precision, choose another FFT(W) library (-DGMX_FFT_LIBRARY), enable the
advanced option to let GROMACS build FFTW 3 for you
(-GMX_BUILD_OWN_FFTW=ON), or use the really slow GROMACS built-in fftpack
library (-DGMX_FFT_LIBRARY=fftpack).
Call Stack (most recent call first):
CMakeLists.txt:733 (include)

— Configuring incomplete, errors occurred!
See also "/home/user/Downloads/gromacs-5.0.1/CMakeFiles/CMakeOutput.log".
See also "/home/user/Downloads/gromacs-5.0.1/CMakeFiles/CMakeError.log".

Lots of little things to address here. We'll get to the Boost problem later. Meantime, you can see the critical error is in (1) the lack of FFTW3 and (2) the lack of my specifically asking for -DGMX_BUILD_OWN_FFTW=ON in the cmake process.

NOTE: If you try to fix the FFTW3 problem as described above, you'll get the following error:

-GMX_BUILD_OWN_FFTW=ON

CMake Error: Could not create named generator MX_BUILD_OWN_FFTW=ON

Make sure to put the "D" in:

-DGMX_BUILD_OWN_FFTW=ON

4. If You Don't Use DGMX_BUILD_OWN_FFTW=ON To Build FFTW3…

This is a skip-able section if you're letting cmake do the dirty work (and letting cmake do it is preferred, at least for getting GROMACS built). In trying sudo apt-get install fftw*, you see (currently) the following: fftw2 fftw-dev fftw-docs

No good. So, the procedure is to build FFTW3 from source (which is just as easy as installing from .deb or .rpm files if you installed everything I mentioned above). That said, your attempts to build FFTW3 and build GROMACS may have run into several errors because of how you built FFTW3. Beginning with your extracting and prep for make:

user@PROTOTYPE:~/Downloads$ tar xvf fftw-3.3.4.tar 
user@PROTOTYPE:~/Downloads$ cd fftw-3.3.4/

Any of the combinations below produce the same error:

user@PROTOTYPE:~/Downloads/fftw-3.3.4$ ./configure 
user@PROTOTYPE:~/Downloads/fftw-3.3.4$ ./configure -enable-shared=yes
user@PROTOTYPE:~/Downloads/fftw-3.3.4$ ./configure --enable-threads --enable-float

checking for a BSD-compatible install… /usr/bin/install -c
checking whether build environment is sane… yes
…
config.status: executing depfiles commands
config.status: executing libtool commands 

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=OFF
user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=OFF -DFFTWF_LIBRARY='/usr/local/lib/libfftw3.a'

— The C compiler identification is GNU 4.8.2
— The CXX compiler identification is GNU 4.8.2
— Check for working C compiler: /usr/bin/cc
…
— Performing Test PTHREAD_SETAFFINITY
— Performing Test PTHREAD_SETAFFINITY – Success
— Could NOT find Boost
Boost >= 1.44 not found. Using minimal internal version. This may cause trouble if you plan on compiling/linking other software that uses Boost against Gromacs.
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so – found
…
— checking for module 'fftw3f'
— package 'fftw3f' not found
— pkg-config could not detect fftw3f, trying generic detection
Could not find fftw3f library named libfftw3f, please specify its location in CMAKE_PREFIX_PATH or FFTWF_LIBRARY by hand (e.g. -DFFTWF_LIBRARY='/path/to/libfftw3f.so')
CMake Error at cmake/gmxManageFFTLibraries.cmake:76 (MESSAGE):
Cannot find FFTW 3 (with correct precision – libfftw3f for mixed-precision
GROMACS or libfftw3 for double-precision GROMACS). Either choose the right
precision, choose another FFT(W) library (-DGMX_FFT_LIBRARY), enable the
advanced option to let GROMACS build FFTW 3 for you
(-GMX_BUILD_OWN_FFTW=ON), or use the really slow GROMACS built-in fftpack
library (-DGMX_FFT_LIBRARY=fftpack).
Call Stack (most recent call first):
CMakeLists.txt:733 (include)

— Configuring incomplete, errors occurred!
See also "/home/user/Downloads/gromacs-5.0.1/CMakeFiles/CMakeOutput.log".
See also "/home/user/Downloads/gromacs-5.0.1/CMakeFiles/CMakeError.log".

Including –enable-shared takes care of this error and gets you to a successful GROMACS build.

user@PROTOTYPE:~/Downloads/fftw-3.3.4$ ./configure --enable-threads --enable-float --enable-shared

— The C compiler identification is GNU 4.8.2
— The CXX compiler identification is GNU 4.8.2
— Check for working C compiler: /usr/bin/cc
…
— Performing Test PTHREAD_SETAFFINITY
— Performing Test PTHREAD_SETAFFINITY – Success
— Could NOT find Boost
Boost >= 1.44 not found. Using minimal internal version. This may cause trouble if you plan on compiling/linking other software that uses Boost against Gromacs.
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so – found
…
— checking for module 'fftw3f'
— found fftw3f, version 3.3.4
— Looking for fftwf_plan_r2r_1d in /usr/local/lib/libfftw3f.so
— Looking for fftwf_plan_r2r_1d in /usr/local/lib/libfftw3f.so – found
— Looking for fftwf_have_simd_avx in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_simd_avx in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_simd_sse2 in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_simd_sse2 in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_simd_avx in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_simd_avx in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_simd_altivec in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_simd_altivec in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_simd_neon in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_simd_neon in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_sse2 in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_sse2 in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_sse in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_sse in /usr/local/lib/libfftw3f.so – not found
— Looking for fftwf_have_altivec in /usr/local/lib/libfftw3f.so
— Looking for fftwf_have_altivec in /usr/local/lib/libfftw3f.so – not found
CMake Warning at cmake/gmxManageFFTLibraries.cmake:89 (message):
The fftw library found is compiled without SIMD support, which makes it
slow. Consider recompiling it or contact your admin
Call Stack (most recent call first):
CMakeLists.txt:733 (include)

— Using external FFT library – FFTW3
— Looking for sgemm_
…
— Configuring done
— Generating done
— Build files have been written to: /home/user/Downloads/gromacs-5.0.1

And out of a first-pass GROMACS build…

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=OFF

Scanning dependencies of target libgromacs
[0%] Building C object src/gromacs/CMakeFiles/libgromacs.dir/__/external/tng_io/src/compression/bwlzh.c.o
[0%] Building C object src/gromacs/CMakeFiles/libgromacs.dir/__/external/tng_io/src/compression/bwt.c.o
…
[100%] Building CXX object src/programs/CMakeFiles/gmx.dir/legacymodules.cpp.o
Linking CXX executable https://www.somewhereville.com/bin/gmx
[100%] Built target gmx

5. But You Let cmake Build FFTW3. So, Continuing The Build Process

With all of the dependencies above installed, the one note I wanted to address was that for Boost:

…
— Performing Test PTHREAD_SETAFFINITY – Success
— Could NOT find Boost
Boost >= 1.44 not found. Using minimal internal version. This may cause trouble if you plan on compiling/linking other software that uses Boost against Gromacs.
— Looking for zlibVersion in /usr/lib/x86_64-linux-gnu/libz.so
…

It certainly isn't a major issue, but I wanted to try to get an warning-free build. Installing Boost 1.56 produced the following negative result:

user@PROTOTYPE:~/Downloads/boost_1_56_0$ ./bootstrap.sh

Building Boost.Build engine with toolset gcc… tools/build/src/engine/bin.linuxx86_64/b2
Detecting Python version… 2.7
Detecting Python root… /usr
Unicode/ICU support for Boost.Regex?… not found.
Generating Boost.Build configuration in project-config.jam…

Bootstrapping is done. To build, run:

./b2

To adjust configuration, edit 'project-config.jam'.
Further information:

– Command line help:
./b2 –help

– Getting started guide:
http://www.boost.org/more/getting_started/unix-variants.html

– Boost.Build documentation:
http://www.boost.org/boost-build2/doc/html/index.html 

user@PROTOTYPE:~/Downloads/boost_1_56_0$ sudo ./b2 install

Performing configuration checks

– 32-bit : no (cached)
– 64-bit : yes (cached)
– arm : no (cached)
…
…failed updating 58 targets…
…skipped 12 targets…
…updated 11322 targets… 

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=ON -DGMX_DOUBLE=OFF
user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ make

[0%] Building NVCC (Device) object src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_copyrite_gpu.cu.o
[0%] Building NVCC (Device) object src/gromacs/gmxlib/cuda_tools/CMakeFiles/cuda_tools.dir//./cuda_tools_generated_pmalloc_cuda.cu.o
…
[7%] Building CXX object src/gromacs/CMakeFiles/libgromacs.dir/commandline/cmdlinehelpwriter.cpp.o
In file included from /home/user/Downloads/gromacs-5.0.1/src/gromacs/options/basicoptions.h:52:0,
from /home/user/Downloads/gromacs-5.0.1/src/gromacs/commandline/cmdlinehelpwriter.cpp:55:
/home/user/Downloads/gromacs-5.0.1/src/gromacs/options/../utility/gmxassert.h:47:57: fatal error: boost/exception/detail/attribute_noreturn.hpp: No such file or directory
#include
^
compilation terminated.
make[2]: *** [src/gromacs/CMakeFiles/libgromacs.dir/commandline/cmdlinehelpwriter.cpp.o] Error 1
make[1]: *** [src/gromacs/CMakeFiles/libgromacs.dir/all] Error 2
make: *** [all] Error 2

Sadly, the solution is to then include -DGMX_EXTERNAL_BOOST=off and stick with the internal boost, which then "makes" just fine. One page references the use of -DGMX_INTERNAL_BOOST=on, but that produced the following:

CMake Warning:
Manually-specified variables were not used by the project:

GMX_INTERNAL_BOOST

— Build files have been written to: /home/user/Downloads/gromacs-5.0.1

There's more on this issue at: gerrit.gromacs.org/#/c/1232/ and t24960.science-biology-gromacs-development.biotalk.us/compiling-boost-problem-and-error-with-icc-t24960.html, but I've opted not to worry about it.

So, with Boost installed, I simply ignore it (and have not installed Boost on my RealBox).

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=ON -DGMX_EXTERNAL_BOOST=off

6. Finishing Step If All Above Goes Well: CUDA-Based GROMACS Build

If everything else above has gone smoothly (and if you ignored the Boost install. If you didn't, remember to add -DGMX_EXTERNAL_BOOST=off to the cmake below), you should be able to cleanly run a cmake for a GPU version of GROMACS (below, with the final result to be placed into /opt/gromacs_gpu. You then specify the $PATH after and run with it).

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ cmake -DGMX_GPU=ON -DCMAKE_INSTALL_PREFIX=/opt/gromacs_gpu -DGMX_BUILD_OWN_FFTW=ON

— The C compiler identification is GNU 4.8.2
— The CXX compiler identification is GNU 4.8.2
…
— Generating done
— Build files have been written to: /home/damianallis/Downloads/gromacs-5.0.1 

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ make

The make starts with the FFTW3 download and build…

Scanning dependencies of target fftwBuild
[ 0%] Performing pre-download step for 'fftwBuild'
— downloading…
src='http://www.fftw.org/fftw-3.3.3.tar.gz'
dest='/home/damianallis/Downloads/gromacs-5.0.1/src/contrib/fftw/fftw.tar.gz'
— [download 0% complete]
…
[100%] Building CXX object src/programs/CMakeFiles/gmx.dir/legacymodules.cpp.o
Linking CXX executable https://www.somewhereville.com/bin/gmx
[100%] Built target gmx

Finally, your (sudo) make install places everything into /opt/gromacs_gpu.

user@PROTOTYPE:~/Downloads/gromacs-5.0.1$ sudo make install

— The GROMACS-managed build of FFTW 3 will configure with the following optimizations: –enable-sse2
— Configuring done
— Generating done
— Build files have been written to: /home/damianallis/Downloads/gromacs-5.0.1
[1%] Built target fftwBuild
…
[100%] Building CXX object src/programs/CMakeFiles/gmx.dir/legacymodules.cpp.o
Linking CXX executable https://www.somewhereville.com/bin/gmx
[100%] Built target gmx

Posted on

Compiling And Running GAMESS-US (1 May 2013(R1)) On 64-bit Ubuntu 12.X/13.X In SMP Mode

Author's Note 1: It is my standard policy to put too much info into guides so that those who are searching for specific problems they come across will find the offending text in their searches. With luck, your "build error" search sent you here.

Author's Note 2: It's not as bad as it looks (I've included lots of output and error messages for easy searching)!

Author's Note 3: I won't be much help for you in diagnosing your errors, but am happy to tweak the text below if something is unclear.

Conventions: I include both the commands you type in your Terminal and some of the output from these commands, the output being where most of the errors appear that I work on in the discussion.

Input is formatted as below:

username – your username (check your prompt)
machinename – your hostname (type hostname or check your prompt)

Text you put in at the (also shown, so you see the directory structure) prompt (copy + paste should be fine)

Text you get out (for checking results and reproducing errors)

Having just recently downloaded the newest version of GAMESS-US (R1 2013), my first few passes at using it under Linux (specifically, Ubuntu 12.04) ran into a few walls that required some straightforward modifications and a little bit of system prep planning. As my first few passes before successful execution are likely the same exact problems you might have run into in your attempts to get GAMESS-US to run (after a successful compilation and linking), I'm posting my problems and solutions here.

Qualifier 1 – My concern at the moment has been to get GAMESS-US to run under 64-bit Ubuntu 12.04 on a multi-core board (ye olde symmetric multiprocessing (which I always called single multi-processor, or SMP)). While some answers may follow in what's below, this post doesn't cover MPI-specific builds (nothing through a router, that is). SMP is the only concern (which is to say, I likely won't have good answers if you send along an MPI-specific question). Also, although I'm VERY interested in trying it, I've not yet attempted to build a GPU-capable version (but plan to in the near future).

Qualifier 2 – It is my standard policy to install apps into /opt, and my steps below will reflect that (specifically because there's a permission issue that needs to be addressed when you first try to build components). You can default to whatever you like, but keep in mind my tweaks when you try to build your local copy.

So, with the qualifiers in mind…

1. Prepping The System (apt-get)

There are few things better than being able to apt-get everything you need to prep your machine for an install, and I'm pleased to report that the (current) process for putting the important files onto Ubuntu 12.X/13.X is easy. Assuming you're not going the Intel / PGI / MKL route, you can do everything by installing gfortran (compiler, presently installing 4.4) and the blas and atlas math libraries.

username@machinename:~$ sudo apt-get install gfortran libblas-dev libatlas-base-dev

Note: your atlas libraries will be installed in /usr/lib64/atlas/ – this will matter when you run config.

After these finish, run the following to determine your installed gfortran version (will be asked for by the new GAMESS config)

username@machinename:~$ gfortran -dumpversion

GNU Fortran (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3
Copyright (C) 2010 Free Software Foundation, Inc.

GNU Fortran comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of GNU Fortran
under the terms of the GNU General Public License.
For more information about these matters, see the file named COPYING

4.4 And you're ready for GAMESS.

2. Downloading GAMESS-US, Placing Into /opt, And Changing Permissions

First, obviously, get the GAMESS source (click on the red text).

After downloading, copy/move gamess-current.tar.gz into /opt

username@machinename:~$ cd ~/Downloads
username@machinename:~/Downloads$ sudo cp gamess-current.tar.gz /opt
username@machinename:~/Downloads$ cd /opt
username@machinename:/opt$ sudo gunzip gamess-cuerent.tar.gz
username@machinename:/opt$ sudo tar xvd gamess-current.tar

gamess/
gamess/gms-files.csh
gamess/tools/
…
gamess/misc/count.code
gamess/misc/vbdum.src
gamess/Makefile.in

At this point, if you go through the config process and get to the point of building ddikick.x, you will get an error when you first try to run ./compddi

username@machinename:/opt/gamess/ddi$ sudo ./compddi >& compddi.log &

[1] 4622
-bash: compddi.log: Permission denied

The problem is with the permission of the entire gamess folder:

drwxr-xr-x  4 root        root              4096 2014-04-04 21:43 .
drwxr-xr-x 22 root        root              4096 2013-12-27 16:17 ..
drwxr-xr-x 14 1300 504              4096 2014-04-04 21:43 gamess
-rw-r–r– 1 root        root         198481920 2014-04-04 21:42 gamess-current.tar

Which you remedy before running into this error by changing the permissions:

username@machinename:/opt$ sudo chown -R username gamess

The next step is recommended when you run config, so I'm performing the step here to get it out of the way. With the atlas libraries installed, generate two symbolic links.

username@machinename:/opt$ cd /usr/lib64/atlas
username@machinename:/usr/lib64/atlas$ sudo ln -s libf77blas.so.3.0 libf77blas.so
username@machinename:/usr/lib64/atlas$ sudo ln -s libatlas.so.3.0 libatlas.so

And, at this point, you're ready to run the new (well, new to me) config script that preps your system install.

3. Building GAMESS-US

Back to the GAMESS-US folder.

username@machinename:/usr/lib64/atlas$ cd /opt/gamess
username@machinename:/opt/gamess$ sudo ./config

This script asks a few questions, depending on your computer system,
to set up compiler names, libraries, message passing libraries,
and so forth.

You can quit at any time by pressing control-C, and then .

Please open a second window by logging into your target machine,
in case this script asks you to 'type' a command to learn something
about your system software situation. All such extra questions will
use the word 'type' to indicate it is a command for the other window.

After the new window is open, please hit to go on.

You can open that second window or blindly assume that what I include below is all you need.

[enter]

GAMESS can compile on the following 32 bit or 64 bit machines:
axp64 – Alpha chip, native compiler, running Tru64 or Linux
cray-xt – Cray's massively parallel system, running CNL
hpux32 – HP PA-RISC chips (old models only), running HP-UX
hpux64 – HP Intel or PA-RISC chips, running HP-UX
ibm32 – IBM (old models only), running AIX
ibm64 – IBM, Power3 chip or newer, running AIX or Linux
ibm64-sp – IBM SP parallel system, running AIX
ibm-bg – IBM Blue Gene (P or L model), these are 32 bit systems
linux32 – Linux (any 32 bit distribution), for x86 (old systems only)
linux64 – Linux (any 64 bit distribution), for x86_64 or ia64 chips
AMD/Intel chip Linux machines are sold by many companies
mac32 – Apple Mac, any chip, running OS X 10.4 or older
mac64 – Apple Mac, any chip, running OS X 10.5 or newer
sgi32 – Silicon Graphics Inc., MIPS chip only, running Irix
sgi64 – Silicon Graphics Inc., MIPS chip only, running Irix
sun32 – Sun ultraSPARC chips (old models only), running Solaris
sun64 – Sun ultraSPARC or Opteron chips, running Solaris
win32 – Windows 32-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition)
win64 – Windows 64-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition)
winazure – Windows Azure Cloud Platform running Windows 64-bit
type 'uname -a' to partially clarify your computer's flavor.
please enter your target machine name:

We're doing a linux64 build, so type the following at the prompt:

linux64

Where is the GAMESS software on your system?
A typical response might be /u1/mike/gamess,
most probably the correct answer is /opt/gamess

GAMESS directory? [/opt/gamess]

Who is this mike and where is my folder u1? We'll get to that in rungms. For now, I'm installing in /opt, so the default directory is fine:

[enter]

Setting up GAMESS compile and link for GMS_TARGET=linux64
GAMESS software is located at GMS_PATH=/opt/gamess

Please provide the name of the build locaation.
This may be the same location as the GAMESS directory.

GAMESS build directory? [/opt/gamess]

Fine as selected.

[enter]

Please provide a version number for the GAMESS executable.
This will be used as the middle part of the binary's name,
for example: gamess.00.x

Version? [00]

Is this important? Maybe, if you plan on building multiple versions of GAMESS-US (you might want a GPU-friendly version, one with a different compiler, one with MPI, etc.). Number as you wish and remember the number when it comes to rungms. That said, the actual linking step seems to really want to produce a 01 version (we'll get to that). Meantime, default value is fine.

[enter]

Linux offers many choices for FORTRAN compilers, including the GNU
compiler set ('g77' in old versions of Linux, or 'gfortran' in
current versions), which are included for free in Unix distributions.

There are also commercial compilers, namely Intel's 'ifort',
Portland Group's 'pgfortran', and Pathscale's 'pathf90'. The last
two are not common, and aren't as well tested as the others.

type 'rpm -aq | grep gcc' to check on all GNU compilers, including gcc
type 'which gfortran' to look for GNU's gfortran (a very good choice),
type 'which g77' to look for GNU's g77,
type 'which ifort' to look for Intel's compiler,
type 'which pgfortran' to look for Portland Group's compiler,
type 'which pathf90' to look for Pathscale's compiler.
Please enter your choice of FORTRAN:

We're using gfortran (currently 4.4.3):

gfortran

gfortran is very robust, so this is a wise choice.

Please type 'gfortran -dumpversion' or else 'gfortran -v' to
detect the version number of your gfortran.
This reply should be a string with at least two decimal points,
such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12.
The reply may be labeled as a 'gcc' version,
but it is really your gfortran version.
Please enter only the first decimal place, such as 4.1 or 4.6: 

4.4

Alas, your version of gfortran does not support REAL*16,
so relativistic integrals cannot use quadruple precision.
Other than this, everything will work properly.
hit to continue to the math library setup.

If this was my biggest concern I'd be a happy quantum chemist. Obviously you can try to install other flavors of gfortran and, possibly, by the time you need the procedure I'm following, a newer version of gfortran will be apt-gotten.

[enter]

Linux distributions do not include a standard math library.

There are several reasonable add-on library choices,
MKL from Intel for 32 or 64 bit Linux (very fast)
ACML from AMD for 32 or 64 bit Linux (free)
ATLAS from www.rpmfind.net for 32 or 64 bit Linux (free)
and one very unreasonable option, namely 'none', which will use
some slow FORTRAN routines supplied with GAMESS. Choosing 'none'
will run MP2 jobs 2x slower, or CCSD(T) jobs 5x slower.

Some typical places (but not the only ones) to find math libraries are
Type 'ls /opt/intel/mkl' to look for MKL
Type 'ls /opt/intel/Compiler/mkl' to look for MKL
Type 'ls /opt/intel/composerxe/mkl' to look for MKL
Type 'ls -d /opt/acml*' to look for ACML
Type 'ls -d /usr/local/acml*' to look for ACML
Type 'ls /usr/lib64/atlas' to look for Atlas

Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none': 

atlas

Where is your Atlas math library installed? A likely place is
/usr/lib64/atlas
Please enter the Atlas subdirectory on your system:

Our location is, in fact, /usr/lib64/atlas, so we type it in accordingly.

NOTE: If you don't type anything but [enter] below, the script closes (/usr/lib64/atlas is listed as the expected location, but it is not defaulted by the script. You need to type it in.

/usr/lib64/atlas

The linking step in GAMESS assumes that a softlink exists
within the system's /usr/lib64/atlas
from libatlas.so to a specific file like libatlas.so.3.0
from libf77blas.so to a specific file like libf77blas.so.3.0
config can carry on for the moment, but the 'root' user should
chdir /usr/lib64/atlas
ln -s libf77blas.so.3.0 libf77blas.so
ln -s libatlas.so.3.0 libatlas.so
prior to the linking of GAMESS to a binary executable.

Math library 'atlas' will be taken from /usr/lib64/atlas

please hit to compile the GAMESS source code activator

The symbolic linking was performed before the GAMESS steps.

[enter]

gfortran -o /home/username/gamess/tools/actvte.x actvte.f
unset echo
Source code activator was successfully compiled.

please hit to set up your network for Linux clusters.

[enter]

If you have a slow network, like Gigabit Ethernet (GE), or
if you have so few nodes you won't run extensively in parallel, or
if you have no MPI library installed, or
if you want a fail-safe compile/link and easy execution,
choose 'sockets'
to use good old reliable standard TCP/IP networking.

If you have an expensive but fast network like Infiniband (IB), and
if you have an MPI library correctly installed,
choose 'mpi'.

communication library ('sockets' or 'mpi')?

Again, I'm not building an mpi-friendly version, so am using sockets.

sockets

64 bit Linux builds can attach a special LIBCCHEM code for fast
MP2 and CCSD(T) runs. The LIBCCHEM code can utilize nVIDIA GPUs,
through the CUDA libraries, if GPUs are available.
Usage of LIBCCHEM requires installation of HDF5 I/O software as well.
GAMESS+LIBCCHEM binaries are unable to run most of GAMESS computations,
and are a bit harder to create due to the additional CUDA/HDF5 software.
Therefore, the first time you run 'config', the best answer is 'no'!
If you decide to try LIBCCHEM later, just run this 'config' again.

Do you want to try LIBCCHEM? (yes/no): 

no

Your configuration for GAMESS compilation is now in
/home/username/gamess/install.info
Now, please follow the directions in
/home/username/gamess/machines/readme.unix
username@machinename:~/gamess$

At this stage, you're ready to build ddikick.x and continue with the compiling.

4. Build ddikick.x

username@machinename:/opt/gamess$ cd ddi
username@machinename:/opt/gamess/ddi$ sudo ./compddi >& compddi.log &

Will dump output into compddi.log (which will now work with the correct permissions).

username@machinename:/opt/gamess/ddi$ sudo mv ddikick.x ..
username@machinename:/opt/gamess/ddi$ cd ..
username@machinename:/opt/gamess$ sudo ./compall >& compall.log &

Feel free to follow along as compall.log dumps results. You're also welcome to follow the readme.unix advice:

This takes a while, so go for coffee, or check the SF Giants web page.

Upon completion, the last step is to link the executable.

Now, it used to be the case that you specified the version number in the lked step. So, if you wanted to stick with the 00 version from the config file, you'd type 

username@machinename:/opt/gamess$ sudo ./lked gamess 00 >& lked.log &

When you do that at present, you get

[1] 7626
username@machinename:/opt/gamess$

[1]+ Stopped sudo ./lked gamess 00 &>lked.log

This then leads you to use the lked call from the readme.unix file.

username@machinename:/opt/gamess$ sudo ./lked gamess 01 >& lked.log &

Which then produces lked.log and gamess.01.x.

Now, if you run with 00 again, you get a successful linking of gamess.00.x . Not sure why this happens, but the version number isn't important so long as you specify the right one when you use rungms (so I've not diagnosed it further).

At this point, you have a gamess.00.x and/or gamess.01.x executable in your /opt/gamess folder:

30828747 2014-04-04 22:41 gamess.01.x

I'm going to ignore the 00 issue out of the config file and use the gamess.01.x executable.

We're ready to run calculations and work through the next set of errors you'll receive if you don't properly modify files.

5. PATH Setting

First, we copy rungms to our home folder, then add /opt/gamess to the PATH: 

username@machinename:/opt/gamess$ cp rungms ~/
username@machinename:/opt/gamess$ cd ~/
username@machinename:~$ nano .bashrc

Add the following to the bottom of .bashrc (or extend your PATH)

PATH=$PATH:/opt/gamess

Quit nano and source.

username@machinename:~$ source .bashrc
[OPTIONAL] username@machinename:~$ echo $PATH

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:…/opt/gamess:

6. rungms (Probably Why You're Here)

If you just go blindly into a run, you'll get the following error:

username@machinename:~$ ./rungms test.inp

—– GAMESS execution script 'rungms' —–
This job is running on host machinename
under operating system Linux at Fri Apr 4 22:47:55 EDT 2014
Available scratch disk space (Kbyte units) at beginning of the job is
df: `/scr/username': No such file or directory
df: no file systems processed
GAMESS temporary binary files will be written to /scr/username
GAMESS supplementary output files will be written to /home/username/scr
Copying input file test.inp to your run's scratch directory…
cp test.inp /scr/username/test.F05
cp: cannot create regular file `/scr/username/test.F05': No such file or directory
unset echo
/u1/mike/gamess/gms-files.csh: No such file or directory.

As is obvious, rungms needs some modifying. 

username@machinename:~$ nano rungms

Scroll down until you see the following:

set TARGET=sockets
set SCR=/scr/$USER
set USERSCR=~$USER/scr
set GMSPATH=/u1/mike/gamess

Given that it's just me on the machine, I tend to simplify this by making SCR and USERSCR the same directory, and I make them both /tmp. If you intend on keeping all of the files, you'll need to make rungms specific for each run case. My only concerns are .dat and .log, so /tmp dumping is fine. Furthermore, we must change GMSPATH from how the ever-helpful Mike Schmidt (he got me through some early issues when I started my GAMESS-US adventure 15ish years ago. Won't complain about his continued default-ed presence in the scripts) has it set up at Iowa to how we want it on our own machines (in my case, /opt/gamess)

set TARGET=sockets
set SCR=/tmp
set USERSCR=/tmp
set GMSPATH=/opt/gamess

With these modifications, your next run will be a bit more successful:

username@machinename:~$ ./rungms test.inp

—– GAMESS execution script 'rungms' —–
This job is running on host machinename
under operating system Linux at Fri Apr 4 22:51:35 EDT 2014
Available scratch disk space (Kbyte units) at beginning of the job is
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 1905222596 249225412 1559217460 14% /
GAMESS temporary binary files will be written to /tmp
GAMESS supplementary output files will be written to /tmp
Copying input file test.inp to your run's scratch directory…
cp test.inp /tmp/test.F05
unset echo
/opt/gamess/ddikick.x /opt/gamess/gamess.00.x test -ddi 1 1 machinename -scr /tmp

Distributed Data Interface kickoff program.
Initiating 1 compute processes on 1 nodes to run the following command:
/opt/gamess/gamess.00.x test

******************************************************
* GAMESS VERSION = 1 MAY 2013 (R1) *
* FROM IOWA STATE UNIVERSITY *
* M.W.SCHMIDT, K.K.BALDRIDGE, J.A.BOATZ, S.T.ELBERT, *
* M.S.GORDON, J.H.JENSEN, S.KOSEKI, N.MATSUNAGA, *
* K.A.NGUYEN, S.J.SU, T.L.WINDUS, *
* TOGETHER WITH M.DUPUIS, J.A.MONTGOMERY *
* J.COMPUT.CHEM. 14, 1347-1363(1993) *
**************** 64 BIT LINUX VERSION ****************
…
INPUT CARD>
DDI Process 0: shmget returned an error.
Error EINVAL: Attempting to create 160525768 bytes of shared memory.
Check system limits on the size of SysV shared memory segments.

The file ~/gamess/ddi/readme.ddi contains information on how to display
the current SystemV memory settings, and how to increase their sizes.
Increasing the setting requires the root password, and usually a sytem reboot.

DDI Process 0: error code 911
ddikick.x: application process 0 quit unexpectedly.
ddikick.x: Fatal error detected.
The error is most likely to be in the application, so check for
input errors, disk space, memory needs, application bugs, etc.
ddikick.x will now clean up all processes, and exit…
ddikick.x: Sending kill signal to DDI processes.
ddikick.x: Execution terminated due to error(s).
unset echo
—– accounting info —–
Files used on the master node machinename were:
-rw-r–r– 1 username username 0 2014-04-04 22:51 /tmp/test.dat
-rw-r–r– 1 username username 1341 2014-04-04 22:51 /tmp/test.F05
ls: No match.
ls: No match.
ls: No match.
Fri Apr 4 22:51:36 EDT 2014
0.0u 0.0s 0:01.08 9.2% 0+0k 0+8io 0pf+0w

Things worked, but with a memory error. This issue is discussed at the Baldridge Group wiki: ocikbapps.uzh.ch/kbwiki/gamess_troubleshooting.html

From the wiki:

If you are sure you are not asking for too much memory in the input file, check that your kernel parameters are not allowing enough memory to be requested. You might have to increase the SHMALL & SHMAX kernel memory values to allow GAMESS to run. (See http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/ for a better explanation.)
For example, on a machine with 4GB of memory, you might add these to /etc/sysctl.conf:
# cat /etc/sysctl.conf | grep shm
kernel.shmmax = 3064372224
kernel.shmall = 748137
Then set the new settings like so:
# sysctl -p
Since they are in /etc/sysctl.conf, they will automatically be set each time the system is booted.

In our case, we modify sysctl.conf with the recommendations from the wiki:

username@machinename:~$ sudo nano /etc/sysctl.conf

Add the following to the bottom of the file:

kernel.shmmax = 3064372224
kernel.shmall = 748137

Save and exit.

username@machinename:~$ sudo sysctl -p

net.ipv4.ip_forward = 1
kernel.shmmax = 3064372224
kernel.shmall = 748137

These memory values will change depending on your system.

Now we empty the /tmp and rerun.

username@machinename:~$ rm /tmp/*
username@machinename:~$ ./rungms test.inp

If your input file is worth it's salt, you'll have successfully run your file on a single processor (single core, that is). If you run into additional memory errors, increase kernel.shmmax and kernel.shmall.

Now, onto the SMP part. My first attempt to run games in parallel (on 4 cores using version 00) produced the following error:

username@machinename:~$ rm /tmp/*
username@machinename:~$ ./rungms test.inp 00 4

—– GAMESS execution script 'rungms' —–
This job is running on host machinename
under operating system Linux at Fri Apr 4 22:52:52 EDT 2014
Available scratch disk space (Kbyte units) at beginning of the job is
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda2 1905222596 249225416 1559217456 14% /
GAMESS temporary binary files will be written to /tmp
GAMESS supplementary output files will be written to /tmp
Copying input file test.inp to your run's scratch directory…
cp test.inp /tmp/test.F05
unset echo
I do not know how to run this node in parallel.

I tried a number of stupid things to get the run to work, finally settling on modifying the rungms file properly. To make gamess know how to run the node in parallel, we need only make the following changes to our rungms file. 

username@machinename:~$ nano rungms

Scroll down until you find the section below:

# 2. This is an example of how to run on a multi-core SMP enclosure,
# where all CPUs (aka COREs) are inside a -single- NODE.
# At other locations, you may wish to consider some of the examples
# that follow below, after commenting out this ISU specific part.
if ($NCPUS > 1) then
switch (`hostname`)
case se.msg.chem.iastate.edu:
case sb.msg.chem.iastate.edu:
if ($NCPUS > 2) set NCPUS=4
set NNODES=1

The change is simple. We remove the cases for $NCPUS > 1 in the file and add the hostname of our linux box (and if you don't know this or it's not in your prompt, simply type hostname at the prompt first). We'll disable the two cases listed and add our hostname to the case list.

# 2. This is an example of how to run on a multi-core SMP enclosure,
# where all CPUs (aka COREs) are inside a -single- NODE.
# At other locations, you may wish to consider some of the examples
# that follow below, after commenting out this ISU specific part.
if ($NCPUS > 1) then
switch (`hostname`)
case machinename:
# case se.msg.chem.iastate.edu:
# case sb.msg.chem.iastate.edu:
if ($NCPUS > 2) set NCPUS=4
set NNODES=1

This gives you parallel functionality, but it's still not using the machine resources (cores) correctly when I ask for anything more than 2 cores (always using only 2 cores).

[minor complaint]
Admittedly, I don't immediately get the logic of this section as currently coded, as one cannot get more than 2 cores to work in this case given how the if statements are written (so far as I can see now. I will assume I am the one missing something but have not decided to ask about it, instead changing the rungms text to the following). You can check this yourself by running top in another window. This is the most simple modification, and assumes you want to run N number of cores each time. Clearly, you can make this more elegant than it is (my modification, that is). Meantime, I want to run 4 cores on this machine, so I change the section to reflect a 4-core board (and commented out much of this section).
[/complaint]

# 2. This is an example of how to run on a multi-core SMP enclosure,
# where all CPUs (aka COREs) are inside a -single- NODE.
# At other locations, you may wish to consider some of the examples
# that follow below, after commenting out this ISU specific part.
if ($NCPUS > 1) then
switch (`hostname`)
case machinename
# case se.msg.chem.iastate.edu:
# case sb.msg.chem.iastate.edu:
# if ($NCPUS > 2) set NCPUS=2
# set NNODES=1
# set HOSTLIST=(`hostname`:cpus=$NCPUS)
# breaksw
# case machinename
# case br.msg.chem.iastate.edu:
if ($NCPUS >= 4) set NCPUS=4
set NNODES=1
set HOSTLIST=(`hostname`:cpus=$NCPUS)
breaksw
case machinename
# case cd.msg.chem.iastate.edu:
# case zn.msg.chem.iastate.edu:
# case ni.msg.chem.iastate.edu:
# case co.msg.chem.iastate.edu:
# case pb.msg.chem.iastate.edu:
# case bi.msg.chem.iastate.edu:
# case po.msg.chem.iastate.edu:
# case at.msg.chem.iastate.edu:
# case sc.msg.chem.iastate.edu:
# if ($NCPUS > 4) set NCPUS=4
# set NNODES=1
# set HOSTLIST=(`hostname`:cpus=$NCPUS)
# breaksw
# case ga.msg.chem.iastate.edu:
# case ge.msg.chem.iastate.edu:
# case gd.msg.chem.iastate.edu:
# if ($NCPUS > 6) set NCPUS=6
# set NNODES=1
# set HOSTLIST=(`hostname`:cpus=$NCPUS)
# breaksw
default:
echo I do not know how to run this node in parallel.
exit 20
endsw
endif
#

And, with this set of changes, I'm using all 4 cores on the board (but have some significant memory issues when running MP2 calks. But that's for another post).

The typical user will never be able to do what the GAMESS group has done in making an excellent program that also happens to be free. That said, the need to make changes to the rungms file is something that would be greatly simplified by having N number of rungms scripts for each case instead of a monolithic file that is mostly useless text to users not using one of the system types. This, for instance, would make rungms modification much easier. If I streamline rungms for my specific system, I may post a new file accordingly.