home

Archive for the 'linux issues' Category

Generating Molecular Orbitals (And Visualizing Assorted Properties) With The Gaussian09 cubegen Utility

Saturday, June 7th, 2014

To begin, this post owes its existence to the efforts of Dr. Douglas Fox at Gaussian, Inc., who provided me with an alternative explanation of how the cubegen utility works. After much wailing and gnashing of teeth, I intend on taking Dr. Fox’s advice and asking Gaussian Support for assistance earlier in my endeavors. What follows below, I hope, will save you some significant frustration (and, given how little there is online that really describes the extra workings of cubegen in a clear and example’ed way, it is my expectation that this page appeared early in your search list).

What I wanted out of cubegen that I couldn’t figure out how to get:

The situation was simple. I wanted my molecule centered and bound within an arbitrarily-sized box (X,Z,Y) for making images and doing additional post-processing. Specifically, I wanted to be able to take many different molecules (from hydrogen gas to big biomolecules) defined within the same-sized box for layering and presentation (different boxes for each, but all the same size).

I am assuming for this that you’re using cubegen from a terminal (not within GaussView or the like) to produce .cub/.cube files for use in some kind of rendering-capable program (like VESTA or VMD) and that cubegen and formchk are in your PATH (either properly placed or by running the Gaussian install script). I’ll be demonstrating usage with benzene (C6H6) and the benzene cation (C6H6+).

1. The Checkpoint File

To extract any kind of data for making .cub/.cube files, you need a checkpoint file (.chk) from your run. This is performed by adding a %chk=FILENAME.chk line to the top of the input file (which, if you’re a Gaussian user, you likely already know). If you want additional properties cube’d, check the Gaussian Tech Document, specifically looking at the Pop keyword for most of the properties you’d want visualized (this data gets placed into the .chk file for .cub/.cube generation after the run). For the standard molecular orbitals, they’re already saved in the .chk file (or their coefficients, anyway).

For benzene.gjf:

%chk=benzene.chk
# b3lyp/6-31G(d,p)

Benzene

0 1
 C                  1.20809735    0.69749533   -0.00000000
 C                  0.00000000    1.39499067   -0.00000000
 C                 -1.20809735    0.69749533   -0.00000000
 C                 -1.20809735   -0.69749533   -0.00000000
 C                  0.00000000   -1.39499067   -0.00000000
 C                  1.20809735   -0.69749533   -0.00000000
 H                  2.16038781    1.24730049   -0.00000000
 H                  0.00000000    2.49460097   -0.00000000
 H                 -2.16038781    1.24730049   -0.00000000
 H                 -2.16038781   -1.24730049   -0.00000000
 H                  0.00000000   -2.49460097   -0.00000000
 H                  2.16038781   -1.24730049   -0.00000000

For benzenecation.gjf:

%chk=benzenecation.chk
# b3lyp/6-31G(d,p)

Benzene cation

1 2
 C                  1.20809735    0.69749533   -0.00000000
 C                  0.00000000    1.39499067   -0.00000000
 C                 -1.20809735    0.69749533   -0.00000000
 C                 -1.20809735   -0.69749533   -0.00000000
 C                  0.00000000   -1.39499067   -0.00000000
 C                  1.20809735   -0.69749533   -0.00000000
 H                  2.16038781    1.24730049   -0.00000000
 H                  0.00000000    2.49460097   -0.00000000
 H                 -2.16038781    1.24730049   -0.00000000
 H                 -2.16038781   -1.24730049   -0.00000000
 H                  0.00000000   -2.49460097   -0.00000000
 H                  2.16038781   -1.24730049   -0.00000000

2. Convert The .chk To .fchk With formchk

As per the Gaussian Tech Doc:

formchk converts the data in a Gaussian checkpoint file into a formatted form which is suitable for input into a variety of visualization software.

Basically, making the .chk file something that cubegen can manipulate to generate .cub/.cube files of orbitals, densities, electrostatic potentials, etc. This run is simple for most users (for the rest, see formchk).

formchk benzene.chk benzene.fchk
formchk benzenecation.chk benzenecation.fchk

3. Using cubegen

And now the fun begins. A typical cubegen run looks like the following:

cubegen 0 MO=HOMO benzene.fchk benzene_HOMO.cub 0 h

cubegen – run cubegen
0 – an old memory flag (must be there, but not important)
MO=HOMO – generate the highest occupied molecular orbital
benzene.fchk – the .fchk file
benzene_HOMO.cub – the generated .cub file
0 – use the default grid point specification (80*80*80 points total in the whole cube file)
h – write out the .cub file with headers

The output you find summarized in VESTA is below for this case.

DEFAULT:
OpenGL version: 2.1 INTEL-8.26.34
Video configuration: Intel HD Graphics 4000 OpenGL Engine
Maximum supported width and height of the viewport: 16384 x 16384
OpenGL depth buffer bit: 16

/Users/damianallis/benzene_HOMO_default_0.cub
====================================================================================
Title Benzene MO=HOMO
Dimensions 87 91 65

Lattice parameters

a b c alpha beta gamma
9.39704 9.82909 7.02078 90.0000 90.0000 90.0000

Unit-cell volume = 648.469273 Å^3

Total number of polygons and unique vertices on slices;
(1 0 0): 0 ( 0), 0 ( 0)
(0 1 0): 0 ( 0), 0 ( 0)
(0 0 1): 0 ( 0), 0 ( 0)
====================================================================================

====================================================================================
Title Benzene

Lattice type P
Space group name P 1
Space group number 1
Setting number 1

Lattice parameters

a b c alpha beta gamma
1.00000 1.00000 1.00000 90.0000 90.0000 90.0000

Unit-cell volume = 1.000000 Å^3

Structure parameters

x y z Occ. B Site Sym.
1 C C1 4.65450 6.23638 3.44640 1.000 1.000 1 -
2 C C2 5.86259 5.53889 3.44640 1.000 1.000 1 -
3 C C3 5.86259 4.14389 3.44640 1.000 1.000 1 -
4 C C4 4.65450 3.44640 3.44640 1.000 1.000 1 -
5 C C5 3.44640 4.14389 3.44640 1.000 1.000 1 -
6 C C6 3.44640 5.53889 3.44640 1.000 1.000 1 -
7 H H1 4.65450 7.33599 3.44640 1.000 1.000 1 -
8 H H2 6.81488 6.08869 3.44640 1.000 1.000 1 -
9 H H3 6.81488 3.59409 3.44640 1.000 1.000 1 -
10 H H4 4.65450 2.34679 3.44640 1.000 1.000 1 -
11 H H5 2.49411 3.59409 3.44640 1.000 1.000 1 -
12 H H6 2.49411 6.08869 3.44640 1.000 1.000 1 -
====================================================================================

Number of polygons and unique vertices on isosurface = 16904 (8460)
12 atoms, 12 bonds, 0 polyhedra; CPU time = 39 ms

For the coarse grid (-2) case:

cubegen 0 MO=HOMO benzene.fchk benzene_HOMO_default_m2.cub -2 h

The output you find summarized in VESTA is below for this case.

/Users/damianallis/benzene_HOMO_default_m2.cub
====================================================================================
Title Benzene MO=HOMO
Dimensions 54 56 40

Lattice parameters

a b c alpha beta gamma
9.52518 9.87796 7.05569 90.0000 90.0000 90.0000

Unit-cell volume = 663.865482 Å^3

Total number of polygons and unique vertices on slices;
(1 0 0): 0 ( 0), 0 ( 0)
(0 1 0): 0 ( 0), 0 ( 0)
(0 0 1): 0 ( 0), 0 ( 0)
====================================================================================

====================================================================================
Title Benzene

Lattice type P
Space group name P 1
Space group number 1
Setting number 1

Lattice parameters

a b c alpha beta gamma
1.00000 1.00000 1.00000 90.0000 90.0000 90.0000

Unit-cell volume = 1.000000 Å^3

Structure parameters

x y z Occ. B Site Sym.
1 C C1 4.65450 6.23638 3.44640 1.000 1.000 1 -
2 C C2 5.86259 5.53889 3.44640 1.000 1.000 1 -
3 C C3 5.86259 4.14389 3.44640 1.000 1.000 1 -
4 C C4 4.65450 3.44640 3.44640 1.000 1.000 1 -
5 C C5 3.44640 4.14389 3.44640 1.000 1.000 1 -
6 C C6 3.44640 5.53889 3.44640 1.000 1.000 1 -
7 H H1 4.65450 7.33599 3.44640 1.000 1.000 1 -
8 H H2 6.81488 6.08869 3.44640 1.000 1.000 1 -
9 H H3 6.81488 3.59409 3.44640 1.000 1.000 1 -
10 H H4 4.65450 2.34679 3.44640 1.000 1.000 1 -
11 H H5 2.49411 3.59409 3.44640 1.000 1.000 1 -
12 H H6 2.49411 6.08869 3.44640 1.000 1.000 1 -
====================================================================================

Number of polygons and unique vertices on isosurface = 6516 (3266)
12 atoms, 12 bonds, 0 polyhedra; CPU time = 10 ms

For the medium grid (-3) case:

cubegen 0 MO=HOMO benzene.fchk benzene_HOMO_default_m3.cub -3 h

The output you find summarized in VESTA is below for this case.

/Users/damianallis/benzene_HOMO_default_m3.cub
====================================================================================
Title Benzene MO=HOMO
Dimensions 107 111 79

Lattice parameters

a b c alpha beta gamma
9.43701 9.78980 6.96751 90.0000 90.0000 90.0000

Unit-cell volume = 643.703858 Å^3

Total number of polygons and unique vertices on slices;
(1 0 0): 0 ( 0), 0 ( 0)
(0 1 0): 0 ( 0), 0 ( 0)
(0 0 1): 0 ( 0), 0 ( 0)
====================================================================================

====================================================================================
Title Benzene

Lattice type P
Space group name P 1
Space group number 1
Setting number 1

Lattice parameters

a b c alpha beta gamma
1.00000 1.00000 1.00000 90.0000 90.0000 90.0000

Unit-cell volume = 1.000000 Å^3

Structure parameters

x y z Occ. B Site Sym.
1 C C1 4.65450 6.23638 3.44640 1.000 1.000 1 -
2 C C2 5.86259 5.53889 3.44640 1.000 1.000 1 -
3 C C3 5.86259 4.14389 3.44640 1.000 1.000 1 -
4 C C4 4.65450 3.44640 3.44640 1.000 1.000 1 -
5 C C5 3.44640 4.14389 3.44640 1.000 1.000 1 -
6 C C6 3.44640 5.53889 3.44640 1.000 1.000 1 -
7 H H1 4.65450 7.33599 3.44640 1.000 1.000 1 -
8 H H2 6.81488 6.08869 3.44640 1.000 1.000 1 -
9 H H3 6.81488 3.59409 3.44640 1.000 1.000 1 -
10 H H4 4.65450 2.34679 3.44640 1.000 1.000 1 -
11 H H5 2.49411 3.59409 3.44640 1.000 1.000 1 -
12 H H6 2.49411 6.08869 3.44640 1.000 1.000 1 -
====================================================================================

Number of polygons and unique vertices on isosurface = 25532 (12774)
12 atoms, 12 bonds, 0 polyhedra; CPU time = 51 ms

For the fine grid (-4) case:

cubegen 0 MO=HOMO benzene.fchk benzene_HOMO_default_m4.cub -4 h

The output you find summarized in VESTA is below for this case.

/Users/damianallis/benzene_HOMO_default_m4.cub
====================================================================================
Title Benzene MO=HOMO
Dimensions 212 221 157

Lattice parameters

a b c alpha beta gamma
9.34876 9.74564 6.92337 90.0000 90.0000 90.0000

Unit-cell volume = 630.786281 Å^3

Total number of polygons and unique vertices on slices;
(1 0 0): 0 ( 0), 0 ( 0)
(0 1 0): 0 ( 0), 0 ( 0)
(0 0 1): 0 ( 0), 0 ( 0)
====================================================================================

====================================================================================
Title Benzene

Lattice type P
Space group name P 1
Space group number 1
Setting number 1

Lattice parameters

a b c alpha beta gamma
1.00000 1.00000 1.00000 90.0000 90.0000 90.0000

Unit-cell volume = 1.000000 Å^3

Structure parameters

x y z Occ. B Site Sym.
1 C C1 4.65450 6.23638 3.44640 1.000 1.000 1 -
2 C C2 5.86259 5.53889 3.44640 1.000 1.000 1 -
3 C C3 5.86259 4.14389 3.44640 1.000 1.000 1 -
4 C C4 4.65450 3.44640 3.44640 1.000 1.000 1 -
5 C C5 3.44640 4.14389 3.44640 1.000 1.000 1 -
6 C C6 3.44640 5.53889 3.44640 1.000 1.000 1 -
7 H H1 4.65450 7.33599 3.44640 1.000 1.000 1 -
8 H H2 6.81488 6.08869 3.44640 1.000 1.000 1 -
9 H H3 6.81488 3.59409 3.44640 1.000 1.000 1 -
10 H H4 4.65450 2.34679 3.44640 1.000 1.000 1 -
11 H H5 2.49411 3.59409 3.44640 1.000 1.000 1 -
12 H H6 2.49411 6.08869 3.44640 1.000 1.000 1 -
====================================================================================

Number of polygons and unique vertices on isosurface = 100680 (50348)
12 atoms, 12 bonds, 0 polyhedra; CPU time = 155 ms

These all generate a file containing the highest occupied molecular orbital (or one of the degenerate HOMO’s in this case. Do I have to qualify that this doesn’t mean what 99.5% of the people coming to this page thinks this means?). The box is generated by something in cubegen to be 9.3ish x 9.7ish x 6.9ish Angstroms on a side and containing X points per Angstrom (and you can change the fineness of the grid points). The image below shows the four cases for the benzene HOMO. Click to see larger versions if you want to see the influence of grid fineness on the final image.

benzene_homo_gaussian_defaults_small

Click for a larger view.

Now, then, while the boxes are almost all identical, the same molecule and input gives four slightly different results. Fine for individual images, but not ideal for the obsessive-compulsive image maker. Also, you see how a box simply bounds the molecule, meaning no standardization of size if you needed that standardization for some reason.

   a        b        c       alpha    beta     gamma
 9.39704  9.82909  7.02078  90.0000  90.0000  90.0000 < - default (0)
 9.52518  9.87796  7.05569  90.0000  90.0000  90.0000 <- coarse (-2)
 9.43701  9.78980  6.96751  90.0000  90.0000  90.0000 <- medium (-3)
 9.34876  9.74564  6.92337  90.0000  90.0000  90.0000 <- fine (-4)

So, for a specific case - suppose I wanted this orbital in a box exactly 15 x 20 x 25 Angstroms on a side with the molecule offset from the center by -1.0 Angstrom in each direction.

I was pleased to finally discover that cubegen allows for that, although you have to ask Gaussian Support to find out how (until now, that is) and you need to do a little bit of math to get the placement right (or use the excel file I've linked in a .zip file found at 2014june7_cubegen_excel_file.xlsx).

You begin with the following:

cubegen 0 MO=HOMO benzene.fchk benzene_HOMO.cub -1 h

But for -1, Where do the numbers go?

From the Gaussian Tech Doc:

A value of -1 says to read the cube specification from the input stream, according to the following format:

IFlag, X0, Y0, Z0 Output unit number and initial point.
N1, X1, Y1, Z1 Number of points and step-size in the X-direction.
N2, X2, Y2, Z2 Number of points and step-size in the Y-direction.
N3, X3, Y3, Z3 Number of points and step-size in the Z-direction.

IFlag is the output unit number. If IFlag is less than 0, then a formatted file will be produced; otherwise, an unformatted file will be written.

Admittedly, “input stream” made no sense to me upon first and second read. I just knew that the program didn’t do anything when I ran it. Now obvious, this means you input the cube specifications by typing in (or, better, pasting in) the 16 numbers it asks for.

Continuing…

The -1 tells cubegen to “expect more input.” In this case, without explanation first, my input would look as below:

-12  -6.50000  -9.00000 -11.50000
 60   0.25000   0.00000   0.00000
 80   0.00000   0.25000   0.00000
100   0.00000   0.00000   0.25000

Which you just paste into your terminal at the new line (having pressed ENTER after typing out the cubegen line above).

How this works (and note the use of minus signs!):

-[# Atoms] -[Start Point For Box In X] -[Start Point For Box In Y] -[Start Point For Box In Z]
[Number of Points In X]   [Grid Fineness In X]   [Grid Fineness In Y]   [Grid Fineness In Z]
[Number of Points In Y]   [Grid Fineness In X]   [Grid Fineness In Y]   [Grid Fineness In Z]
[Number of Points In Y]   [Grid Fineness In X]   [Grid Fineness In Y]   [Grid Fineness In Z]

Assuming orthogonality in your box, the off-diagonals for the grid fineness matrix are zero.

-[# Atoms] -[Start Point For Box In X] -[Start Point For Box In Y] -[Start Point For Box In Z]
[Number of Points In X]  [Grid Fineness In X]   0.000   0.000
[Number of Points In Y]  0.000   [Grid Fineness In Y]   0.000
[Number of Points In Y]  0.000   0.000   [Grid Fineness In Z]

4. -6.5, -9, -11.5?

You build the box around your molecule in cubegen, which means you combine (1) where you want the molecule positions with (2) the number of grid points along each direction and (3) the fineness of the grid to generate the box. Here, I’m starting my hypothetical box at -6.5 in X, -9 in Y, and -11.5 in Z, then building out my molecule 121*.25 points in X, 161*.25 in Y, 201*.25 in Z. This will produce the intended box size with the molecule technically centered at the origin in the box (0,0,0), but the generation of all 121, 161, and 201 points in X, Y, and Z will result in the box going from -6.5 to 8.5, -9 to 11, and -11.5 to 13.5 (and there’s your asymmetry in the box). Alternatively, you could think of it as generating a box 15 x 20 x 25, then placing the center of the molecule at 6.5, 9, 11.5 (but you don’t specify the box size directly, instead relying on the relative position of the molecule and the fineness of the grid to determine the position (from which you could work back to get the number of points you needed in each direction if you knew the size of the box you wanted. Yes, you might have to re-read that a few times).

I demonstrate this below for a benzene orbital “walk” along X using direct output from VESTA. The rest of the numbers in my matrix above are the same except for the “-[Start Point For The Box In X]” value.

benzene_homo_walk

The benzene walk (numbers show the spacing based on the cubegen input above).

5. Formula For Boxes And Grid Points

You can, in fact, work from the box size you want and relative position of the molecule in that box with some simple math. That looks like the table below:

-(# Atoms)           -(X Position)  -(Y Position)  -(Z Position)
(Box Size / X Mesh)    X Mesh         0.00000        0.00000
(Box Size / Y Mesh)    0.00000        Y Mesh         0.00000
(Box Size / Z Mesh)    0.00000        0.00000        Z Mesh

You specify # Atoms, X Position, Y Position, Z Position, X Mesh, Y Mesh and Z Mesh, then decide on how big your box is going to be. Also, note that X Position, Y Position, Z Position all need to be 1/2 the size of your box if you want the molecule centered. A way to help force this is to force the molecule to have its center of mass shifted to the origin using Symm=COM in your input file.

As mentioned above, a simple excel file for performing this task is provided for download at 2014june7_cubegen_excel_file.xlsx.

6. Lastly, A Procedure For Scripting The Generation Of Many Orbitals

That first stone passed, everything about making custom .cub/.cube files finally made sense. But it lead to another problem. Suppose I want to generate many molecular orbitals. Does one have to paste in the IFlag…Z3 block each time?

Thankfully, this process can be scripted to automation as well, although it’s not just a matter of pasting IFlag…Z3 below each run of cubegen. Doing that produces the following…

Example:

This isn’t a cubegen problem, it’s a Linux issue with the interpretation of stdin. The cubegen script needs to be fed in the matrix in a file (say cubegen.dat if you always want the same .cub/.cube file generated) or via the use of an EOF call.

Cubegen.dat:

cubegen 0 MO=1 benzene.fchk benzene_MO1.cub -1 h < cubegen.dat
cubegen 0 MO=2 benzene.fchk benzene_MO2.cub -1 h < cubegen.dat
cubegen 0 MO=3 benzene.fchk benzene_MO3.cub -1 h < cubegen.dat
...

EOF

cubegen 0 MO=1 benzene.fchk benzene_MO1.cub -1 h << EOF
-12  -6.50000  -9.00000 -11.50000
 60   0.25000   0.00000   0.00000
 80   0.00000   0.25000   0.00000
100   0.00000   0.00000   0.25000
EOF
cubegen 0 MO=2 benzene.fchk benzene_MO2.cub -1 h << EOF
-12  -6.50000  -9.00000 -11.50000
 60   0.25000   0.00000   0.00000
 80   0.00000   0.25000   0.00000
100   0.00000   0.00000   0.25000
EOF
cubegen 0 MO=3 benzene.fchk benzene_MO3.cub -1 h << EOF
-12  -6.50000  -9.00000 -11.50000
 60   0.25000   0.00000   0.00000
 80   0.00000   0.25000   0.00000
100   0.00000   0.00000   0.25000
EOF
...

7. What’s The Deal With The Benzene Cation?

Nothing, except I saw a question in my perusing of cubegen problems and found one related to UHF wavefunctions. How do you render alpha spin orbitals and beta spin orbitals? The answer is you dig into the .log file for the orbital energies and count (to the best of my knowledge).

Benzene (21 alpha/beta-occupied)

 Alpha  occ. eigenvalues -- -10.18955 -10.18928 -10.18928 -10.18872 -10.18872
 Alpha  occ. eigenvalues -- -10.18845  -0.84761  -0.73971  -0.73971  -0.59595
 Alpha  occ. eigenvalues --  -0.59595  -0.51588  -0.45423  -0.43943  -0.41518
 Alpha  occ. eigenvalues --  -0.41518  -0.36090  -0.33862  -0.33862  -0.24750
 Alpha  occ. eigenvalues --  -0.24750
 Alpha virt. eigenvalues --   0.00266   0.00266   0.08636   0.14126   0.14126
 Alpha virt. eigenvalues --   0.16238   0.17957   0.17957   0.18681   0.29989
 Alpha virt. eigenvalues --   0.29989   0.31908   0.31908   0.46637   0.52628
 Alpha virt. eigenvalues --   0.54782   0.55099   0.56222   0.59294   0.60077
 Alpha virt. eigenvalues --   0.60077   0.60084   0.60084   0.62384   0.62384
 Alpha virt. eigenvalues --   0.66653   0.66653   0.74180   0.81178   0.81178
 Alpha virt. eigenvalues --   0.82134   0.83694   0.83694   0.91676   0.93745
 Alpha virt. eigenvalues --   0.93745   0.95812   1.08054   1.08054   1.12992
 Alpha virt. eigenvalues --   1.12992   1.20098   1.26111   1.30051   1.40786
 Alpha virt. eigenvalues --   1.40786   1.42585   1.42585   1.42914   1.42914
 Alpha virt. eigenvalues --   1.74102   1.76078   1.80542   1.87583   1.90680
 Alpha virt. eigenvalues --   1.90680   1.97195   1.97195   1.97924   1.97924
 Alpha virt. eigenvalues --   2.02762   2.07664   2.07664   2.29609   2.29609
 Alpha virt. eigenvalues --   2.34429   2.34429   2.35491   2.39944   2.40328
 Alpha virt. eigenvalues --   2.40328   2.44636   2.44636   2.48731   2.48731
 Alpha virt. eigenvalues --   2.50802   2.58538   2.58538   2.60300   2.65987
 Alpha virt. eigenvalues --   2.75521   2.80103   2.80103   3.03123   3.03123
 Alpha virt. eigenvalues --   3.18490   3.20485   3.21867   3.21867   3.37166
 Alpha virt. eigenvalues --   3.48298   3.48298   3.93339   4.13215   4.16289
 Alpha virt. eigenvalues --   4.16289   4.43754   4.43754   4.82384

RHF wave functions are easy as the alpha and beta spin orbitals are identical (so you just call one).

Benzene Cation (21 alpha occ, 20 beta occ)

 Alpha  occ. eigenvalues --  -10.44746 -10.44745 -10.44690 -10.44689 -10.41307
 Alpha  occ. eigenvalues --  -10.41306  -1.09893  -0.99649  -0.97270  -0.83278
 Alpha  occ. eigenvalues --   -0.83268  -0.74423  -0.68358  -0.67574  -0.65278
 Alpha  occ. eigenvalues --   -0.63494  -0.61047  -0.56837  -0.56618  -0.51141
 Alpha  occ. eigenvalues --   -0.47878
 Alpha virt. eigenvalues --   -0.25225  -0.22671  -0.10624  -0.07758  -0.05310
 Alpha virt. eigenvalues --   -0.04280  -0.01821  -0.00871   0.00401   0.08260
 Alpha virt. eigenvalues --    0.08579   0.09642   0.10056   0.25206   0.29439
 Alpha virt. eigenvalues --    0.31399   0.31852   0.34121   0.36475   0.36906
 Alpha virt. eigenvalues --    0.37451   0.38343   0.38500   0.39459   0.40284
 Alpha virt. eigenvalues --    0.43576   0.45334   0.52549   0.60260   0.60770
 Alpha virt. eigenvalues --    0.61287   0.62929   0.64337   0.70989   0.71650
 Alpha virt. eigenvalues --    0.71731   0.74333   0.85713   0.86949   0.90112
 Alpha virt. eigenvalues --    0.90952   0.98816   1.00856   1.05831   1.15646
 Alpha virt. eigenvalues --    1.17792   1.17972   1.18789   1.20601   1.20854
 Alpha virt. eigenvalues --    1.49713   1.52475   1.57000   1.65756   1.66784
 Alpha virt. eigenvalues --    1.68337   1.73545   1.74011   1.74167   1.74723
 Alpha virt. eigenvalues --    1.80258   1.82880   1.84586   2.04024   2.06015
 Alpha virt. eigenvalues --    2.12117   2.12667   2.14025   2.17682   2.18940
 Alpha virt. eigenvalues --    2.19096   2.22084   2.22451   2.24748   2.25480
 Alpha virt. eigenvalues --    2.28544   2.35165   2.36888   2.39005   2.41062
 Alpha virt. eigenvalues --    2.52629   2.57091   2.57724   2.79730   2.80863
 Alpha virt. eigenvalues --    2.95189   2.99029   2.99731   3.01110   3.14403
 Alpha virt. eigenvalues --    3.25310   3.26537   3.70063   3.88553   3.90763
 Alpha virt. eigenvalues --    3.92953   4.18629   4.20462   4.58339
  Beta  occ. eigenvalues --  -10.44304 -10.44303 -10.44252 -10.44250 -10.41463
  Beta  occ. eigenvalues --  -10.41462  -1.08758  -0.97673  -0.97028  -0.82708
  Beta  occ. eigenvalues --   -0.82377  -0.74165  -0.67883  -0.67164  -0.64793
  Beta  occ. eigenvalues --   -0.63478  -0.57727  -0.56637  -0.56323  -0.47270
  Beta virt. eigenvalues --   -0.41639  -0.21435  -0.21139  -0.10438  -0.05496
  Beta virt. eigenvalues --   -0.05056  -0.04232  -0.01054  -0.00739   0.00754
  Beta virt. eigenvalues --    0.08748   0.08784   0.10027   0.10356   0.25410
  Beta virt. eigenvalues --    0.30875   0.31655   0.33033   0.34430   0.37599
  Beta virt. eigenvalues --    0.38243   0.38423   0.38827   0.38857   0.40471
  Beta virt. eigenvalues --    0.40510   0.45633   0.45687   0.53548   0.60543
  Beta virt. eigenvalues --    0.61003   0.61366   0.63303   0.64325   0.71163
  Beta virt. eigenvalues --    0.71910   0.72371   0.74501   0.86611   0.87153
  Beta virt. eigenvalues --    0.90721   0.90982   0.99163   1.02443   1.07028
  Beta virt. eigenvalues --    1.17547   1.18130   1.19642   1.19672   1.20955
  Beta virt. eigenvalues --    1.21374   1.51458   1.52709   1.57335   1.66396
  Beta virt. eigenvalues --    1.67580   1.68460   1.73895   1.74747   1.75260
  Beta virt. eigenvalues --    1.75568   1.80924   1.84865   1.84936   2.06229
  Beta virt. eigenvalues --    2.06582   2.12479   2.12665   2.14334   2.18350
  Beta virt. eigenvalues --    2.18883   2.19283   2.22289   2.22978   2.25783
  Beta virt. eigenvalues --    2.25938   2.29233   2.36212   2.37068   2.39062
  Beta virt. eigenvalues --    2.42549   2.53376   2.57824   2.57840   2.79980
  Beta virt. eigenvalues --    2.80952   2.95964   2.99101   2.99875   3.01115
  Beta virt. eigenvalues --    3.14561   3.25632   3.26592   3.70353   3.89317
  Beta virt. eigenvalues --    3.92008   3.93146   4.19813   4.20623   4.58989

In the case of UHF wave functions, you specify alpha or beta using AMO= or BMO= when you run cubegen.

Compiling And Running GAMESS-US (1 May 2013(R1)) On 64-bit Ubuntu 12.X/13.X In SMP Mode

Saturday, April 5th, 2014

Author’s Note 1: It is my standard policy to put too much info into guides so that those who are searching for specific problems they come across will find the offending text in their searches. With luck, your “build error” search sent you here.

Author’s Note 2: It’s not as bad as it looks (I’ve included lots of output and error messages for easy searching)!

Author’s Note 3: I won’t be much help for you in diagnosing your errors, but am happy to tweak the text below if something is unclear.

Conventions: I include both the commands you type in your Terminal and some of the output from these commands, the output being where most of the errors appear that I work on in the discussion.

Input is formatted as below:

username – your username (check your prompt)
machinename – your hostname (type hostname or check your prompt)

Text you put in at the (also shown, so you see the directory structure) prompt (copy + paste should be fine)

Text you get out (for checking results and reproducing errors)

Having just recently downloaded the newest version of GAMESS-US (R1 2013), my first few passes at using it under Linux (specifically, Ubuntu 12.04) ran into a few walls that required some straightforward modifications and a little bit of system prep planning. As my first few passes before successful execution are likely the same exact problems you might have run into in your attempts to get GAMESS-US to run (after a successful compilation and linking), I’m posting my problems and solutions here.

Qualifier 1 – My concern at the moment has been to get GAMESS-US to run under 64-bit Ubuntu 12.04 on a multi-core board (ye olde symmetric multiprocessing (which I always called single multi-processor, or SMP)). While some answers may follow in what’s below, this post doesn’t cover MPI-specific builds (nothing through a router, that is). SMP is the only concern (which is to say, I likely won’t have good answers if you send along an MPI-specific question). Also, although I’m VERY interested in trying it, I’ve not yet attempted to build a GPU-capable version (but plan to in the near future).

Qualifier 2 – It is my standard policy to install apps into /opt, and my steps below will reflect that (specifically because there’s a permission issue that needs to be addressed when you first try to build components). You can default to whatever you like, but keep in mind my tweaks when you try to build your local copy.

So, with the qualifiers in mind…

1. Prepping The System (apt-get)

There are few things better than being able to apt-get everything you need to prep your machine for an install, and I’m pleased to report that the (current) process for putting the important files onto Ubuntu 12.X/13.X is easy. Assuming you’re not going the Intel / PGI / MKL route, you can do everything by installing gfortran (compiler, presently installing 4.4) and the blas and atlas math libraries.

username@machinename:~$ sudo apt-get install gfortran libblas-dev libatlas-base-dev

Note: your atlas libraries will be installed in /usr/lib64/atlas/ – this will matter when you run config.

After these finish, run the following to determine your installed gfortran version (will be asked for by the new GAMESS config)

username@machinename:~$ gfortran -dumpversion

GNU Fortran (Ubuntu 4.4.3-4ubuntu5.1) 4.4.3 Copyright (C) 2010 Free Software Foundation, Inc. GNU Fortran comes with NO WARRANTY, to the extent permitted by law. You may redistribute copies of GNU Fortran under the terms of the GNU General Public License. For more information about these matters, see the file named COPYING

4.4 And you’re ready for GAMESS.

2. Downloading GAMESS-US, Placing Into /opt, And Changing Permissions

First, obviously, get the GAMESS source (click on the red text).

After downloading, copy/move gamess-current.tar.gz into /opt

username@machinename:~$ cd ~/Downloads
username@machinename:~/Downloads$ sudo cp gamess-current.tar.gz /opt
username@machinename:~/Downloads$ cd /opt
username@machinename:/opt$ sudo gunzip gamess-cuerent.tar.gz
username@machinename:/opt$ sudo tar xvd gamess-current.tar

gamess/ gamess/gms-files.csh gamess/tools/ ... gamess/misc/count.code gamess/misc/vbdum.src gamess/Makefile.in

At this point, if you go through the config process and get to the point of building ddikick.x, you will get an error when you first try to run ./compddi

username@machinename:/opt/gamess/ddi$ sudo ./compddi >& compddi.log &
[1] 4622 -bash: compddi.log: Permission denied

The problem is with the permission of the entire gamess folder:

drwxr-xr-x  4 root        root              4096 2014-04-04 21:43 . drwxr-xr-x 22 root        root              4096 2013-12-27 16:17 .. drwxr-xr-x 14 1300 504              4096 2014-04-04 21:43 gamess -rw-r--r-- 1 root        root         198481920 2014-04-04 21:42 gamess-current.tar

Which you remedy before running into this error by changing the permissions:

username@machinename:/opt$ sudo chown -R username gamess

The next step is recommended when you run config, so I’m performing the step here to get it out of the way. With the atlas libraries installed, generate two symbolic links.

username@machinename:/opt$ cd /usr/lib64/atlas
username@machinename:/usr/lib64/atlas$ sudo ln -s libf77blas.so.3.0 libf77blas.so
username@machinename:/usr/lib64/atlas$ sudo ln -s libatlas.so.3.0 libatlas.so

And, at this point, you’re ready to run the new (well, new to me) config script that preps your system install.

3. Building GAMESS-US

Back to the GAMESS-US folder.

username@machinename:/usr/lib64/atlas$ cd /opt/gamess
username@machinename:/opt/gamess$ sudo ./config
This script asks a few questions, depending on your computer system, to set up compiler names, libraries, message passing libraries, and so forth. You can quit at any time by pressing control-C, and then . Please open a second window by logging into your target machine, in case this script asks you to 'type' a command to learn something about your system software situation. All such extra questions will use the word 'type' to indicate it is a command for the other window. After the new window is open, please hit to go on.

You can open that second window or blindly assume that what I include below is all you need.

[enter]

GAMESS can compile on the following 32 bit or 64 bit machines: axp64 - Alpha chip, native compiler, running Tru64 or Linux cray-xt - Cray's massively parallel system, running CNL hpux32 - HP PA-RISC chips (old models only), running HP-UX hpux64 - HP Intel or PA-RISC chips, running HP-UX ibm32 - IBM (old models only), running AIX ibm64 - IBM, Power3 chip or newer, running AIX or Linux ibm64-sp - IBM SP parallel system, running AIX ibm-bg - IBM Blue Gene (P or L model), these are 32 bit systems linux32 - Linux (any 32 bit distribution), for x86 (old systems only) linux64 - Linux (any 64 bit distribution), for x86_64 or ia64 chips AMD/Intel chip Linux machines are sold by many companies mac32 - Apple Mac, any chip, running OS X 10.4 or older mac64 - Apple Mac, any chip, running OS X 10.5 or newer sgi32 - Silicon Graphics Inc., MIPS chip only, running Irix sgi64 - Silicon Graphics Inc., MIPS chip only, running Irix sun32 - Sun ultraSPARC chips (old models only), running Solaris sun64 - Sun ultraSPARC or Opteron chips, running Solaris win32 - Windows 32-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition) win64 - Windows 64-bit (Windows XP, Vista, 7, Compute Cluster, HPC Edition) winazure - Windows Azure Cloud Platform running Windows 64-bit type 'uname -a' to partially clarify your computer's flavor. please enter your target machine name:

We’re doing a linux64 build, so type the following at the prompt:

linux64
Where is the GAMESS software on your system? A typical response might be /u1/mike/gamess, most probably the correct answer is /opt/gamess GAMESS directory? [/opt/gamess]

Who is this mike and where is my folder u1? We’ll get to that in rungms. For now, I’m installing in /opt, so the default directory is fine:

[enter]

Setting up GAMESS compile and link for GMS_TARGET=linux64 GAMESS software is located at GMS_PATH=/opt/gamess Please provide the name of the build locaation. This may be the same location as the GAMESS directory. GAMESS build directory? [/opt/gamess]

Fine as selected.

[enter]

Please provide a version number for the GAMESS executable. This will be used as the middle part of the binary's name, for example: gamess.00.x Version? [00]

Is this important? Maybe, if you plan on building multiple versions of GAMESS-US (you might want a GPU-friendly version, one with a different compiler, one with MPI, etc.). Number as you wish and remember the number when it comes to rungms. That said, the actual linking step seems to really want to produce a 01 version (we’ll get to that). Meantime, default value is fine.

[enter]

Linux offers many choices for FORTRAN compilers, including the GNU compiler set ('g77' in old versions of Linux, or 'gfortran' in current versions), which are included for free in Unix distributions. There are also commercial compilers, namely Intel's 'ifort', Portland Group's 'pgfortran', and Pathscale's 'pathf90'. The last two are not common, and aren't as well tested as the others. type 'rpm -aq | grep gcc' to check on all GNU compilers, including gcc type 'which gfortran' to look for GNU's gfortran (a very good choice), type 'which g77' to look for GNU's g77, type 'which ifort' to look for Intel's compiler, type 'which pgfortran' to look for Portland Group's compiler, type 'which pathf90' to look for Pathscale's compiler. Please enter your choice of FORTRAN:

We’re using gfortran (currently 4.4.3):

gfortran

gfortran is very robust, so this is a wise choice. Please type 'gfortran -dumpversion' or else 'gfortran -v' to detect the version number of your gfortran. This reply should be a string with at least two decimal points, such as 4.1.2 or 4.6.1, or maybe even 4.4.2-12. The reply may be labeled as a 'gcc' version, but it is really your gfortran version. Please enter only the first decimal place, such as 4.1 or 4.6:
4.4

Alas, your version of gfortran does not support REAL*16, so relativistic integrals cannot use quadruple precision. Other than this, everything will work properly. hit to continue to the math library setup.

If this was my biggest concern I’d be a happy quantum chemist. Obviously you can try to install other flavors of gfortran and, possibly, by the time you need the procedure I’m following, a newer version of gfortran will be apt-gotten.

[enter]

Linux distributions do not include a standard math library. There are several reasonable add-on library choices, MKL from Intel for 32 or 64 bit Linux (very fast) ACML from AMD for 32 or 64 bit Linux (free) ATLAS from www.rpmfind.net for 32 or 64 bit Linux (free) and one very unreasonable option, namely 'none', which will use some slow FORTRAN routines supplied with GAMESS. Choosing 'none' will run MP2 jobs 2x slower, or CCSD(T) jobs 5x slower. Some typical places (but not the only ones) to find math libraries are Type 'ls /opt/intel/mkl' to look for MKL Type 'ls /opt/intel/Compiler/mkl' to look for MKL Type 'ls /opt/intel/composerxe/mkl' to look for MKL Type 'ls -d /opt/acml*' to look for ACML Type 'ls -d /usr/local/acml*' to look for ACML Type 'ls /usr/lib64/atlas' to look for Atlas Enter your choice of 'mkl' or 'atlas' or 'acml' or 'none':
atlas

Where is your Atlas math library installed? A likely place is /usr/lib64/atlas Please enter the Atlas subdirectory on your system:

Our location is, in fact, /usr/lib64/atlas, so we type it in accordingly.

NOTE: If you don’t type anything but [enter] below, the script closes (/usr/lib64/atlas is listed as the expected location, but it is not defaulted by the script. You need to type it in.

/usr/lib64/atlas
 
The linking step in GAMESS assumes that a softlink exists within the system's /usr/lib64/atlas from libatlas.so to a specific file like libatlas.so.3.0 from libf77blas.so to a specific file like libf77blas.so.3.0 config can carry on for the moment, but the 'root' user should chdir /usr/lib64/atlas ln -s libf77blas.so.3.0 libf77blas.so ln -s libatlas.so.3.0 libatlas.so prior to the linking of GAMESS to a binary executable. Math library 'atlas' will be taken from /usr/lib64/atlas please hit to compile the GAMESS source code activator

The symbolic linking was performed before the GAMESS steps.

[enter]

gfortran -o /home/username/gamess/tools/actvte.x actvte.f unset echo Source code activator was successfully compiled. please hit to set up your network for Linux clusters.
[enter]

If you have a slow network, like Gigabit Ethernet (GE), or if you have so few nodes you won't run extensively in parallel, or if you have no MPI library installed, or if you want a fail-safe compile/link and easy execution, choose 'sockets' to use good old reliable standard TCP/IP networking. If you have an expensive but fast network like Infiniband (IB), and if you have an MPI library correctly installed, choose 'mpi'. communication library ('sockets' or 'mpi')?

Again, I’m not building an mpi-friendly version, so am using sockets.

sockets

64 bit Linux builds can attach a special LIBCCHEM code for fast MP2 and CCSD(T) runs. The LIBCCHEM code can utilize nVIDIA GPUs, through the CUDA libraries, if GPUs are available. Usage of LIBCCHEM requires installation of HDF5 I/O software as well. GAMESS+LIBCCHEM binaries are unable to run most of GAMESS computations, and are a bit harder to create due to the additional CUDA/HDF5 software. Therefore, the first time you run 'config', the best answer is 'no'! If you decide to try LIBCCHEM later, just run this 'config' again. Do you want to try LIBCCHEM? (yes/no):
no

Your configuration for GAMESS compilation is now in /home/username/gamess/install.info Now, please follow the directions in /home/username/gamess/machines/readme.unix username@machinename:~/gamess$

At this stage, you’re ready to build ddikick.x and continue with the compiling.

4. Build ddikick.x

username@machinename:/opt/gamess$ cd ddi
username@machinename:/opt/gamess/ddi$ sudo ./compddi >& compddi.log &

Will dump output into compddi.log (which will now work with the correct permissions).

username@machinename:/opt/gamess/ddi$ sudo mv ddikick.x ..
username@machinename:/opt/gamess/ddi$ cd ..
username@machinename:/opt/gamess$ sudo ./compall >& compall.log &

Feel free to follow along as compall.log dumps results. You’re also welcome to follow the readme.unix advice:

This takes a while, so go for coffee, or check the SF Giants web page.

Upon completion, the last step is to link the executable.

Now, it used to be the case that you specified the version number in the lked step. So, if you wanted to stick with the 00 version from the config file, you’d type

username@machinename:/opt/gamess$ sudo ./lked gamess 00 >& lked.log &

When you do that at present, you get

[1] 7626 username@machinename:/opt/gamess$ [1]+ Stopped sudo ./lked gamess 00 &>lked.log

This then leads you to use the lked call from the readme.unix file.

username@machinename:/opt/gamess$ sudo ./lked gamess 01 >& lked.log &

Which then produces lked.log and gamess.01.x.

Now, if you run with 00 again, you get a successful linking of gamess.00.x . Not sure why this happens, but the version number isn’t important so long as you specify the right one when you use rungms (so I’ve not diagnosed it further).

At this point, you have a gamess.00.x and/or gamess.01.x executable in your /opt/gamess folder:

30828747 2014-04-04 22:41 gamess.01.x

I’m going to ignore the 00 issue out of the config file and use the gamess.01.x executable.

We’re ready to run calculations and work through the next set of errors you’ll receive if you don’t properly modify files.

5. PATH Setting

First, we copy rungms to our home folder, then add /opt/gamess to the PATH:

username@machinename:/opt/gamess$ cp rungms ~/
username@machinename:/opt/gamess$ cd ~/
username@machinename:~$ nano .bashrc

Add the following to the bottom of .bashrc (or extend your PATH)

PATH=$PATH:/opt/gamess

Quit nano and source.

username@machinename:~$ source .bashrc
[OPTIONAL] username@machinename:~$ echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:.../opt/gamess:

6. rungms (Probably Why You’re Here)

If you just go blindly into a run, you’ll get the following error:

username@machinename:~$ ./rungms test.inp

----- GAMESS execution script 'rungms' ----- This job is running on host machinename under operating system Linux at Fri Apr 4 22:47:55 EDT 2014 Available scratch disk space (Kbyte units) at beginning of the job is df: `/scr/username': No such file or directory df: no file systems processed GAMESS temporary binary files will be written to /scr/username GAMESS supplementary output files will be written to /home/username/scr Copying input file test.inp to your run's scratch directory... cp test.inp /scr/username/test.F05 cp: cannot create regular file `/scr/username/test.F05': No such file or directory unset echo /u1/mike/gamess/gms-files.csh: No such file or directory.

As is obvious, rungms needs some modifying.

username@machinename:~$ nano rungms

Scroll down until you see the following:

set TARGET=sockets set SCR=/scr/$USER set USERSCR=~$USER/scr set GMSPATH=/u1/mike/gamess

Given that it’s just me on the machine, I tend to simplify this by making SCR and USERSCR the same directory, and I make them both /tmp. If you intend on keeping all of the files, you’ll need to make rungms specific for each run case. My only concerns are .dat and .log, so /tmp dumping is fine. Furthermore, we must change GMSPATH from how the ever-helpful Mike Schmidt (he got me through some early issues when I started my GAMESS-US adventure 15ish years ago. Won’t complain about his continued default-ed presence in the scripts) has it set up at Iowa to how we want it on our own machines (in my case, /opt/gamess)

set TARGET=sockets set SCR=/tmp set USERSCR=/tmp set GMSPATH=/opt/gamess

With these modifications, your next run will be a bit more successful:

username@machinename:~$ ./rungms test.inp

----- GAMESS execution script 'rungms' ----- This job is running on host machinename under operating system Linux at Fri Apr 4 22:51:35 EDT 2014 Available scratch disk space (Kbyte units) at beginning of the job is Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 1905222596 249225412 1559217460 14% / GAMESS temporary binary files will be written to /tmp GAMESS supplementary output files will be written to /tmp Copying input file test.inp to your run's scratch directory... cp test.inp /tmp/test.F05 unset echo /opt/gamess/ddikick.x /opt/gamess/gamess.00.x test -ddi 1 1 machinename -scr /tmp Distributed Data Interface kickoff program. Initiating 1 compute processes on 1 nodes to run the following command: /opt/gamess/gamess.00.x test ****************************************************** * GAMESS VERSION = 1 MAY 2013 (R1) * * FROM IOWA STATE UNIVERSITY * * M.W.SCHMIDT, K.K.BALDRIDGE, J.A.BOATZ, S.T.ELBERT, * * M.S.GORDON, J.H.JENSEN, S.KOSEKI, N.MATSUNAGA, * * K.A.NGUYEN, S.J.SU, T.L.WINDUS, * * TOGETHER WITH M.DUPUIS, J.A.MONTGOMERY * * J.COMPUT.CHEM. 14, 1347-1363(1993) * **************** 64 BIT LINUX VERSION **************** ... INPUT CARD> DDI Process 0: shmget returned an error. Error EINVAL: Attempting to create 160525768 bytes of shared memory. Check system limits on the size of SysV shared memory segments. The file ~/gamess/ddi/readme.ddi contains information on how to display the current SystemV memory settings, and how to increase their sizes. Increasing the setting requires the root password, and usually a sytem reboot. DDI Process 0: error code 911 ddikick.x: application process 0 quit unexpectedly. ddikick.x: Fatal error detected. The error is most likely to be in the application, so check for input errors, disk space, memory needs, application bugs, etc. ddikick.x will now clean up all processes, and exit... ddikick.x: Sending kill signal to DDI processes. ddikick.x: Execution terminated due to error(s). unset echo ----- accounting info ----- Files used on the master node machinename were: -rw-r--r-- 1 username username 0 2014-04-04 22:51 /tmp/test.dat -rw-r--r-- 1 username username 1341 2014-04-04 22:51 /tmp/test.F05 ls: No match. ls: No match. ls: No match. Fri Apr 4 22:51:36 EDT 2014 0.0u 0.0s 0:01.08 9.2% 0+0k 0+8io 0pf+0w

Things worked, but with a memory error. This issue is discussed at the Baldridge Group wiki: ocikbapps.uzh.ch/kbwiki/gamess_troubleshooting.html

From the wiki:

If you are sure you are not asking for too much memory in the input file, check that your kernel parameters are not allowing enough memory to be requested. You might have to increase the SHMALL & SHMAX kernel memory values to allow GAMESS to run. (See http://www.pythian.com/news/245/the-mysterious-world-of-shmmax-and-shmall/ for a better explanation.)
For example, on a machine with 4GB of memory, you might add these to /etc/sysctl.conf:
# cat /etc/sysctl.conf | grep shm
kernel.shmmax = 3064372224
kernel.shmall = 748137
Then set the new settings like so:
# sysctl -p
Since they are in /etc/sysctl.conf, they will automatically be set each time the system is booted.

In our case, we modify sysctl.conf with the recommendations from the wiki:

username@machinename:~$ sudo nano /etc/sysctl.conf

Add the following to the bottom of the file:

kernel.shmmax = 3064372224 kernel.shmall = 748137

Save and exit.

username@machinename:~$ sudo sysctl -p

net.ipv4.ip_forward = 1 kernel.shmmax = 3064372224 kernel.shmall = 748137

These memory values will change depending on your system.

Now we empty the /tmp and rerun.

username@machinename:~$ rm /tmp/*
username@machinename:~$ ./rungms test.inp

If your input file is worth it’s salt, you’ll have successfully run your file on a single processor (single core, that is). If you run into additional memory errors, increase kernel.shmmax and kernel.shmall.

Now, onto the SMP part. My first attempt to run games in parallel (on 4 cores using version 00) produced the following error:

username@machinename:~$ rm /tmp/*
username@machinename:~$ ./rungms test.inp 00 4

----- GAMESS execution script 'rungms' ----- This job is running on host machinename under operating system Linux at Fri Apr 4 22:52:52 EDT 2014 Available scratch disk space (Kbyte units) at beginning of the job is Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda2 1905222596 249225416 1559217456 14% / GAMESS temporary binary files will be written to /tmp GAMESS supplementary output files will be written to /tmp Copying input file test.inp to your run's scratch directory... cp test.inp /tmp/test.F05 unset echo I do not know how to run this node in parallel.

I tried a number of stupid things to get the run to work, finally settling on modifying the rungms file properly. To make gamess know how to run the node in parallel, we need only make the following changes to our rungms file.

username@machinename:~$ nano rungms

Scroll down until you find the section below:

# 2. This is an example of how to run on a multi-core SMP enclosure, # where all CPUs (aka COREs) are inside a -single- NODE. # At other locations, you may wish to consider some of the examples # that follow below, after commenting out this ISU specific part. if ($NCPUS > 1) then switch (`hostname`) case se.msg.chem.iastate.edu: case sb.msg.chem.iastate.edu: if ($NCPUS > 2) set NCPUS=4 set NNODES=1

The change is simple. We remove the cases for $NCPUS > 1 in the file and add the hostname of our linux box (and if you don’t know this or it’s not in your prompt, simply type hostname at the prompt first). We’ll disable the two cases listed and add our hostname to the case list.

# 2. This is an example of how to run on a multi-core SMP enclosure, # where all CPUs (aka COREs) are inside a -single- NODE. # At other locations, you may wish to consider some of the examples # that follow below, after commenting out this ISU specific part. if ($NCPUS > 1) then switch (`hostname`) case machinename: # case se.msg.chem.iastate.edu: # case sb.msg.chem.iastate.edu: if ($NCPUS > 2) set NCPUS=4 set NNODES=1

This gives you parallel functionality, but it’s still not using the machine resources (cores) correctly when I ask for anything more than 2 cores (always using only 2 cores).

[minor complaint]
Admittedly, I don’t immediately get the logic of this section as currently coded, as one cannot get more than 2 cores to work in this case given how the if statements are written (so far as I can see now. I will assume I am the one missing something but have not decided to ask about it, instead changing the rungms text to the following). You can check this yourself by running top in another window. This is the most simple modification, and assumes you want to run N number of cores each time. Clearly, you can make this more elegant than it is (my modification, that is). Meantime, I want to run 4 cores on this machine, so I change the section to reflect a 4-core board (and commented out much of this section).
[/complaint]

# 2. This is an example of how to run on a multi-core SMP enclosure, # where all CPUs (aka COREs) are inside a -single- NODE. # At other locations, you may wish to consider some of the examples # that follow below, after commenting out this ISU specific part. if ($NCPUS > 1) then switch (`hostname`) case machinename # case se.msg.chem.iastate.edu: # case sb.msg.chem.iastate.edu: # if ($NCPUS > 2) set NCPUS=2 # set NNODES=1 # set HOSTLIST=(`hostname`:cpus=$NCPUS) # breaksw # case machinename # case br.msg.chem.iastate.edu: if ($NCPUS >= 4) set NCPUS=4 set NNODES=1 set HOSTLIST=(`hostname`:cpus=$NCPUS) breaksw case machinename # case cd.msg.chem.iastate.edu: # case zn.msg.chem.iastate.edu: # case ni.msg.chem.iastate.edu: # case co.msg.chem.iastate.edu: # case pb.msg.chem.iastate.edu: # case bi.msg.chem.iastate.edu: # case po.msg.chem.iastate.edu: # case at.msg.chem.iastate.edu: # case sc.msg.chem.iastate.edu: # if ($NCPUS > 4) set NCPUS=4 # set NNODES=1 # set HOSTLIST=(`hostname`:cpus=$NCPUS) # breaksw # case ga.msg.chem.iastate.edu: # case ge.msg.chem.iastate.edu: # case gd.msg.chem.iastate.edu: # if ($NCPUS > 6) set NCPUS=6 # set NNODES=1 # set HOSTLIST=(`hostname`:cpus=$NCPUS) # breaksw default: echo I do not know how to run this node in parallel. exit 20 endsw endif #

And, with this set of changes, I’m using all 4 cores on the board (but have some significant memory issues when running MP2 calks. But that’s for another post).

The typical user will never be able to do what the GAMESS group has done in making an excellent program that also happens to be free. That said, the need to make changes to the rungms file is something that would be greatly simplified by having N number of rungms scripts for each case instead of a monolithic file that is mostly useless text to users not using one of the system types. This, for instance, would make rungms modification much easier. If I streamline rungms for my specific system, I may post a new file accordingly.

CudaMiner Installation In Ubuntu 12.04 LTS Using CUDA Toolkit 5.5 And “Additional NVIDIA Drivers”

Saturday, December 28th, 2013

Author’s Note 1: It is my standard policy to put too much info into guides so that those who are searching for specific problems they come across will find the offending text in their searches. With luck, your “build error” search sent you here.

Author’s Note 2: It’s not as bad as it looks (I’ve included lots of output and error messages for easy searching)!

Author’s Note 3: I won’t be much help for you in diagnosing your errors, but am happy to tweak the text below if something is unclear.

Conventions: I include both the commands you type in your Terminal and some of the output from these commands, the output being where most of the errors appear that I work on in the discussion.

Input is formatted as below:

Text you put in (copy + paste should be fine)

Output is formatted as below:

Text you get out (for checking results and reproducing errors)

1. Introduction

This work began as an attempt to build a CUDA-friendly version of the molecular dynamics package GROMACS (which will come later) but, for reasons stemming from a new local Syracuse Meetup Group (Bitcoin’s of New York – Miner’s of Syracuse. Consider joining!), the formation of our very own local mining pool (Salt City Miners, miner.saltcityminers.com. Consider joining!), plus a “what the hell” to see if it was an easy build or not, transformed into the CudaMiner-centric compiling post you see here.

NOTE: This will be a 64-bit-centric install but I’ll include 32-bit content as I’ve found the info on other sites.

2. Installing The NVIDIA Drivers (Two Methods, The Easy One Described)

Having run through this process many times in a fresh install of Ubuntu 12.04 LTS (so nothing else is on the machine except 12.04 LTS, its updates, a few extra installs, and the CUDA/CudaMiner codes), I can say that what is below should work without hitch AFTER you install the NVIDIA drivers. Once your NVIDIA card is installed and Ubuntu recognizes it, you’ve two options.

2A. Install The Drivers From An NVIDIA Download (The Hard Version)

A few websites (and several repostings of the same content) describe the process of installing the NVIDIA drivers the olde-fashioned way, in which you’ll see references to “blacklist nouveau,” “sudo service lightdm stop,” Ctrl+Alt+F1 (to get you to a text-only session), etc. You hopefully don’t need to do this much work for your own NVIDIA install, as Ubuntu will do it for you (with only one restart required).

2B. Install The Drivers After The “Restricted Drivers Available” Pop-Up Or Go To System Settings > Available Drivers (The Easy, Teenage New York Version)

I took the easy way out by letting Ubuntu do the dirty work. The result is the installation of the (currently, as of 28 Dec 2013) v. 319 NVIDIA accelerated graphics driver. For my NVIDIA cards (GTX 690 and a GTX 650 Ti, although I assume it’s similar for a whole class of NVIDIA cards), you’re (currently, check the date again) given the option of v. 304. Don’t! I’ve seen several mentions of CudaMiner (and some of the cuds toolkit) requiring v. 319.

2013dec28_nvidia_1

Caption: You may see it in the upper right (after an install or if you’ve not clicked on it before)

2013dec28_nvidia_2

Caption: Or System Settings > Additional Drivers

2013dec28_nvidia_3

Caption: Either way, you’ll hopefully get to an NVIDIA driver list like above.

3. Pre-CUDA Toolkit Install

There are a few apt-get’s you need to do before installing the CUDA Toolkit (or, at least, the consensus is that these must be done. I’ve not seen a different list in any posts and I didn’t bother to install one-by-one to see which of these might not be needed).

If you perform the most commonly posted apt-get (plus and update and upgrade if you’ve not done so lately):

user@host:~/$ sudo apt-get update
user@host:~/$ sudo apt-get upgrade
user@host:~/$ sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev

You’ll get the following error from a fresh 12.04 LTS install:

Reading package lists… Done
Building dependency tree
Reading state information… Done
libglu1-mesa is already the newest version.
libglu1-mesa set to manually installed.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libgl1-mesa-glx : Depends: libglapi-mesa (= 8.0.4-0ubuntu0.6)
Recommends: libgl1-mesa-dri (>= 7.2)
E: Unable to correct problems, you have held broken packages.

The solution here is simple. Add libglapi-mesa and libgl1-mesa-dri to your install.

user@host:~/$ sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev libglapi-mesa libgl1-mesa-dri

Doing this will add a bunch of programs and libraries (listed below):

The following extra packages will be installed:
dpkg-dev freeglut3 g++ g++-4.6 libalgorithm-diff-perl libalgorithm-diff-xs-perl
libalgorithm-merge-perl libdpkg-perl libdrm-dev libgl1-mesa-dev libice-dev libkms1 libllvm3.0
libpthread-stubs0 libpthread-stubs0-dev libsm-dev libstdc++6-4.6-dev libtimedate-perl libx11-doc
libxau-dev libxcb1-dev libxdmcp-dev libxext-dev libxmu-headers libxt-dev mesa-common-dev
x11proto-core-dev x11proto-input-dev x11proto-kb-dev x11proto-xext-dev xorg-sgml-doctools
xserver-xorg xserver-xorg-core xserver-xorg-input-evdev xtrans-dev
Suggested packages:
debian-keyring g++-multilib g++-4.6-multilib gcc-4.6-doc libstdc++6-4.6-dbg libglide3
libstdc++6-4.6-doc libxcb-doc xfonts-100dpi xfonts-75dpi
The following packages will be REMOVED:
libgl1-mesa-dri-lts-raring libgl1-mesa-glx-lts-raring libglapi-mesa-lts-raring
libxatracker1-lts-raring x11-xserver-utils-lts-raring xserver-common-lts-raring
xserver-xorg-core-lts-raring xserver-xorg-input-all-lts-raring xserver-xorg-input-evdev-lts-raring
xserver-xorg-input-mouse-lts-raring xserver-xorg-input-synaptics-lts-raring
xserver-xorg-input-vmmouse-lts-raring xserver-xorg-input-wacom-lts-raring xserver-xorg-lts-raring
xserver-xorg-video-all-lts-raring xserver-xorg-video-ati-lts-raring
xserver-xorg-video-cirrus-lts-raring xserver-xorg-video-fbdev-lts-raring
xserver-xorg-video-intel-lts-raring xserver-xorg-video-mach64-lts-raring
xserver-xorg-video-mga-lts-raring xserver-xorg-video-modesetting-lts-raring
xserver-xorg-video-neomagic-lts-raring xserver-xorg-video-nouveau-lts-raring
xserver-xorg-video-openchrome-lts-raring xserver-xorg-video-r128-lts-raring
xserver-xorg-video-radeon-lts-raring xserver-xorg-video-s3-lts-raring
xserver-xorg-video-savage-lts-raring xserver-xorg-video-siliconmotion-lts-raring
xserver-xorg-video-sis-lts-raring xserver-xorg-video-sisusb-lts-raring
xserver-xorg-video-tdfx-lts-raring xserver-xorg-video-trident-lts-raring
xserver-xorg-video-vesa-lts-raring xserver-xorg-video-vmware-lts-raring
The following NEW packages will be installed:
build-essential dpkg-dev freeglut3 freeglut3-dev g++ g++-4.6 libalgorithm-diff-perl
libalgorithm-diff-xs-perl libalgorithm-merge-perl libdpkg-perl libdrm-dev libgl1-mesa-dev
libgl1-mesa-dri libgl1-mesa-glx libglapi-mesa libglu1-mesa-dev libice-dev libkms1 libllvm3.0
libpthread-stubs0 libpthread-stubs0-dev libsm-dev libstdc++6-4.6-dev libtimedate-perl libx11-dev
libx11-doc libxau-dev libxcb1-dev libxdmcp-dev libxext-dev libxi-dev libxmu-dev libxmu-headers
libxt-dev mesa-common-dev x11proto-core-dev x11proto-input-dev x11proto-kb-dev x11proto-xext-dev
xorg-sgml-doctools xserver-xorg xserver-xorg-core xserver-xorg-input-evdev xtrans-dev

And, remarkably, that’s it for the pre-install.

4. CUDA Toolkit 5.5(.22) Install

The CUDA Toolkit install starts with its 810 MB download at developer.NVIDIA.com/cuda-downloads.

Obviously, be aware of the 32- and 64-bit options. Also, the .deb doesn’t currently download, leaving you to grab the .run file (same difference, I haven’t bothered to find out why the .deb doesn’t fly yet).

Off to your Terminal and into the Downloads folder:

user@host:~/$ cd Downloads
user@host:~/Downloads$ chmod +x cuda_5.5.22_linux_64.run
user@host:~/Downloads$ sudo ./cuda_5.5.22_linux_64.run

Which will produce:

Logging to /tmp/cuda_install_14755.log
Using more to view the EULA.
End User License Agreement
————————–
. . .
and cannot be linked to any personally identifiable
information. Personally identifiable information such as your
username or hostname is not collected.

————————————————————-

Finally, some input to be had after the scrolling:

Do you accept the previously read EULA? (accept/decline/quit): accept     
Install NVIDIA Accelerated Graphics Driver for Linux-x86_64 319.37? ((y)es/(n)o/(q)uit): n
Install the CUDA 5.5 Toolkit? ((y)es/(n)o/(q)uit): y
Enter Toolkit Location [ default is /usr/local/cuda-5.5 ]: 
Install the CUDA 5.5 Samples? ((y)es/(n)o/(q)uit): y
Enter CUDA Samples Location [ default is /home/user/NVIDIA_CUDA-5.5_Samples ]: 

NOTE 1: Don’t install the NVIDIA Accelerated Graphics Driver!
NOTE 2: Yes, install the Toolkit.
NOTE 3: I will assume this location for all of the below, setting the location in the PATH.
NOTE 4: I installed the samples for testing (and found a few extra things that need installation for them).
NOTE 5: Default is fine. Once built and tested, can be deleted (although the Mandelbrot is a keeper)

Installing the CUDA Toolkit in /usr/local/cuda-5.5 …
Installing the CUDA Samples in /home/user/NVIDIA_CUDA-5.5_Samples …
Copying samples to /home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples now…
Finished copying samples.

===========
= Summary =
===========

Driver: Not Selected
Toolkit: Installed in /usr/local/cuda-5.5
Samples: Installed in /home/user/NVIDIA_CUDA-5.5_Samples

* Please make sure your PATH includes /usr/local/cuda-5.5/bin

* Please make sure your LD_LIBRARY_PATH
* for 32-bit Linux distributions includes /usr/local/cuda-5.5/lib
* for 64-bit Linux distributions includes /usr/local/cuda-5.5/lib64:/lib
* OR
* for 32-bit Linux distributions add /usr/local/cuda-5.5/lib
* for 64-bit Linux distributions add /usr/local/cuda-5.5/lib64 and /lib
* to /etc/ld.so.conf and run ldconfig as root

* To uninstall CUDA, remove the CUDA files in /usr/local/cuda-5.5
* Installation Complete

Please see CUDA_Getting_Started_Linux.pdf in /usr/local/cuda-5.5/doc/pdf for detailed information on setting up CUDA.

***WARNING: Incomplete installation! This installation did not install the CUDA Driver. A driver of version at least 319.00 is required for CUDA 5.5 functionality to work.
To install the driver using this installer, run the following command, replacing with the name of this run file:
sudo
.run -silent -driver

Logfile is /tmp/cuda_install_14755.log

And ignore the WARNING.

As per the “make sure” above, add the CUDA distro folders to your path and LD_LIBRARY_PATH (I chose not to modify ld.so.conf)

user@host:~/Downloads$ cd
user@host:~/$ nano .bashrc

Add the PATH and LD_LIBRARY_PATH as follows:

PATH=$PATH:/usr/local/cuda-5.5/bin
LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-5.5/lib64:/lib

And then source the .bashrc file.

user@host:~/$ source .bashrc

5. NVIDIA_CUDA-5.5_Samples (And Finishing The Toolkit Install To Build CudaMiner)

The next set of installs and file modifications came from attempting to build the Samples in the NVIDIA_CUDA-5.5_Samples (or NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples depending on how your install did it) library, during which time I think I managed to hit all of the post-Toolkit install modifications needed to make the CudaMiner build problem-free. The OpenMPI install is optional, but I do hate error messages.

5A. Error 1: /usr/bin/ld: cannot find -lcuda

My first make attempt produced the following error:

user@host:~/$ cd NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples
user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples$ make

make[1]: Entering directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/0_Simple/asyncAPI’
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_10,code=sm_10 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=\”sm_35,compute_35\” -o asyncAPI.o -c asyncAPI.cu
. . .
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -I../../common/inc -m64 -o vectorAddDrv.o -c vectorAddDrv.cpp
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -m64 -o vectorAddDrv vectorAddDrv.o -L/usr/lib/NVIDIA-current -lcuda
/usr/bin/ld: cannot find -lcuda
collect2: ld returned 1 exit status
make[1]: *** [vectorAddDrv] Error 1
make[1]: Leaving directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/0_Simple/vectorAddDrv’
make: *** [0_Simple/vectorAddDrv/Makefile.ph_build] Error 2

This is solved by making a symbolic link for libcuda.so out of /usr/lib/NVIDIA-319/ and into /usr/lib/

NOTE: It doesn’t matter what directory you do this from. I’ve left off the NVIDIA_CUDA-5.5_Samples/ yadda yadda below.

user@host:~/$ sudo ln -s /usr/lib/NVIDIA-319/libcuda.so /usr/lib/libcuda.so

If you’re working through the build process and hit the error, run a “make clean” before rerunning.

5B. WARNING – No MPI compiler found.

The second build attempt produced the MPI Warning above.

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples$ make

make[1]: Entering directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/0_Simple/asyncAPI’
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_10,code=sm_10 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=\”sm_35,compute_35\” -o asyncAPI.o -c asyncAPI.cu
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -m64 -o asyncAPI asyncAPI.o
. . .
cp simpleCubemapTexture ../../bin/x86_64/linux/release
make[1]: Leaving directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/0_Simple/simpleCubemapTexture’
———————————————————————————————–
WARNING – No MPI compiler found.
———————————————————————————————–
CUDA Sample “simpleMPI” cannot be built without an MPI Compiler.
This will be a dry-run of the Makefile.
For more information on how to set up your environment to build and run this
sample, please refer the CUDA Samples documentation and release notes
———————————————————————————————–
make[1]: Entering directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/0_Simple/simpleMPI’
[@] mpicxx -I../../common/inc -o simpleMPI.o -c simpleMPI.cpp
. . .
mkdir -p ../../bin/x86_64/linux/release
cp histEqualizationNPP ../../bin/x86_64/linux/release
make[1]: Leaving directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/7_CUDALibraries/histEqualizationNPP’
Finished building CUDA samples

But otherwise finishes successfully.

To get around this warning, install OpenMPI (which is needed for multi-board GROMACS runs anyway. But, again, not needed for CudaMiner). The specific issue is the need for mpicc, which is in libopenmpi-dev (not openmpi-bin or openmpi-common).

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples$ mpicc
The program ‘mpicc’ can be found in the following packages:
* lam4-dev
* libmpich-mpd1.0-dev
* libmpich-shmem1.0-dev
* libmpich1.0-dev
* libmpich2-dev
* libopenmpi-dev
* libopenmpi1.5-dev
Try: sudo apt-get install

For completeness, I grab all three (and I’m ingnoring the NVIDIA_CUDA-5.5_Samples directory structure below).

user@host:~/$ sudo apt-get install openmpi-bin openmpi-common libopenmpi-dev

Running mpicc will now produce the following (so it’s there):

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples$ mpicc
gcc: fatal error: no input files
compilation terminated.

Now run a “make clean” if needed and make. The build should go without problem.

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples$ make

make[1]: Entering directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/0_Simple/asyncAPI’
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -I../../common/inc -m64 -gencode arch=compute_10,code=sm_10 -gencode arch=compute_20,code=sm_20 -gencode arch=compute_30,code=sm_30 -gencode arch=compute_35,code=\”sm_35,compute_35\” -o asyncAPI.o -c asyncAPI.cu
“/usr/local/cuda-5.5″/bin/nvcc -ccbin g++ -m64 -o asyncAPI asyncAPI.o
mkdir -p ../../bin/x86_64/linux/release
. . .
mkdir -p ../../bin/x86_64/linux/release
cp histEqualizationNPP ../../bin/x86_64/linux/release
make[1]: Leaving directory `/home/user/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/7_CUDALibraries/histEqualizationNPP’
Finished building CUDA samples

5C. Needed post-processing (lib glut, cuda.conf, NVIDIA.conf, and ldconfig)

The next round of problems stemmed from not being able to run the randomFog program in the new ~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release folder. I suspect the steps taken to remedy this also make all future CUDA-specific work easier, so list the issues and clean-up steps below.

Out of the list of build samples, I selected a few that worked without issue and, finally, randomFog that decidedly had issues:

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/$ cd bin/x86_64/linux/release
user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release$ ls
alignedTypes HSOpticalFlow simpleCUBLAS
asyncAPI imageDenoising simpleCUDA2GL
bandwidthTest imageSegmentationNPP simpleCUFFT
batchCUBLAS inlinePTX simpleDevLibCUBLAS
bicubicTexture interval simpleGL
bilateralFilter jpegNPP simpleHyperQ
bindlessTexture lineOfSight simpleIPC
binomialOptions Mandelbrot simpleLayeredTexture
BlackScholes marchingCubes simpleMPI
boxFilter matrixMul simpleMultiCopy
boxFilterNPP matrixMulCUBLAS simpleMultiGPU
cdpAdvancedQuicksort matrixMulDrv simpleP2P
cdpBezierTessellation matrixMulDynlinkJIT simplePitchLinearTexture
cdpLUDecomposition matrixMul_kernel64.ptx simplePrintf
cdpQuadtree MC_EstimatePiInlineP simpleSeparateCompilation
cdpSimplePrint MC_EstimatePiInlineQ simpleStreams
cdpSimpleQuicksort MC_EstimatePiP simpleSurfaceWrite
clock MC_EstimatePiQ simpleTemplates
concurrentKernels MC_SingleAsianOptionP simpleTexture
conjugateGradient mergeSort simpleTexture3D
conjugateGradientPrecond MersenneTwisterGP11213 simpleTextureDrv
convolutionFFT2D MonteCarloMultiGPU simpleTexture_kernel64.ptx
convolutionSeparable nbody simpleVoteIntrinsics
convolutionTexture newdelete simpleZeroCopy
cppIntegration oceanFFT smokeParticles
cppOverload particles SobelFilter
cudaOpenMP postProcessGL SobolQRNG
dct8x8 ptxjit sortingNetworks
deviceQuery quasirandomGenerator stereoDisparity
deviceQueryDrv radixSortThrust template
dwtHaar1D randomFog template_runtime
dxtc recursiveGaussian threadFenceReduction
eigenvalues reduction threadMigration
fastWalshTransform scalarProd threadMigration_kernel64.ptx
FDTD3d scan transpose
fluidsGL segmentationTreeThrust vectorAdd
freeImageInteropNPP shfl_scan vectorAddDrv
FunctionPointers simpleAssert vectorAdd_kernel64.ptx
grabcutNPP simpleAtomicIntrinsics volumeFiltering
histEqualizationNPP simpleCallback volumeRender
histogram simpleCubemapTexture

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release$ ./randomFog

And you get the following error:

./randomFog: error while loading shared libraries: libcurand.so.5.5: cannot open shared object file: No such file or directory

I originally thought this error might have something to with libglut based on other install sites I ran across. I therefore took the step of adding the symbolic link from /usr/lib/x86_64-linux-gnu to /usr/lib

user@host:~$ sudo ln -s /usr/lib/x86_64-linux-gnu/libglut.so.3 /usr/lib/libglut.so

That said, same issue:

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release$ ./randomFog
./randomFog: error while loading shared libraries: libcurand.so.5.5: cannot open shared object file: No such file or directory

I then found references to adding a cuda.conf file to /etc/ld.so.conf.d – and so did that (doesn’t help but it came up enough that I suspect it doesn’t hurt either).

user@host:~$ sudo nano /etc/ld.so.conf.d/cuda.conf 

This file should contain the following:

/usr/local/cuda-5.5/lib64
/usr/local/cuda-5.5/lib

Which also didn’t help.

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release$ ./randomFog 

./randomFog: error while loading shared libraries: libcurand.so.5.5: cannot open shared object file: No such file or directory

To find the location (or presence) of libcurand, ldconfig -v

user@host:~/$ ldconfig -v

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release$ ldconfig -v
/sbin/ldconfig.real: Path `/lib/x86_64-linux-gnu’ given more than once
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu’ given more than once
/usr/local/cuda-5.5/lib64:
libcuinj64.so.5.5 -> libcuinj64.so.5.5.22
libcufft.so.5.5 -> libcufft.so.5.5.22
libcurand.so.5.5 -> libcurand.so.5.5.22
libcusparse.so.5.5 -> libcusparse.so.5.5.22
. . .
libnvToolsExt.so.1 -> libnvToolsExt.so.1.0.0
/usr/local/cuda-5.5/lib:
libcufft.so.5.5 -> libcufft.so.5.5.22
libcurand.so.5.5 -> libcurand.so.5.5.22
libcusparse.so.5.5 -> libcusparse.so.5.5.22
. . .
/usr/lib/NVIDIA-319/tls: (hwcap: 0×8000000000000000)
libNVIDIA-tls.so.319.32 -> libNVIDIA-tls.so.319.32
/usr/lib32/NVIDIA-319/tls: (hwcap: 0×8000000000000000)
libNVIDIA-tls.so.319.32 -> libNVIDIA-tls.so.319.32
/sbin/ldconfig.real: Can’t create temporary cache file /etc/ld.so.cache~: Permission denied

Present twice. Instead of risking making multiple symbolic links as I walked through the dependency gauntlet, I stumbled across another reference in the form of a new /etc/ld.so.conf.d/NVIDIA.conf that contains the same content as cuda.conf (so one may not be needed, but I didn’t bother to backtrack to see. Happy to change the page if someone says otherwise).

user@host:~/$ sudo nano /etc/ld.so.conf.d/NVIDIA.conf

/usr/local/cuda-5.5/lib64
/usr/local/cuda-5.5/lib

Then run ldconfig.

user@host:~/$ sudo ldconfig

With that, randomFog works just fine (and you can assume that a problem in one is a problem in several. Having not taken the full symbolic link route in favor of adding to /etc/ld.so.conf.d, I’m assuming I hit most of the potential errors for the other programs.

user@host:~/NVIDIA_CUDA-5.5_Samples/NVIDIA_CUDA-5.5_Samples/bin/x86_64/linux/release$ ./randomFog 

Random Fog
==========

CURAND initialized

Random number visualization

6. Build CudaMiner

The good news is that there are only a few more steps. The bad news is that any errors you come across in your attempt to build CudaMiner that relate to NOT having done the above are (likely) not represented here, so hopefully your search was sufficiently vague.

Download CudaMiner-master.zip from Christian Buchner’s github account. Extracting CudaMiner-master.zip (with unzip, not gunzip. Damn Windows users) and running configure produces only one obvious error.

user@host:~/WHEREVER_YOU_ARE/$ cd
user@host:~/$ cd Downloads
user@host:~/Downloads$ unzip CudaMiner-master.zip 
user@host:~/Downloads$ cd CudaMiner-master/
user@host:~/Downloads/CudaMiner-master$ chmod a+wrx configure
user@host:~/Downloads/CudaMiner-master$ ./configure

checking build system type… x86_64-unknown-linux-gnu
checking host system type… x86_64-unknown-linux-gnu
checking target system type… x86_64-unknown-linux-gnu
checking for a BSD-compatible install… /usr/bin/install -c
. . .
checking for gawk… (cached) mawk
checking for curl-config… no
checking whether libcurl is usable… no
configure: error: Missing required libcurl >= 7.15.2

This error is remedied by installing libcurl4-gnutls-dev.

user@host:~/Downloads/CudaMiner-master$ sudo apt-get install libcurl4-gnutls-dev 

Which adds and modifies the following from my clean 12.04 LTS install and update

The following packages were automatically installed and are no longer required:
gir1.2-ubuntuoneui-3.0 libxcb-dri2-0 libxrandr-ltsr2 libubuntuoneui-3.0-1 libxvmc1 thunderbird-globalmenu
libllvm3.2
Use ‘apt-get autoremove’ to remove them.
The following extra packages will be installed:
comerr-dev krb5-multidev libgcrypt11-dev libgnutls-dev libgnutls-openssl27 libgnutlsxx27 libgpg-error-dev
libgssrpc4 libidn11-dev libkadm5clnt-mit8 libkadm5srv-mit8 libkdb5-6 libkrb5-dev libldap2-dev
libp11-kit-dev librtmp-dev libtasn1-3-dev zlib1g-dev
Suggested packages:
krb5-doc libcurl3-dbg libgcrypt11-doc gnutls-doc gnutls-bin krb5-user
The following NEW packages will be installed:
comerr-dev krb5-multidev libcurl4-gnutls-dev libgcrypt11-dev libgnutls-dev libgnutls-openssl27
libgnutlsxx27 libgpg-error-dev libgssrpc4 libidn11-dev libkadm5clnt-mit8 libkadm5srv-mit8 libkdb5-6
libkrb5-dev libldap2-dev libp11-kit-dev librtmp-dev libtasn1-3-dev zlib1g-dev

After a make clean, configure and make for CudaMiner went without problem.

user@host:~/Downloads/CudaMiner-master$ make clean
user@host:~/Downloads/CudaMiner-master$ ./configure

checking build system type… x86_64-unknown-linux-gnu
checking host system type… x86_64-unknown-linux-gnu
checking target system type… x86_64-unknown-linux-gnu
checking for a BSD-compatible install… /usr/bin/install -c
checking whether build environment is sane… yes
checking for a thread-safe mkdir -p… /bin/mkdir -p
. . .
configure: creating ./config.status
config.status: creating Makefile
config.status: creating compat/Makefile
config.status: creating compat/jansson/Makefile
config.status: creating cpuminer-config.h
config.status: cpuminer-config.h is unchanged
config.status: executing depfiles commands

user@host:~/Downloads/CudaMiner-master$ make

make all-recursive
make[1]: Entering directory `/home/user/Downloads/CudaMiner-master’
Making all in compat
make[2]: Entering directory `/home/user/Downloads/CudaMiner-master/compat’
Making all in jansson
make[3]: Entering directory `/home/user/Downloads/CudaMiner-master/compat/jansson’
. . .
./spinlock_kernel.cu(387): Warning: Cannot tell what pointer points to, assuming global memory space
./spinlock_kernel.cu(387): Warning: Cannot tell what pointer points to, assuming global memory space
./spinlock_kernel.cu(387): Warning: Cannot tell what pointer points to, assuming global memory space
. . .
nvcc -g -O2 -Xptxas “-abi=no -v” -arch=compute_10 –maxrregcount=64 –ptxas-options=-v -I./compat/jansson -o legacy_kernel.o -c legacy_kernel.cu
./legacy_kernel.cu(310): Warning: Cannot tell what pointer points to, assuming global memory space
./legacy_kernel.cu(310): Warning: Cannot tell what pointer points to, assuming global memory space
./legacy_kernel.cu(310): Warning: Cannot tell what pointer points to, assuming global memory space
. . .
g++ -g -O2 -pthread -L/usr/local/cuda/lib64 -o cudaminer cudaminer-cpu-miner.o cudaminer-util.o cudaminer-sha2.o cudaminer-scrypt.o salsa_kernel.o spinlock_kernel.o legacy_kernel.o fermi_kernel.o kepler_kernel.o test_kernel.o titan_kernel.o -L/usr/lib/x86_64-linux-gnu -lcurl -Wl,-Bsymbolic-functions -Wl,-z,relro compat/jansson/libjansson.a -lpthread -lcudart -fopenmp
make[2]: Leaving directory `/home/user/Downloads/CudaMiner-master’
make[1]: Leaving directory `/home/user/Downloads/CudaMiner-master’

A few warnings (well, several hundred of the same warnings) appeared during the build process (but don’t affect the program operation. Just pointing them out above).

With luck, you should be able to run a benchmark calculation immediately.

user@host:~/Downloads/CudaMiner-master$ ./cudaminer -d 0 -i 0 --benchmark

*** CudaMiner for NVIDIA GPUs by Christian Buchner ***
This is version 2013-12-18 (beta)
based on pooler-cpuminer 2.3.2 (c) 2010 Jeff Garzik, 2012 pooler
Cuda additions Copyright 2013 Christian Buchner
My donation address: LKS1WDKGED647msBQfLBHV3Ls8sveGncnm

[2013-12-25 00:05:38] 1 miner threads started, using ‘scrypt’ algorithm.
[2013-12-25 00:05:58] GPU #0: GeForce GTX 690 with compute capability 3.0
[2013-12-25 00:05:58] GPU #0: the ‘K’ kernel requires single memory allocation
[2013-12-25 00:05:58] GPU #0: interactive: 0, tex-cache: 0 , single-alloc: 1
[2013-12-25 00:05:58] GPU #0: Performing auto-tuning (Patience…)
[2013-12-25 00:05:58] GPU #0: maximum warps: 447
[2013-12-25 00:07:40] GPU #0: 288.38 khash/s with configuration K27x4
[2013-12-25 00:07:40] GPU #0: using launch configuration K27x4
[2013-12-25 00:07:40] GPU #0: GeForce GTX 690, 6912 hashes, 0.06 khash/s
[2013-12-25 00:07:40] Total: 0.06 khash/s
[2013-12-25 00:07:40] GPU #0: GeForce GTX 690, 3456 hashes, 141.56 khash/s
[2013-12-25 00:07:40] Total: 141.56 khash/s
[2013-12-25 00:07:43] GPU #0: GeForce GTX 690, 708480 hashes, 251.11 khash/s
[2013-12-25 00:07:43] Total: 251.11 khash/s
[2013-12-25 00:07:48] GPU #0: GeForce GTX 690, 1257984 hashes, 251.19 khash/s
[2013-12-25 00:07:48] Total: 251.19 khash/s
. . .

Then spend the rest of the week optimizing parameters for your particular card and mining proclivity:

user@host:~/Downloads/CudaMiner-master$ ./cudaminer -h
	   *** CudaMiner for NVIDIA GPUs by Christian Buchner ***
	             This is version 2013-12-18 (beta)
	based on pooler-cpuminer 2.3.2 (c) 2010 Jeff Garzik, 2012 pooler
	       Cuda additions Copyright 2013 Christian Buchner
	   My donation address: LKS1WDKGED647msBQfLBHV3Ls8sveGncnm

Usage: cudaminer [OPTIONS]
Options:
  -a, --algo=ALGO       specify the algorithm to use
                          scrypt    scrypt(1024, 1, 1) (default)
                          sha256d   SHA-256d
  -o, --url=URL         URL of mining server (default: http://127.0.0.1:9332/)
  -O, --userpass=U:P    username:password pair for mining server
  -u, --user=USERNAME   username for mining server
  -p, --pass=PASSWORD   password for mining server
      --cert=FILE       certificate for mining server using SSL
  -x, --proxy=[PROTOCOL://]HOST[:PORT]  connect through a proxy
  -t, --threads=N       number of miner threads (default: number of processors)
  -r, --retries=N       number of times to retry if a network call fails
                          (default: retry indefinitely)
  -R, --retry-pause=N   time to pause between retries, in seconds (default: 30)
  -T, --timeout=N       network timeout, in seconds (default: 270)
  -s, --scantime=N      upper bound on time spent scanning current work when
                          long polling is unavailable, in seconds (default: 5)
      --no-longpoll     disable X-Long-Polling support
      --no-stratum      disable X-Stratum support
  -q, --quiet           disable per-thread hashmeter output
  -D, --debug           enable debug output
  -P, --protocol-dump   verbose dump of protocol-level activities
      --no-autotune     disable auto-tuning of kernel launch parameters
  -d, --devices         takes a comma separated list of CUDA devices to use.
                        This implies the -t option with the threads set to the
                        number of devices.
  -l, --launch-config   gives the launch configuration for each kernel
                        in a comma separated list, one per device.
  -i, --interactive     comma separated list of flags (0/1) specifying
                        which of the CUDA device you need to run at inter-
                        active frame rates (because it drives a display).
  -C, --texture-cache   comma separated list of flags (0/1) specifying
                        which of the CUDA devices shall use the texture
                        cache for mining. Kepler devices will profit.
  -m, --single-memory   comma separated list of flags (0/1) specifying
                        which of the CUDA devices shall allocate their
                        scrypt scratchbuffers in a single memory block.
  -H, --hash-parallel   1 to enable parallel SHA256 hashing on the CPU. May
                        use more CPU overall, but distributes hashing load
                        neatly across all CPU cores. 0 is now the default
                        which assigns one static CPU core to each GPU.
  -S, --syslog          use system log for output messages
  -B, --background      run the miner in the background
      --benchmark       run in offline benchmark mode
  -c, --config=FILE     load a JSON-format configuration file
  -V, --version         display version information and exit
  -h, --help            display this help text and exit

I’ve only had a few problems with CudaMiner to date. The most annoying problem has been the inability to run tests to optimize card performance without having to put the machine to sleep and wake it back up again (better than a full restart). CudaMiner will, without this, simply hang on a script line:

[2013-12-25 00:49:08] 1 miner threads started, using ‘scrypt’ algorithm.

The sleep + wake does the trick, although I’d love to find out how to not have this happen.

The second annoying problem was:

“. . . result does not validate on CPU (i=NNNN, s=0)!

This error is due to your “K16x16″ configuration (the most prominent one I’ve found in google searches, so placed here to help others find it. Your values may vary) being too much for the card (so vary them down a spell until you don’t get there error). There’s a wealth of proper card settings available on the litecoin hardware comparison site, so I direct you there:

litecoin.info/Mining_hardware_comparison

7. And Finally. . .

By all accounts, CudaMiner is a much faster mining tool for NVIDIA owners. To that end, please note that Christian Buchner has made your life much easier (and your virtual wallet hopefully a little fuller). As mentioned above, his donation address is:

LKS1WDKGED647msBQfLBHV3Ls8sveGncnm

Do consider showing him some love.

This post was made in the interest of helping others get their mining going. If this guide helped and you score blocks early, my wallet’s always open as well (can’t blame someone for trying).

Bitcoin: 1P5f7GbnNW9a83zFwBLJz6kgFpkuquyeyu

Litecoin: LTmicpwpGgrZiyiJmMUdyqq4CG8CqiBqrm

Dogecoin: DBwXMoQ4scAqZfYUJgc3SYqTED7eywSHdB

The timing for getting the guide up is based on a new mining operation here in Syracuse, NY in the form of Salt City Miners, currently the Cloud City of mining operations (also appropriate for the weather conditions). Parties interested in adding their power to the fold are more than welcome to sign up at miner.saltcityminers.com/.

2013dec28_scm_logo

And don’t forget the Meetup group: Syracuse Meetup Group – Bitcoin’s of New York – Miner’s of Syracuse

Compiling LAMMPS (3Apr13, But Likely Others) In Ubuntu 10.04 Part 1. Using MPICH2 And FFTW2 (And Ubuntu Notes On Installing Intel Fortran And C++ Composers XE for Linux)

Saturday, April 6th, 2013

I’ll qualify this post by saying that (1) I have given up on Ubuntu 11.x and 12.x because they are consistently unstable on my hardware (so, if you have issues running this installation on those versions, I may not be of much help (although I suspect things should work)), (2) I am starting this install from a fresh 32-bit Desktop Ubuntu 10.04 install (so do not know if there are any issues with other software one might have installed on a Linux box if a problem comes up), and (3) the procedure comes out of the current lack of an Ubuntu binary currently listed as available (as of 6 April 2013) from the LAMMPS website (lammps.sandia.gov/download.html#ubuntu). If (3) changes and is available in an MPI form, what’s below will hopefully be unnecessary.

Building Trouble And Solutions

My initial “just unzip, untar, and make linux” attempt on a fresh 10.04 install produced the following error (which I’m reproducing in the expectation that you found this page by typing one of the errors below into a search engine, so you’ll find the error and the solutions). NOTE: I build all my programs in /opt for organizational purposes (so replace accordingly):

user@machine:/opt/lammps-3Apr13/src$ sudo make linux

make[1]: Entering directory `/opt/lammps-3Apr13/src/Obj_linux'
icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../write_restart.cpp > write_restart.d
/bin/sh: icc: not found
icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../write_data.cpp > write_data.d
/bin/sh: icc: not found
icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../verlet.cpp > verlet.d
/bin/sh: icc: not found
icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../velocity.cpp > velocity.d
/bin/sh: icc: not found
...
icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../angle_charmm.cpp > angle_charmm.d
/bin/sh: icc: not found
icc -O  -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -c ../angle_charmm.cpp
make[1]: icc: Command not found
make[1]: *** [angle_charmm.o] Error 127
make[1]: Leaving directory `/opt/lammps-3Apr13/src/Obj_linux'
make: *** [linux] Error 2

Obviously, problem Number 1 is the lack of the Intel C Compiler (icc). My solution to this was to download the non-commercial version of the Intel C++ Composer XE for Linux (then I grabbed the Intel Fortran Composer XE for Linux because, well, why not?) currently available from software.intel.com/en-us/non-commercial-software-development (which means this link may be subject to change, so search for “intel c++ noncommercial” in the event).

Unzipping, untaring, and running ./install.sh will, on a fresh Ubuntu install, give you errors that g++ and a proper Java runtime environment are not available on the computer (as part of the pre-requisite search). This is easily solved before the Intel installs by the following (one of which is needed for LAMMPS anyway). I specifically chose the openJDK, but Java 6 or 7 should also do.

sudo apt-get install build-essential openjdk-6-*

After these installs, both Intel Composers should install just fine. If you’re installing LAMMPS into a directory that you, the user, has access to, then adding /opt/intel/bin to your PATH will provide you no compiler errors (related to location). If you attempt to install LAMMPS in a directory you, the user, do not have access to, you have to run the install with sudo, which then doesn’t likely have the /opt/intel/bin directory in the root PATH, in which case the solution is simply to add a symbolic link for icc into /user/local/bin.

sudo ln -s /opt/intel/bin/icc /usr/local/bin

With icc accessible, running another sudo make linux produces the following errors:

sudo make clean-all
sudo make linux

make[1]: Entering directory `/opt/lammps-3Apr13/src/Obj_linux'
icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../write_restart.cpp > write_restart.d
../write_restart.cpp(15): catastrophic error: cannot open source file "mpi.h"
  #include "mpi.h"
                  ^

icc -O -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../write_data.cpp > write_data.d
../write_data.cpp(15): catastrophic error: cannot open source file "mpi.h"
  #include "mpi.h"
                  ^
...

icc -O  -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -c ../angle_charmm.cpp
../pointers.h(25): catastrophic error: cannot open source file "mpi.h"
  #include "mpi.h"
                  ^

compilation aborted for ../angle_charmm.cpp (code 4)
make[1]: *** [angle_charmm.o] Error 4
make[1]: Leaving directory `/opt/lammps-3Apr13/src/Obj_linux'
make: *** [linux] Error 2

The new problem is now an MPI problem, requiring the installation of (as recommended from the LAMMPS linux Makefile) MPICH. Reading the Makefile (you mean you didn’t check this first?) also indicates the need for FFTW2. We can install all of the needed files in one shot with apt-get (and I include build-essential here for completeness, as you either did or didn’t install it while trying to solve the icc problem above).

sudo apt-get install build-essential mpich-bin libmpich1.0-dev mpi-doc fftw2 fftw-dev libxaw7-dev libmpich2-dev

This still does not solve the mpi.h issue above, which is a problem with the settings in Makefile.linux (which, apparently, are not Ubuntu-compatible). The solution is, and this is why this is Part 1, to change Makefile.linux to make it compatible with the mpicc compiler in Ubuntu (I will attempt to get it working with icc next). The tweak to my Makefile.linux is provided below (an amalgam of some other Makefile settings. Comment out or delete what’s there and add the sections without the “#” to the file):

#######################################################################
## My modified settings are below:

CC = mpicxx
CCFLAGS = -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK
DEPFLAGS =	-M

LINK = $(CC)
LINKFLAGS = -O
SIZE = size

USRLIB = -lfftw

ARCHIVE =	ar
ARFLAGS =	-rc

#######################################################################
## Comment the following out or delete them:

# CC =		icc
# CCFLAGS =	-O
# SHFLAGS =	-fPIC
# DEPFLAGS =	-M

# LINK =		icc
# LINKFLAGS =	-O
# LIB =           -lstdc++
# SIZE =		size

# ARCHIVE =	ar
# ARFLAGS =	-rc
# SHLIBFLAGS =	-shared

I will say that, in some of my searches, I found reference to the use of USRLIB -lmpi. If this is included in the Makefile.linux, you will get the following error upon mpicc (MPICH2) compilation:

/usr/bin/ld: cannot find -lmpl
collect2: ld returned 1 exit status
make[1]: *** [../lmp_linux] Error 1
make[1]: Leaving directory `/opt/lammps-3Apr13/src/Obj_linux'
make: *** [linux] Error 2

So, don’t include -lmpi.

With the above Makefile modifications, building of LAMMPS seems to go quite well until the following error is produced:

mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK  -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -c ../dump_dcd.cpp
mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK  -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -c ../dump_image.cpp
../dump_image.cpp:32:21: error: jpeglib.h: No such file or directory
../dump_image.cpp: In member function ‘virtual int LAMMPS_NS::DumpImage::modify_param(int, char**)':
../dump_image.cpp:904: warning: suggest parentheses around assignment used as truth value
../dump_image.cpp:911: warning: suggest parentheses around assignment used as truth value
../dump_image.cpp:964: warning: suggest parentheses around assignment used as truth value
../dump_image.cpp:971: warning: suggest parentheses around assignment used as truth value
../dump_image.cpp: In member function ‘void LAMMPS_NS::DumpImage::create_image()':
../dump_image.cpp:611: warning: ‘diameter' may be used uninitialized in this function
../dump_image.cpp:612: warning: ‘color' may be used uninitialized in this function
../dump_image.cpp:612: warning: ‘color1' may be used uninitialized in this function
../dump_image.cpp:612: warning: ‘color2' may be used uninitialized in this function
make[1]: *** [dump_image.o] Error 1
make[1]: Leaving directory `/opt/lammps-3Apr13/src/Obj_linux'
make: *** [linux] Error 2

The jpeglib.h error can likely be solved by installing only one of the libraries below, but I didn’t bother to identify which one, instead indiscriminately installing a whole set based on the recommendation of dev.xonotic.org/projects/xonotic/wiki/Repository_Access (having found this link at forums.xonotic.org/showthread.php?tid=1252 – but I removed some of the install that I’m sure was not needed):

sudo apt-get install libxxf86dga-dev libxcb-xf86dri0-dev libxpm-dev libxxf86vm-dev libsdl1.2-dev libsdl-image1.2-dev libclalsadrv-dev libasound2-dev libxext-dev

Finally, a successful build!

make[1]: Entering directory `/opt/lammps-3Apr13/src/Obj_linux'
mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../write_restart.cpp > write_restart.d
mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../write_data.cpp > write_data.d
mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../verlet.cpp > verlet.d
mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../velocity.cpp > velocity.d
mpicxx -O -DFFT_FFTW -DLAMMPS_GZIP -DMPICH_IGNORE_CXX_SEEK -DLAMMPS_GZIP -DLAMMPS_JPEG  -DMPICH_SKIP_MPICXX  -DFFT_FFTW   -M ../variable.cpp > variable.d
…
write_data.o write_restart.o  -lmpich -lpthread -lfftw -ljpeg   -o ../lmp_linux
size ../lmp_linux
   text	   data	    bss	    dec	    hex	filename
3654398	   6928	    264	3661590	 37df16	../lmp_linux
make[1]: Leaving directory `/opt/lammps-3Apr13/src/Obj_linux'

At this point, you can move the lmp_linux executable anywhere (or, as I find myself doing, just keep it in the lammps-DATE/src folder for when you perform subsequent builds).

Running An MPI Calculations

If you’ve never run through an MPI calculation, you’ll get the following errors when you first try to run lmp_linux:

cannot connect to local mpd (/tmp/mpd2.console_user); possible causes:
  1. no mpd is running on this host
  2. an mpd is running but was started without a "console" (-n option)
In case 1, you can start an mpd on this host with:
    mpd &
and you will be able to run jobs just on this host.
For more details on starting mpds on a set of hosts, see
the MPICH2 Installation Guide.

**********

user@machine:/opt/lammps-3Apr13/src$ mpd &

configuration file /home/damianallis/.mpd.conf not found
A file named .mpd.conf file must be present in the user's home
directory (/etc/mpd.conf if root) with read and write access
only for the user, and must contain at least a line with:
MPD_SECRETWORD=
One way to safely create this file is to do the following:
  cd $HOME
  touch .mpd.conf
  chmod 600 .mpd.conf
and then use an editor to insert a line like
  MPD_SECRETWORD=mr45-j9z
into the file.  (Of course use some other secret word than mr45-j9z.)

[1]+  Exit 255                mpd

Simply follow the instructions to put a .mdp.conf in your ~/ folder, start mpd with “mpd &,” and run LAMMPS with the following command below (which uses 2 cores (-np 2)).

NOTE: My LAMMPS test (being brand new to it) was to run the first test available from icme.hpc.msstate.edu/mediawiki/index.php/LAMMPS_Help. Following the instructions therein (DOWNLOAD: Al99.eam.alloy (780 KB) and calc_fcc.in (1 KB)):

sudo mpirun -np 2 ./lmp_linux < calc_fcc.in

Which should produce the following output:

LAMMPS (3 Apr 2013)
Lattice spacing in x,y,z = 4 4 4
Created orthogonal box = (0 0 0) to (4 4 4)
  1 by 1 by 2 MPI processor grid
Lattice spacing in x,y,z = 4 4 4
Created 4 atoms
Replicating atoms ...
  orthogonal box = (0 0 0) to (4 4 4)
  1 by 1 by 2 MPI processor grid
  4 atoms
WARNING: Resetting reneighboring criteria during minimization (../min.cpp:173)
Setting up minimization ...
Memory usage per processor = 2.40372 Mbytes
Step PotEng Lx Ly Lz Press Pxx Pyy Pzz eatoms 
       0   -13.417787            4            4            4     29590.11     29590.11     29590.11     29590.11   -13.417787 
      10   -13.439104         4.04         4.04         4.04    5853.9553    5853.9553    5853.9553    5853.9553   -13.439104 
      14       -13.44         4.05         4.05         4.05     2.726913     2.726913     2.726913     2.726913       -13.44 
Loop time of 0.0287241 on 2 procs for 14 steps with 4 atoms

Minimization stats:
  Stopping criterion = linesearch alpha is zero
  Energy initial, next-to-last, final = 
        -13.4177872966     -13.4399999525     -13.4399999525
  Force two-norm initial, final = 3.54599 0.000335006
  Force max component initial, final = 3.54599 0.000335006
  Final line search alpha, max atom move = 0.0625 2.09379e-05
  Iterations, force evaluations = 14 19

Pair  time (%) = 0.00206506 (7.18931)
Neigh time (%) = 0 (0)
Comm  time (%) = 0.00481999 (16.7803)
Outpt time (%) = 0.000162005 (0.564006)
Other time (%) = 0.021677 (75.4664)

Nlocal:    2 ave 2 max 2 min
Histogram: 2 0 0 0 0 0 0 0 0 0
Nghost:    603 ave 603 max 603 min
Histogram: 2 0 0 0 0 0 0 0 0 0
Neighs:    140 ave 162 max 118 min
Histogram: 1 0 0 0 0 0 0 0 0 1

Total # of neighbors = 280
Ave neighs/atom = 70
Neighbor list builds = 0
Dangerous builds = 0
Total energy (eV) = -13.439999952539944061;
Number of atoms = 4;
Lattice constant (Angstoms) = 4.049999999999998046;
Cohesive energy (eV) = -3.3599999881349860154;
All done!

Brief Update: Amber 11 And AmberTools 1.5 In Ubuntu 12.04 LTS

Saturday, June 30th, 2012

This post is a brief update to a much longer and more involved discussion of Amber 11 and AmberTools installation in Ubuntu 10.04 LTS (Lucid Lynx) (as the changes are minor and the parallelization setup remains largely the same). You can find this more involved discussion at www.somewhereville.com/?p=1422.

Long/Short – the installation under Ubuntu 12.04 LTS (Precise Pangolin) is not much different and goes without hitch provided you keep your locations organized. NOTE 1: I’ve not a copy of Amber12, so cannot speak for any changes to its installation procedure. NOTE 2: This install assumes 32-bit only.

Updated Procedure

If you tried installing all of the build software from the 10.04 LTS post, you’ll receive errors like the following (as usual, I include error messages for those who are searching against error messages)…

user@machine:~$ sudo apt-get install build-essential cmake doxygen freeglut3-dev g++-multilib gcc-multilib gettext gnuplot ia32-libs lib32asound2 lib32gcc1 lib32gcc1-dbg lib32gfortran3 lib32gomp1 lib32mudflap0 lib32ncurses5 lib32nss-mdns lib32z1 libavdevice52 libc6-dev-i386 libc6-i386 libfreeimage-dev libglew1.5-dev libopenal1 libopenexr-dev libpng12-dev libqt4-dev libssl-dev libstdc++6-4.3-dbg libstdc++6-4.3-dev libstdc++6-4.3-doc libxi-dev libxml-simple-perl libxmu-dev mercurial nfs-common nfs-kernel-server portmap python2.6-dev rpm ssh

Errors…

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'rpcbind' instead of 'portmap'
Package libc6-i386 is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
However the following packages replace it:
  libc6

Package libstdc++6-4.3-dbg is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

Package libstdc++6-4.3-doc is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

...

Package lib32nss-mdns is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'ia32-libs' has no installation candidate
E: Package 'lib32asound2' has no installation candidate
E: Package 'lib32gcc1' has no installation candidate
E: Package 'lib32gcc1-dbg' has no installation candidate
E: Package 'lib32gfortran3' has no installation candidate
E: Package 'lib32gomp1' has no installation candidate
E: Package 'lib32mudflap0' has no installation candidate
E: Package 'lib32ncurses5' has no installation candidate
E: Package 'lib32nss-mdns' has no installation candidate
E: Package 'lib32z1' has no installation candidate
E: Unable to locate package libavdevice52
E: Package 'libc6-dev-i386' has no installation candidate
E: Package 'libc6-i386' has no installation candidate
E: Package 'libstdc++6-4.3-dbg' has no installation candidate
E: Package 'libstdc++6-4.3-dev' has no installation candidate
E: Package 'libstdc++6-4.3-doc' has no installation candidate
E: Unable to locate package python2.6-dev
E: Couldn't find any package by regex 'python2.6-dev'

The actual install list is smaller than above (reproduced below) to install the necessary additionals to the base 12.04 LTS install. The procedure begins with an aptitude install, update, and upgrade (perform or do not perform as you like).

administrator@ChemistryLab:~$ sudo apt-get install aptitude
administrator@ChemistryLab:~$ sudo aptitude update
administrator@ChemistryLab:~$ sudo aptitude upgrade
administrator@ChemistryLab:~$ sudo apt-get install build-essential cmake doxygen freeglut3-dev g++-multilib gcc-multilib gettext gnuplot ia32-libs bison csh flex fort77 g++ gcc gfortran libbz2-dev libnetcdf-dev libopenmpi-dev libxext-dev libxt-dev openmpi-bin patch tcsh xorg-dev zlib1g-dev fftw-dev 

Amber11 and AmberTools 1.5 Install Specifics

Those used to sudo’ing everything will run into a compile complication with Amber11 as the $AMBERHOME assignment is not carried over into the make serial step:

make[1]: Entering directory `/opt/amber11/AmberTools/src/mmpbsa_py'
/bin/bash setup.sh
AMBERHOME is not set.  Assuming it is /opt/amber11
 Using AmberTools' python
Error importing MMPBSA python modules! MMPBSA.py will not work.
make[1]: *** [install] Error 1
make[1]: Leaving directory `/opt/amber11/AmberTools/src/mmpbsa_py'
make: *** [serial] Error 2

This is an avoidable annoyance by simply building in your $HOME directory and copying the resulting ~/amber11 folder to your desired location (and setting $PATH statements accordingly).

AmberTools 1.5 Install

Procedure is as found in the previous post (I am assuming that the files are sitting in your $HOME (cd ~/) fold…

tar xjf AmberTools-1.5.tar.bz2
cd amber11/
echo "export AMBERHOME=$PWD" >> ~/.bashrc
echo "export PATH=$PATH:$AMBERHOME/bin" >> ~/.bashrc
source ~/.bashrc
wget http://ambermd.org/bugfixes/AmberTools/1.5/bugfix.all
patch -p0 < bugfix.all
rm bugfix.all
cd AmberTools/src/
./configure gnu
make install

The Amber11 install is made slightly different than the previous instruction set by the download of the bugfix.all.tar.bz2 file and the different run of apply_bugfix.x. For a serial install...

tar xfj Amber11.tar.bz2
cd ~/
wget http://ambermd.org/bugfixes/11.0/bugfix.all.tar.bz2
wget http://ambermd.org/bugfixes/11.0/apply_bugfix.x
chmod +x ./apply_bugfix.x
./apply_bugfix.x bugfix.all.tar.bz2

You’ll get a few CUDA-specific errors during the bugfix. I skipped the last two patches as I’m not compiling a CUDA-specific version.

cd AmberTools/src/
./configure gnu
cd $AMBERHOME
./AT15_Amber11.py
cd src/
make serial

With this completed, move the amber11 folder to /opt (or wherever), modify your .bashrc, and run the tests.

cd ~/
sudo mv amber11 /opt
nano .bashrc

Place the following into the .bashrc file

export AMBERHOME=/opt/amber11

And run the tests…

cd /opt/amber11/test/
make -f Makefile

Obligatory

  • CNYO

  • Sol. Sys. Amb.

  • Salt City Miners

  • Ubuntu 4 Nano

  • NMT Review

  • N-Fact. Collab.

  • T R P Nanosys

  • Nano Gallery

  • nano gallery
  • Aerial Photos

    More @ flickr.com

    Syracuse Scenes

    More @ flickr.com