More On The Virtues Of VirtualBox – ACID (or AICD) Under Ubuntu 14.04 (By Way Of OpenSuse 11.2)

“Stop that!” – George Carlin

If you’ve obtained source code from an academic lab that was last developed some time ago and you spent a whole day installing libraries and symbolic links and redefining variables in your .bashrc and downgrading libraries and redefining paths and have 20 tabs open in your browser that all go to 20 different obscure error discussions on Stack Overflow and it’s late and you’re tired and you think you might not need the program after all if you do a bunch of other workaround things instead – what’s below is for you.

Academics have been developing small code for (nearly) millions of years to make their lives easier – and we all benefit when that code is made available to others (esp. when it helps in data analysis). When that code is a series of perl or python scripts, there’s generally little reason why you should have any run issues. When they call on external libraries or specific tools, generally that information is available in the README somewhere. Generally speaking, there’s no reason why a code shouldn’t work in a straightforward manner when the developer doesn’t make it known that something else needs be installed to make it work.

So, why doesn’t code A work on your linux box? A few possibilities.

0. You didn’t read the README right.

1. The developer skipped some documentation and you’re missing a library or program that they installed months earlier but forgot about (and they forgot to acknowledge that that was a needed item to run their program).

2. Your linux distro is too new, they developed it on something much older, have never updated their own machine, and something fundamental changed between, for instance, Ubuntu 8.04 and 14.04 (I’ve run into a few in that category – depending on the code, this can sometimes just be chalked up to a compiler difference issue that went unnoticed by the developers).

3. The one you learn the harder way – Ubuntu is not Fedora is not OpenSuse is not [insert distro here]. Half of the Ubuntu posts on this website stem from my having needed to do something under Ubuntu to make it look a little tiny bit more like whatever distro a developer developed their code with.

Let’s hammer on point 3 for a minute – I know precious few academics who use computers as part of research who are both (1) experts in coding and (2) experts in interpreting the output of some code. A properly working code in the hands of a productive researcher can mean the difference between an OK interpretation of data and a very descriptive interpretation of data that anyone else could follow (which, let’s not lie to ourselves, may also mean the difference between a journal with an impact factor of 0.5 and one of 5.0). Disregarding any “it was hard to code, it should be hard to use!” mentality, finding out that a program runs painlessly on distro A with NO effort after trying in vain on distro B with CONSIDERABLE effort is a rather important piece of information.

For the record – this isn’t about a user’s ability to program, install, admin their machine, etc. It’s simply about ease-of-use for ALL users regardless of their computational aptitude (which is why many of the Ubuntu posts on this website are exhaustive in their reporting of errors as well as work-arounds).

Which brings me to one of the most versatile (and storage-consuming) programs on my Ubuntu 14.04 machine – VirtualBox. VirtualBox (and related) are absolutely awesome for their abilities to let you try – very quickly – the quick fix that is the building or running of a strange code “somewhere else.” For the record, I ALWAYS try something new using a clean Ubuntu 14.04 image under VirtualBox before I ever let it touch my main machine – and that’s because I’ve had more than one instance in the past of trying to build a piece of software that then destroyed (well, sort of) my machine. When you destroy an image with a bad installation attempt – all you have to do is trash the image and copy a fresh one over from some “clean image” folder. Could not be easier. Furthermore, I’ve still ONE app that works in Windows XP that doesn’t work anywhere else. And I’ve two or three under Windows that I use generally – all ready for use under VirtualBox without having to reboot anything.

Additionally importantly – Virtual Machines allow you to “sample in time” – working around new & improved features and functionality in new compilers and libraries by instead building code on what was available at the time of the code’s release.

Anisotropy of the Current (Induced) Density – ACID

2016mar29_benzol.nmr_40000_0.050_1_0_0_Aniso_4

Sample output (from a successful Gaussian09 benzene tutorial run).

From the website of Prof. Dr. Rainer Herges

The “density of delocalized electrons” is a concept that is intuitively used to explain the electronic structure of conjugated systems. Unfortunately, however, there is no rigorous way to separate the total electron density in a density of localized and delocalized electrons. Like aromaticity, bond order, point charge, and other important concepts of chemistry, a definition for the density of delocalized electrons has to be derived from more fundamental quantum theoretical parameters.

As observables, magnetic properties are a suitable starting point for a general description of of delocalization and conjugation. In analogy to the anisotropy of the magnetic susceptibility, which is a powerful measure of aromaticity, we investigate the anisotropy of the current (induced) density (ACID). Similar to the square of the wavefunction which defines the total electron density, the ACID scalar field defines the density of delocalized electrons.

The ACID method proved to be an extremely versatile and descriptive tool to analyze delocalization in ground-state molecules, excited states, transition states, organometallics, hyperconjugation and other through-bond and through-space interactions.

Having spent quite some time determining the best way to account for electron delocalization in an upcoming project, the ACID method (“Delocalization of Electrons in Molecules,” J. Phys. Chem. A 2001, 105, 3214-3220. dx.doi.org/10.1021/jp0034426) struck me as near-ideal, leading to my request for the code from the developers.

With the code graciously provided, my first run attempt under Ubuntu 14.04 (and Cygwin, and Ubuntu 10.04 because it was waaaaay back when) went less-than successfully after the successful Gaussian run.

Ubuntu 14.04 make Results (Note: Using gcc 4.8)

user@hostname:~/AICD-2.0.0/$ make

sh modify_shellpath_in_scripts.sh AICD AICD-extract.pl ModifyPov.pl AICD-convert.pl
wd=`pwd` ; \
	sed -i -e "2s+^AICD_BaseDir=.*+AICD_BaseDir=$wd+" AICD
g++ -O2 -ffast-math -march=native -o AICD-smooth_isosurface.o -c AICD-smooth_isosurface.cpp
g++ AICD-smooth_isosurface.o -Wl,-s -o AICD-smooth_isosurface
g++ -O2 -ffast-math -march=native -o AICD-isosurface.o -c AICD-isosurface.cpp
g++ AICD-isosurface.o -Wl,-s -o AICD-isosurface
g++ -O2 -ffast-math -march=native -o AICD-extract.o -c AICD-extract.cpp
g++ AICD-extract.o -Wl,-s -o AICD-extract
g++ -O2 -ffast-math -march=native -o AICD-cube.o -c AICD-cube.cpp
g++ AICD-cube.o -Wl,-s -o AICD-cube
g++ -O2 -ffast-math -march=native -o AICD-remap.o -c AICD-remap.cpp
g++ AICD-remap.o -Wl,-s -o AICD-remap
g++ -O2 -ffast-math -march=native -o AICD-opt_remap.o -c AICD-opt_remap.cpp
g++ AICD-opt_remap.o -Wl,-s -o AICD-opt_remap
g++ -O2 -ffast-math -march=native -o AICD-rotate_mol.o -c AICD-rotate_mol.cpp
g++ AICD-rotate_mol.o -Wl,-s -o AICD-rotate_mol
g++ -O2 -ffast-math -march=native -o AICD-isocut.o -c AICD-isocut.cpp
g++ AICD-isocut.o -Wl,-s -o AICD-isocut
cc -O2 -ffast-math -march=native -o povchem/povchem.o -c povchem/povchem.c
povchem/povchem.c: In function ‘Get_User_Input’:
povchem/povchem.c:1173:10: warning: ignoring return value of ‘scanf’, declared with attribute warn_unused_result [-Wunused-result]
     scanf("%c",&string[i]);
          ^
povchem/povchem.c: In function ‘main’:
povchem/povchem.c:2958:13: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
       system(tmp);
             ^
povchem/povchem.c:2965:17: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
           system(tmp);
                 ^
povchem/povchem.c:2985:15: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
         system(tmp);
               ^
povchem/povchem.c:2989:15: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
         system(tmp);
               ^
povchem/povchem.c:2994:15: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
         system(tmp);
               ^
povchem/povchem.c:3003:15: warning: ignoring return value of ‘system’, declared with attribute warn_unused_result [-Wunused-result]
         system(tmp);
               ^
g++ povchem/povchem.o -Wl,-s -o povchem/povchem

Warnings are not errors, so meh. Google searching indicated a code issue (version issue) that someone on Stack Overflow proposed fixing by turning off optimization (removing -O2 from the Makefile). The error goes away.

sh modify_shellpath_in_scripts.sh AICD AICD-extract.pl ModifyPov.pl AICD-convert.pl
wd=`pwd` ; \
	sed -i -e "2s+^AICD_BaseDir=.*+AICD_BaseDir=$wd+" AICD
g++ -O2 -ffast-math -march=native -o AICD-smooth_isosurface.o -c AICD-smooth_isosurface.cpp
g++ AICD-smooth_isosurface.o -Wl,-s -o AICD-smooth_isosurface
g++ -O2 -ffast-math -march=native -o AICD-isosurface.o -c AICD-isosurface.cpp
g++ AICD-isosurface.o -Wl,-s -o AICD-isosurface
g++ -O2 -ffast-math -march=native -o AICD-extract.o -c AICD-extract.cpp
g++ AICD-extract.o -Wl,-s -o AICD-extract
g++ -O2 -ffast-math -march=native -o AICD-cube.o -c AICD-cube.cpp
g++ AICD-cube.o -Wl,-s -o AICD-cube
g++ -O2 -ffast-math -march=native -o AICD-remap.o -c AICD-remap.cpp
g++ AICD-remap.o -Wl,-s -o AICD-remap
g++ -O2 -ffast-math -march=native -o AICD-opt_remap.o -c AICD-opt_remap.cpp
g++ AICD-opt_remap.o -Wl,-s -o AICD-opt_remap
g++ -O2 -ffast-math -march=native -o AICD-rotate_mol.o -c AICD-rotate_mol.cpp
g++ AICD-rotate_mol.o -Wl,-s -o AICD-rotate_mol
g++ -O2 -ffast-math -march=native -o AICD-isocut.o -c AICD-isocut.cpp
g++ AICD-isocut.o -Wl,-s -o AICD-isocut
cc -O2 -ffast-math -march=native -o povchem/povchem.o -c povchem/povchem.c
g++ povchem/povchem.o -Wl,-s -o povchem/povchem

And the tutorial run output itself is below – complete with error.

$ AICD -m 2 -rot 90 0 0 -b 1 0 0 -runpov benzol.nmr.log

Datei benzol.nmr.log wird bearbeitet.
rufe jetzt AICDcube auf

1. Dateienpaar: benzol.nmr.icd40000 und benzol.nmr.pdb

Molek?linformationen:
6 0 0 0 2.6192
6 0 0 2.2677 1.3096
6 0 0 2.2677 -1.3096
6 0 0 0 -2.6192
6 0 0 -2.2677 -1.3096
6 0 0 -2.2677 1.3096
1 0 0 0 4.6525
1 0 0 4.0289 2.3263
1 0 0 4.0289 -2.3263
1 0 0 0 -4.6525
1 0 0 -4.0289 -2.3263
1 0 0 -4.0289 2.3263

40128 = 19 * 44 * 48
Grenzen:
-0.066811 < = Isotropie <= 0.066811
0 <= Anisotropie <= 1.1042
-2.0667 <= X <= 2.0667
-4.7203 <= Y <= 4.7203
-5.1407 <= Z <= 5.1407

Cube-file wird geschrieben. Bitte warten.

1. Input-file: benzol.nmr.icd40000
40128 = 19 * 44 * 48
Grenzen:
-0.0668113 <= Isotropie <= 0.0668113
0 <= Anisotropie <= 1.10423
-2.06672 <= X <= 2.06672
-4.72031 <= Y <= 4.72032
-5.14072 <= Z <= 5.14072

Punkte auf der Isooberfl?che
Z = 0
...................
...................
[…]
...................
...................

Es wurden 2604 Pfeile generiert.
Davon zeigen 1302 in die Isooberfl?che hinein (50%).

Mittelwert der Pfeill?nge: 3.65747
Pfeilstatistik:
0  0  0  0  0  0  0  0  0  0  0  2604  
Dreiecke werden generiert. Bitte warten.
Povray-input wird geschrieben. Bitte warten.
Bildgrenzen:
0 <= Isotropie <= 0
0.05 <= Anisotropie <= 0.05
-1.4586 <= X <= 1.4586
-4.29131 <= Y <= 4.29131
-4.64666 <= Z <= 4.64666

producing benzol.nmr.pov

Welcome to PovChem! This is version 1.00. For general
instructions see http://ludwig.scs.uiuc.edu/~paul/Manual.html

Read configuration file "/home/damianallis/AICD-2.0.0/povray-AICD-templates/povchem.cfg".
Read periodic table "/home/damianallis/AICD-2.0.0/povray-AICD-templates/periodic.tab".

Found 12 atoms...
...loaded into memory.
Wrote 12 spheres...
...and read 24 bonds into memory.

Wrote 0 single, 12 double, 0 triple, 0 higher order, and 0 hydrogen bonds.

benzol.nmr.inc created.
Writing color and atom definitions...
Writing bond definitions...
Pov header benzol.nmr.pov created.


modifying benzol.nmr.pov to benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov
*** Error in `AICD-rotate_mol': free(): invalid next size (fast): 0x0000000001f22160 ***
/opt/AICD-2.0.0/AICD: line 656: 17519 Aborted                 (core dumped) $Animation $inputbasename.pdb 
$Atom1 $Atom2 $Atom3 < "$RenderMich" > $Pov_RenderMich_pov
Replacing Camera Postion to \<0, 0, -250\>
Inserting #version directive in RenderMich.pov
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.inc
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.RenderMich.pov
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Rotate.inc
calling AICD-isocut -m -1 -M -1 -r 10000000 < benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc.noncut > benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc
This is AICD-isocut....
 arrow_maxlength = -1
 arrow_minlength = -1
 Point of origin:  0 0 0
 Maximum distance: 1e+07
Number of deleted arrows: 0
povray: cannot open the user configuration file /home/damianallis/.povray/3.7/povray.conf: No such file or directory
Persistence of Vision(tm) Ray Tracer Version 3.7.0.unofficial (g++ 4.8 @
x86_64-pc-linux-gnu)
This is an unofficial version compiled by:
Felix Geyer  for Debian 
The POV-Ray Team is not responsible for supporting this version.

POV-Ray is based on DKBTrace 2.12 by David K. Buck & Aaron A. Collins
Copyright 1991-2013 Persistence of Vision Raytracer Pty. Ltd.

Primary POV-Ray 3.7 Architects/Developers: (Alphabetically)
 Chris Cason         Thorsten Froehlich  Christoph Lipka   

...

Support libraries used by POV-Ray:
 ZLib 1.2.8, Copyright 1995-2012 Jean-loup Gailly and Mark Adler
 LibPNG 1.2.50, Copyright 1998-2012 Glenn Randers-Pehrson
 LibJPEG 80, Copyright 1991-2013 Thomas G. Lane, Guido Vollbeding
 LibTIFF 4.0.3, Copyright 1988-1997 Sam Leffler, 1991-1997 SGI
 Boost 1.54, http://www.boost.org/
 OpenEXR, Copyright (c) 2004-2007, Industrial Light & Magic.

Parser Options
 Input file: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.RenderMich.pov
 Remove bounds........On 
 Split unions.........Off
 Library paths:
   /usr/share/povray-3.7
   /usr/share/povray-3.7/ini
   /usr/share/povray-3.7/include
   /usr/lib/povray3
   /usr/lib/povray3/include
 Clock value:    0.000  (Animation off)
Image Output Options
 Image resolution.....1024 by 768 (rows 1 to 768, columns 1 to 1024).
 Output file..........benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.png, 24 bpp PNG
 Dithering............Off
 Graphic display......Off
 Mosaic preview.......Off
 Continued trace......Off
Information Output Options
 All Streams to console..........On 
 Debug Stream to console.........On 
 Fatal Stream to console.........On 
 Render Stream to console........On 
 Statistics Stream to console....On 
 Warning Stream to console.......On 
==== [Parsing...] ==========================================================
Parse Warning: No objects in scene.
Parse Warning: assumed_gamma not specified, so gamma_correction is turned off
for compatibility with this pre POV-Ray 3.7 scene. See the documentation for
more details.
Parse Warning: This scene did not contain a #version directive. Please be aware
that as of POV-Ray 3.7, unless already specified via an INI option, a #version
is expected as the first declaration in a scene file. POV-Ray may apply
settings to some features that are intended to maintain compatibility with
pre-3.7 scenes. You are strongly encouraged to add a #version statement to the
scene to make your intent clear. Future versions of POV-Ray may make the
presence of a #version statement mandatory.
----------------------------------------------------------------------------
Parser Statistics
----------------------------------------------------------------------------
Finite Objects:            0
Infinite Objects:          0
Light Sources:             0
Total:                     0
----------------------------------------------------------------------------
Parser Time
 Parse Time:       0 hours  0 minutes  0 seconds (0.000 seconds)
             using 1 thread(s) with 0.000 CPU-seconds total
 Bounding Time:    0 hours  0 minutes  0 seconds (0.000 seconds)
             using 1 thread(s) with 0.000 CPU-seconds total
----------------------------------------------------------------------------
Render Options
 Quality:  9
 Bounding boxes.......On   Bounding threshold: 3
 Antialiasing.........Off
==== [Rendering...] ========================================================
Rendered 786432 of 786432 pixels (100%)
----------------------------------------------------------------------------
Render Statistics
Image Resolution 1024 x 768
----------------------------------------------------------------------------
Pixels:           786432   Samples:               0   Smpls/Pxl: 0.00
Rays:             786432   Saved:                 0   Max Level: 1/5
----------------------------------------------------------------------------
Ray->Shape Intersection          Tests       Succeeded  Percentage
----------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
----------------------------------------------------------------------------
Render Time:
 Photon Time:      No photons
 Radiosity Time:   No radiosity
 Trace Time:       0 hours  0 minutes  0 seconds (0.031 seconds)
             using 12 thread(s) with 0.143 CPU-seconds total
POV-Ray finished

*******************************************************************************************************
*******************************************************************************************************

Not a particularly Ubuntu-looking problem, and not one for which anything online was available that helped at all. The .pov file is written, but the file is empty. Googling the error above…

*** Error in `AICD-rotate_mol’: free(): invalid next size (fast): 0x0000000001f22160 ***
/opt/AICD-2.0.0/AICD: line 656: 17519 Aborted … (core dumped) $Animation $inputbasename.pdb

… sends you to, among other pages, one Stack Overflow discussion that starts with the error and ends with recommendations of finding a good debugger – not something I’d any interest in pursuing as part of getting this code to work.

I did not find any version-dependence discussion with the errors above, but opted to try using an older gcc – that I wasn’t about to install on my new Ubuntu box. Ergo, find an older distro that installs an older gcc. I threw it on an OpenSUSE 11.2 image because the developer group is in Germany – that’s basically why – and because I couldn’t find an old enough Ubuntu distro that would update/upgrade/install apps from the current repositories.

OpenSUSE 11.2 make Results (Note: Using gcc 4.5)

From a fresh OpenSUSE 11.2 install, I added the appropriate developer tools to get the ACID code made (note the specific request for gcc 4.5):

sudo zypper install autoconf automake binutils-gold gcc gcc45 glibc-devel kernel-desktop-devel kernel-devel kernel-source linux-glibc-devel m4 make gcc-c++

Then added povray (and nano, but that’s because I hate vi)

sudo zypper install povray nano

The build process then goes as follows – with no issues.

sh modify_shellpath_in_scripts.sh AICD AICD-extract.pl ModifyPov.pl AICD-convert.pl
wd=`pwd` ; \
	sed -i -e "2s+^AICD_BaseDir=.*+AICD_BaseDir=$wd+" AICD
g++ -O2 -ffast-math -march=native -o AICD-smooth_isosurface.o -c AICD-smooth_isosurface.cpp
g++ AICD-smooth_isosurface.o -Wl,-s -o AICD-smooth_isosurface
g++ -O2 -ffast-math -march=native -o AICD-isosurface.o -c AICD-isosurface.cpp
g++ AICD-isosurface.o -Wl,-s -o AICD-isosurface
g++ -O2 -ffast-math -march=native -o AICD-extract.o -c AICD-extract.cpp
g++ AICD-extract.o -Wl,-s -o AICD-extract
g++ -O2 -ffast-math -march=native -o AICD-cube.o -c AICD-cube.cpp
g++ AICD-cube.o -Wl,-s -o AICD-cube
g++ -O2 -ffast-math -march=native -o AICD-remap.o -c AICD-remap.cpp
g++ AICD-remap.o -Wl,-s -o AICD-remap
g++ -O2 -ffast-math -march=native -o AICD-opt_remap.o -c AICD-opt_remap.cpp
g++ AICD-opt_remap.o -Wl,-s -o AICD-opt_remap
g++ -O2 -ffast-math -march=native -o AICD-rotate_mol.o -c AICD-rotate_mol.cpp
g++ AICD-rotate_mol.o -Wl,-s -o AICD-rotate_mol
g++ -O2 -ffast-math -march=native -o AICD-isocut.o -c AICD-isocut.cpp
g++ AICD-isocut.o -Wl,-s -o AICD-isocut
cc -O2 -ffast-math -march=native -o povchem/povchem.o -c povchem/povchem.c
g++ povchem/povchem.o -Wl,-s -o povchem/povchem

And then there’s the run through the tutorial again…

Datei benzol.nmr.log wird bearbeitet.
Weder benzol.nmr.icd40000 noch benzol.nmr.icd40000.gz existiert.
Weder benzol.nmr.icd noch benzol.nmr.icd.gz existieren.
Starting AICDextractsh
Separate AICD-output in file: test.txt
Extrahiere_Gitterpunkte

1. Input-file: benzol.nmr.icd-raw
Es wurden 84726 Stromdichtetensoren eingelesen.

Grenzen:
-0.0668113 < = Isotropie <= 0.0668113
0 <= Anisotropie <= 1.10423
-20.3137 <= X <= 20.3137
-23.6411 <= Y <= 23.6411
-24.2643 <= Z <= 24.2643

Atomposition:
0.0214824  
0  
2.61948  

Radius: 26.8838
Datei benzol.nmr wird bearbeitet.
Weder benzol.nmr.icd40000 noch benzol.nmr.icd40000.gz existiert.

Input-file: benzol.nmr.icd

Output-file: benzol.nmr.icd40000
========================
 1. Iteration
========================
AICD-remap 40000 1 - benzol.nmr.icd40000 benzol.nmr.icd 

Input-file: benzol.nmr.icd
Anzahl der Stromdichtetensoren: 84726
Grenzen:
-0.0668113 <= Isotropie <= 0.0668113
0 <= Anisotropie <= 1.10423
-10.3137 <= X <= 10.3137
-13.6411 <= Y <= 13.6411
-14.2643 <= Z <= 14.2643


Grenzen des kartesischen Netzes:
-0.0668113 <= Isotropie <= 0.0668113
0 <= Anisotropie <= 1.10423
-10.3137 <= X <= 10.3137
-13.6411 <= Y <= 13.6411
-14.2643 <= Z <= 14.2643

Globales Netz: 40404 = 28 * 37 * 39

Input-file: benzol.nmr.icd

1 %
2 %
3 %

...

Z = 47
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................
...................

Es wurden 2604 Pfeile generiert.
Davon zeigen 1302 in die Isooberfl‰che hinein (50%).

Mittelwert der Pfeill‰nge: 0.0642683
Pfeilstatistik:
0  32  80  52  124  268  476  716  412  260  144  40  
Dreiecke werden generiert. Bitte warten.
Povray-input wird geschrieben. Bitte warten.
Bildgrenzen:
0 <= Isotropie <= 0
0.05 <= Anisotropie <= 0.05
-1.4586 <= X <= 1.4586
-4.29131 <= Y <= 4.29131
-4.64666 <= Z <= 4.64666

producing benzol.nmr.pov

Welcome to PovChem! This is version 1.00. For general
instructions see http://ludwig.scs.uiuc.edu/~paul/Manual.html

Read configuration file "/home/damianallis/ACID/AICD-2.0.0/povray-AICD-templates/povchem.cfg".
Read periodic table "/home/damianallis/ACID/AICD-2.0.0/povray-AICD-templates/periodic.tab".

Found 12 atoms...
 ...loaded into memory.
Wrote 12 spheres...
 ...and read 24 bonds into memory.

Wrote 0 single, 12 double, 0 triple, 0 higher order, and 0 hydrogen bonds.

benzol.nmr.inc created.
Writing color and atom definitions...
Writing bond definitions...
Pov header benzol.nmr.pov created.


modifying benzol.nmr.pov to benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov
Replacing Camera Postion to \<0, 0, -250\>
Inserting #version directive in RenderMich.pov
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.inc
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.RenderMich.pov
calling Replace_Pov_Filenames benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Rotate.inc
calling AICD-isocut -m -1 -M -1 -r 10000000 < benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc.noncut > benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc
This is AICD-isocut....
  arrow_maxlength = -1
  arrow_minlength = -1
  Point of origin:  0 0 0
  Maximum distance: 1e+07
Number of deleted arrows: 0
povray: cannot open the user configuration file /home/damianallis/.povray/3.6/povray.conf: No such file or directory
Persistence of Vision(tm) Ray Tracer Version 3.6.1 (g++ @ x86_64-unknown-linux-g
nu)
This is an unofficial version compiled by:
 SUSE LINUX Products GmbH, Nuernberg, Germany
 The POV-Ray Team(tm) is not responsible for supporting this version.
POV-Ray is based on DKBTrace 2.12 by David K. Buck & Aaron A. Collins
Copyright 1991-2003 Persistence of Vision Team
Copyright 2003-2004 Persistence of Vision Raytracer Pty. Ltd.

...

Support libraries used by POV-Ray:
  ZLib 1.2.5, Copyright 1995-1998 Jean-loup Gailly and Mark Adler
  LibPNG 1.4.4, Copyright 1998-2002 Glenn Randers-Pehrson
  LibJPEG 6b, Copyright 1998 Thomas G. Lane
  LibTIFF 3.9.4, Copyright 1988-1997 Sam Leffler, 1991-1997 SGI
Redirecting Options
  All Streams to console..........On 
  Debug Stream to console.........On 
  Fatal Stream to console.........On 
  Render Stream to console........On 
  Statistics Stream to console....On 
  Warning Stream to console.......On 
Parsing Options
  Input file: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.RenderMich.pov (compatible
 to version 3.61)
  Remove bounds........On 
  Split unions.........Off
  Library paths:
    /usr/share/povray-3.6
    /usr/share/povray-3.6/ini
    /usr/share/povray-3.6/include
    /usr/lib/povray3
    /usr/lib/povray3/include
Output Options
  Image resolution 1024 by 768 (rows 1 to 768, columns 1 to 1024).
  Output file: /home/damianallis/ACID/AICD-2.0.0/tutorial-data/benzol.nmr_40000_
0.050_1_0_0_Aniso_4.2.png, 24 bpp PNG
  Graphic display......Off
  Mosaic preview.......Off
  CPU usage histogram..Off
  Continued trace......Off
Tracing Options
  Quality:  9
  Bounding boxes.......On   Bounding threshold: 3
  Light Buffer.........On 
  Vista Buffer.........On   Draw Vista Buffer....Off
  Antialiasing.........Off
  Clock value:    0.000  (Animation off)

  0:00:00 Parsing
File: /usr/share/povray-3.6/include/glass_old.inc  Line: 18
Parse Warning:  Due to changes in version 3.1, you must add interior {I_Glass}
 to all objects calling glass_old.inc textures and finishes... 
File: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov  Line: 1
Possible Parse Error: All #version and #declares of float, vector, and color
 require semi-colon ';' at end.
File: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov  Line: 2
Possible Parse Error: All #version and #declares of float, vector, and color
 require semi-colon ';' at end.
File: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Molekuel.pov  Line: 21
Possible Parse Error: All #version and #declares of float, vector, and color
 require semi-colon ';' at end.
File: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc  Line: 9
Possible Parse Error: Text may not be displayed as expected. Please refer to the
 user manual regarding changes in POV-Ray 3.5 and later.
File: benzol.nmr_40000_0.050_1_0_0_Aniso_4.2.Isoober.inc  Line: 17
Possible Parse Error: Text may not be displayed as expected. Please refer to the
 user manual regarding changes in POV-Ray 3.5 and later.

  0:00:01 Creating bounding slabs
  0:00:01 Creating vista buffer
  0:00:01 Creating light buffers
  0:00:01 Creating light buffers
  0:00:01 Creating light buffers 508K tokens
Scene Statistics
  Finite objects:         5233
  Infinite objects:          0
  Light sources:             4
  Total:                  5237

  0:00:00 Rendering line 1 of 768
  0:00:01 Rendering line 237 of 768
  0:00:02 Rendering line 292 of 768
  0:00:03 Rendering line 383 of 768
  0:00:04 Rendering line 479 of 768
  0:00:05 Rendering line 538 of 768
  0:00:05 Done Tracing
Render Statistics
Image Resolution 1024 x 768
----------------------------------------------------------------------------
Pixels:           786432   Samples:          786432   Smpls/Pxl: 1.00
Rays:             786432   Saved:                 0   Max Level: 1/5
----------------------------------------------------------------------------
Ray->Shape Intersection          Tests       Succeeded  Percentage
----------------------------------------------------------------------------
Cone/Cylinder                  1927296          164391      8.53
Mesh                            862314          164798     19.11
Sphere                           21764            5562     25.56
Bounding Box                  39215172        13715953     34.98
Light Buffer                  60713588        16726656     27.55
Vista Buffer                  11930298         6504493     54.52
----------------------------------------------------------------------------
Calls to Noise:                   0   Calls to DNoise:              10
----------------------------------------------------------------------------
Shadow Ray Tests:           1215156   Succeeded:                109903
----------------------------------------------------------------------------
Smallest Alloc:                   9 bytes
Largest  Alloc:             2097160 bytes
Peak memory used:          24172005 bytes
Total Scene Processing Times
  Parse Time:    0 hours  0 minutes  1 seconds (1 seconds)
  Photon Time:   0 hours  0 minutes  0 seconds (0 seconds)
  Render Time:   0 hours  0 minutes  5 seconds (5 seconds)
  Total Time:    0 hours  0 minutes  6 seconds (6 seconds)

Producing the beautiful and informative image shown above (despite the Parse Warning and Possible Parse Errors above).

For the record, compiler version differences can be subtle beasts – and my whole discussion above is not directly related to getting ACID to work. The above is a mix of brute force and resource consumption that provided a much faster turnaround in positive results than any amount of work spent in Ubuntu itself to make ACID run successfully – and does not require modifying anyone’s code (an added bonus for those who might have to ask questions of the developers).

So, I would posit that the issue turns out to be a 4.5 vs. 4.8 thing, but I’ve not explored the issue further (frankly, no time – and it’s working – so I don’t know at what point in gcc development that the make process began to produce errors in the ACID code). This isn’t technically the way to run it natively in Ubuntu, but is a way to get it to run in Ubuntu, which still counts in my book. In theory, a VirtualBox install and some book work on finding the dates of certain distros provides yet another way of getting stuff to just work.

That said, getting OpenSUSE to play with the Networking in VirtualBox appears to be a real pain. My solution for that for getting files back-and-forth to process?

DropBox.

OrthoDB 1.6 Installation On Ubuntu 14.04 (And Related) – Build Errors And The Simple Fixes

UPDATE: 20 May 2016 – With thanks to OrthoDB’s very own Fredrik Tegenfeldt, here’s a more distro-complete fix for the Makefile.rules file.

# BRH flags
ifeq ($(USE_BRH),1)
BRH_DIR        = $(BASE_DIR)
CPPFLAGS    += -I$(BRH_DIR)/src
ifeq ($(WITH_RPATH),1)
LDFLAGS        += -L$(LIB_DIR) -Wl,-rpath,$(LIB_INSTDIR)
else
LDFLAGS        += -L$(LIB_DIR)
endif
LDLIBS        += -lbrh
endif

ifeq ($(USE_BOOST_THREAD),1)
CPPFLAGS    += -pthread -DHAS_BOOST_THREADS
LDFLAGS        += -pthread
LDLIBS      += -L$(DIR_BOOST_LIB) -lboost_system -L$(DIR_BOOST_THREAD) -l$(LIB_BOOST_THREAD) -lpthread
endif

And Happy New Year,

Yet another random Ubuntu-centric bioinformatics aside in the event others run into the same build issues (with errors included below, as you likely googled those first). For those wondering…

The Hierarchical Catalog of Orthologs v8

Orthology (website, download) is the cornerstone of comparative genomics and gene function prediction. OrthoDB aims to classify protein-coding genes from the increasing number of available sequenced genomes into groups of orthologs descended from a single gene of the last common ancestor (LCA) of each clade of species. Applying this concept to the hierarchy of LCAs along the species phylogeny results in multiple levels of orthology: the more closely-related the species, the more finely-resolved the orthologous relations.

The build here was on a fresh installation of 64-bit Ubuntu 14.04 (Trusty Tahr). All of the errors produced come from running on that clean install, meaning you’ll run into dumb errors (like missing build-essential), didn’t-know-we-needed-that errors (boost), and that’s-probbly-an-Ubuntu-oddity errors (with a modified Makefile.rules file with explicit boost calls linked to below; I suspect the developers are working on a non-Ubuntu distro).

Relevant apt-get’s

sudo apt-get install build-essential libboost-all-dev

Relevant Downloads

* tclap (1.2.1) – tclap.sourceforge.net/

Stage 1 – Just Trying The Easy Install

I build everything in /opt by default and propose you do the same (whatever). First steps included.

username@ubuntu:~$ cd Downloads/
username@ubuntu:~/Downloads$ sudo mv OrthoDB_soft_1.6.tgz /opt/
username@ubuntu:~/Downloads$ cd /opt/
username@ubuntu:/opt$ sudo tar xvf OrthoDB_soft_1.6.tgz 

OrthoDB_soft_1.6/
OrthoDB_soft_1.6/BRHCLUS-1.12.7/
...
OrthoDB_soft_1.6/README
OrthoDB_soft_1.6/LICENSE
username@ubuntu:/opt$ cd OrthoDB_soft_1.6/
username@ubuntu:/opt/OrthoDB_soft_1.6$ sudo make WITH_RPATH=1 prefix=$(pwd) all
make: *** No rule to make target `all'.  Stop.

If you read the README file, you figure out to move to the BRHCLUS folder, then discover the need for the boost libraries, available via an apt-get:

username@ubuntu:/opt/OrthoDB_soft_1.6$ cd BRHCLUS-1.12.7/
username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo ./configure.sh 
FATAL: cannot find boost libraries!
- Is boost installed (boost_system) ?
- Is boost installed in any of the search paths ?
  If not, add the path to PATH_SEARCH_LIB at the top of this script.

Stage 2 – sudo apt-get install libboost-all-dev

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo apt-get install libboost-all-dev 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  account-plugin-windows-live libupstart1
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  icu-devtools libboost-atomic-dev libboost-atomic1.54-dev
  libboost-atomic1.54.0 libboost-chrono-dev libboost-chrono1.54-dev
...
  libxml2-dev mpi-default-bin mpi-default-dev ocl-icd-libopencl1 openmpi-bin
  openmpi-common zlib1g-dev
0 upgraded, 102 newly installed, 0 to remove and 14 not upgraded.
Need to get 46.7 MB of archives.
After this operation, 262 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main libboost-atomic1.54.0 amd64 1.54.0-4ubuntu3.1 [8,076 B]
Get:2 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main libboost-chrono1.54.0 amd64 1.54.0-4ubuntu3.1 [12.5 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/universe libboost-context1.54.0 amd64 1.54.0-4ubuntu3.1 [8,028 B]
Get:4 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main libboost-filesystem1.54.0 amd64 1.54.0-4ubuntu3.1 [34.2 kB]
...
Setting up libboost-wave-dev:amd64 (1.54.0.1ubuntu1) ...
Setting up libboost-all-dev (1.54.0.1ubuntu1) ...
Setting up libhwloc-plugins (1.8-1ubuntu1.14.04.1) ...
Processing triggers for libc-bin (2.19-0ubuntu6.6) ...

The configure stage now works…

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo ./configure.sh 
OK - boost libraries
OK - boost thread libraries
OK - boost include dir
OK - tclap include dir
OK - thread pool include dir
Config file (/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/Makefile.config) successfully created

But, right off the bat, a clean install is missing build tools.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make WITH_RPATH=1 prefix=$(pwd) all
  Compiling brh_version.cpp
/bin/sh: 1: g++: not found
make[2]: *** [.linux/brh_version.o] Error 127
make[1]: *** [src] Error 2
make: *** [all] Error 2

Stage 3 – sudo apt-get install build-essential

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo apt-get install build-essential 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  account-plugin-windows-live libupstart1
...
0 upgraded, 9 newly installed, 0 to remove and 14 not upgraded.
Need to get 15.9 MB of archives.
After this operation, 27.9 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main g++-4.8 amd64 4.8.4-2ubuntu1~14.04 [15.0 MB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ trusty/main g++ amd64 4:4.8.2-1ubuntu6 [1,490 B]
Get:3 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main dpkg-dev all 1.17.5ubuntu5.5 [726 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ trusty/main build-essential amd64 11.6ubuntu6 [4,838 B]
...
update-alternatives: using /usr/bin/fakeroot-sysv to provide /usr/bin/fakeroot (fakeroot) in auto mode
Setting up libalgorithm-diff-perl (1.19.02-3) ...
Setting up libalgorithm-diff-xs-perl (0.04-2build4) ...
Setting up libalgorithm-merge-perl (0.08-2) ...

And the build process now takes steps forward.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make WITH_RPATH=1 prefix=$(pwd) all
  Compiling brh_version.cpp
  Compiling brh_common.cpp
  Compiling pairdataset.cpp
  Compiling brh_functions.cpp
  Compiling keymap.cpp
  Compiling inparalog.cpp
  Compiling clusterdata.cpp
  Compiling cluster.cpp
  Compiling clusterize.cpp
  Compiling cluster_functions.cpp
  Compiling modularity.cpp
  Compiling tri4algo.cpp
  Linking libbrh.so
  Compiling brhclus.cpp
  Linking brhclus
.linux/brhclus.o: In function `__static_initialization_and_destruction_0':
/usr/include/boost/system/error_code.hpp:222: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:223: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:224: undefined reference to `boost::system::system_category()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::detail::get_current_thread_data()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::disable_interruption::disable_interruption()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::interruption_point()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::start_thread_noexcept()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::interrupt()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::hiden::sleep_until(timespec const&)'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::join_noexcept()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `vtable for boost::detail::thread_data_base'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::detail::thread_data_base::~thread_data_base()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::native_handle()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::detach()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::disable_interruption::~disable_interruption()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `typeinfo for boost::detail::thread_data_base'
collect2: error: ld returned 1 exit status
make[2]: *** [/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/bin/brhclus] Error 1
make[1]: *** [tools/brhclus] Error 2
make: *** [all] Error 2

Stage 4 – tclap Install

A quick google search produces the following from info.circ.rochester.edu/BlueHive/Software/Bioinformatics/orthodb.html indicating that tclap is an OrthoDB dependency that needs to be satisfied. The version I’ve built below (1.2.1) can be downloaded from tclap.sourceforge.net

username@ubuntu:/opt$ ls
OrthoDB_soft_1.6  OrthoDB_soft_1.6.tgz  tclap-1.2.1.tar.gz

Extract and expand.

username@ubuntu:/opt$ sudo tar xvf tclap-1.2.1.tar.gz 
tclap-1.2.1/
tclap-1.2.1/aclocal.m4
tclap-1.2.1/AUTHORS
tclap-1.2.1/ChangeLog
...
tclap-1.2.1/config/Makefile.am
tclap-1.2.1/config/Makefile.in
tclap-1.2.1/config/missing
tclap-1.2.1/config/mkinstalldirs

The build process goes without issue.

username@ubuntu:/opt$ cd tclap-1.2.1/
username@ubuntu:/opt/tclap-1.2.1$ sudo ./configure 
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
...
config.status: creating tests/Makefile
config.status: creating config/config.h
config.status: executing depfiles commands
config.status: executing default commands

Onto make…

username@ubuntu:/opt/tclap-1.2.1$ sudo make
Making all in include
make[1]: Entering directory `/opt/tclap-1.2.1/include'
Making all in tclap
make[2]: Entering directory `/opt/tclap-1.2.1/include/tclap'
make[2]: Nothing to be done for `all'.
...
make[1]: Leaving directory `/opt/tclap-1.2.1/config'
make[1]: Entering directory `/opt/tclap-1.2.1'
make[1]: Nothing to be done for `all-am'.
make[1]: Leaving directory `/opt/tclap-1.2.1'

And make install…

username@ubuntu:/opt/tclap-1.2.1$ sudo make install
Making install in include
make[1]: Entering directory `/opt/tclap-1.2.1/include'
Making install in tclap
make[2]: Entering directory `/opt/tclap-1.2.1/include/tclap'
...
test -z "/usr/local/lib/pkgconfig" || /bin/mkdir -p "/usr/local/lib/pkgconfig"
 /usr/bin/install -c -m 644 'tclap.pc' '/usr/local/lib/pkgconfig/tclap.pc'
make[2]: Leaving directory `/opt/tclap-1.2.1'
make[1]: Leaving directory `/opt/tclap-1.2.1'

Stage 5 – Re-attempt The OrthoDB Build

The configure goes without issue again…

username@ubuntu:/opt$ cd OrthoDB_soft_1.6/
username@ubuntu:/opt/OrthoDB_soft_1.6$ cd BRHCLUS-1.12.7/
username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make clean
username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo ./configure.sh 
OK - boost libraries
OK - boost thread libraries
OK - boost include dir
OK - tclap include dir
OK - thread pool include dir
Config file (/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/Makefile.config) successfully created

But there’s still an error at the make stage.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make all
  Compiling brh_version.cpp
  Compiling brh_common.cpp
  Compiling pairdataset.cpp
  Compiling brh_functions.cpp
  Compiling keymap.cpp
  Compiling inparalog.cpp
  Compiling clusterdata.cpp
  Compiling cluster.cpp
  Compiling clusterize.cpp
  Compiling cluster_functions.cpp
  Compiling modularity.cpp
  Compiling tri4algo.cpp
  Linking libbrh.so
  Compiling brhclus.cpp
  Linking brhclus
.linux/brhclus.o: In function `__static_initialization_and_destruction_0':
/usr/include/boost/system/error_code.hpp:222: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:223: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:224: undefined reference to `boost::system::system_category()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::detail::get_current_thread_data()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::disable_interruption::disable_interruption()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::interruption_point()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::start_thread_noexcept()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::interrupt()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::hiden::sleep_until(timespec const&)'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::join_noexcept()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `vtable for boost::detail::thread_data_base'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::detail::thread_data_base::~thread_data_base()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::native_handle()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::detach()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::disable_interruption::~disable_interruption()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `typeinfo for boost::detail::thread_data_base'
collect2: error: ld returned 1 exit status
make[2]: *** [/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/bin/brhclus] Error 1
make[1]: *** [tools/brhclus] Error 2
make: *** [all] Error 2

Stage 6 – Google Says Make Some Symbolic Links

A first fix for the boost error was to make symbolic links for libboost_thread.so and libboost_timer.so in /usr/lib.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ cd /usr/lib/x86_64-linux-gnu/
username@ubuntu:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_thread.so /usr/lib
username@ubuntu:/usr/lib/x86_64-linux-gnu$ sudo ln -s libboost_timer.so /usr/lib
username@ubuntu:/usr/lib/x86_64-linux-gnu$ cd /opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/
username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make clean
username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo ./configure.sh 
OK - boost libraries
OK - boost thread libraries
OK - boost include dir
OK - tclap include dir
OK - thread pool include dir
Config file (/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/Makefile.config) successfully created

But this didn’t work either.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make all
  Compiling brh_version.cpp
  Compiling brh_common.cpp
  Compiling pairdataset.cpp
  Compiling brh_functions.cpp
  Compiling keymap.cpp
  Compiling inparalog.cpp
  Compiling clusterdata.cpp
  Compiling cluster.cpp
  Compiling clusterize.cpp
  Compiling cluster_functions.cpp
  Compiling modularity.cpp
  Compiling tri4algo.cpp
  Linking libbrh.so
  Compiling brhclus.cpp
  Linking brhclus
.linux/brhclus.o: In function `__static_initialization_and_destruction_0':
/usr/include/boost/system/error_code.hpp:222: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:223: undefined reference to `boost::system::generic_category()'
/usr/include/boost/system/error_code.hpp:224: undefined reference to `boost::system::system_category()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::detail::get_current_thread_data()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::disable_interruption::disable_interruption()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::interruption_point()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::start_thread_noexcept()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::interrupt()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::hiden::sleep_until(timespec const&)'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::join_noexcept()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `vtable for boost::detail::thread_data_base'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::detail::thread_data_base::~thread_data_base()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::native_handle()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::thread::detach()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `boost::this_thread::disable_interruption::~disable_interruption()'
/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib/libbrh.so: undefined reference to `typeinfo for boost::detail::thread_data_base'
collect2: error: ld returned 1 exit status
make[2]: *** [/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/bin/brhclus] Error 1
make[1]: *** [tools/brhclus] Error 2
make: *** [all] Error 2

Stage 7 – The Boost Issue May Be Ubuntu-Specific

The final solution was to modify the Makefile.rules file to explicitly call on boost in Ubuntu. Make a backup copy of the Makefile.rules in the OrthoDB folder, then place this new Makefile.rules file into the same folder. The modified Makefile.rules is below for copy+paste, else you can download it here: 2016mar25_orthodb_Makefile_rules.txt (just rename the file and place into OrthoDB_soft_1.6/BRHCLUS-1.12.7).

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo nano Makefile.rules 
#
# Build rules
#

# Keep secondary files
.SECONDARY:

include $(ROOT_DIR)/Makefile.config
ifeq ($(strip $(DIR_TCLAP)),)
$(error "Build requires TCLAP package (http://tclap.sourceforge.net)")
endif

# if install, read build details
ifeq "$(MAKE-TARGET)" "install"
include $(ROOT_DIR)/.makefile.build
endif

# Object directory
OBJ_DIR		= .linux

# Use gcc4
CXX		= g++

# Use rpath or not
WITH_RPATH	?= 1

LDFLAGS		= 
ifeq "$(MAKE-TARGET)" "static"
WITH_RPATH	= 0
LDFLAGS		+= -static -lboost_system -lboost_thread -lpthread
endif

# check BOOST
USE_BOOST ?= 1

# check boost threads
USE_BOOST_THREAD ?= 1

ifeq ($(strip $(LIB_BOOST_THREAD)),)
USE_BOOST_THREAD = 0
USE_THREADPOOL = 0
endif

ifeq ($(strip $(DIR_THREADPOOL)),)
USE_THREADPOOL = 0
endif

# Library and binary directories
BASE_DIR	:= $(shell cd $(ROOT_DIR) ; pwd)

# Install directories
# With rpath, enforce install dir same as build
# Without, use same as build as default
# prefix - install topdir
# libdir/bindir - install lib/bin
# BASE_DIR - build top dir
# LIB_DIR/BIN_DIR - build top dir
#
# with rpath :
#   build and install dirs are the same
#   default top is BASE_DIR
# else :
#   build is always under BASE_DIR
#   install under prefix (default BASE_DIR)
#
#ifeq ($(WITH_RPATH),1)
#prefix		?= $(BASE_DIR)
#libdir		= $(prefix)/lib
#bindir		= $(prefix)/bin
##LIB_DIR		:= $(shell mkdir -p $(libdir) ; cd $(libdir) ; pwd)
##BIN_DIR		:= $(shell mkdir -p $(bindir) ; cd $(bindir) ; pwd)
#LIB_DIR		:= $(libdir)
#BIN_DIR		:= $(bindir)
#else
#LIB_DIR		:= $(shell mkdir -p $(BASE_DIR)/lib ; cd $(BASE_DIR)/lib ; pwd)
#BIN_DIR		:= $(shell mkdir -p $(BASE_DIR)/bin ; cd $(BASE_DIR)/bin ; pwd)
#prefix		?= $(BASE_DIR)
#libdir		?= $(LIB_DIR)
#bindir		?= $(BIN_DIR)
#endif
LIB_DIR		:= $(shell mkdir -p $(BASE_DIR)/lib ; cd $(BASE_DIR)/lib ; pwd)
BIN_DIR		:= $(shell mkdir -p $(BASE_DIR)/bin ; cd $(BASE_DIR)/bin ; pwd)
prefix		?= $(BASE_DIR)
libdir		?= $(LIB_DIR)
bindir		?= $(BIN_DIR)
LIB_INSTDIR := $(prefix)/lib
BIN_INSTDIR := $(prefix)/bin

# Install user/group
INST_USR	 = $(shell whoami)
INST_GRP	?= $(shell groups | cut -f 1 -d ' ' )

# Establish install command
# if rpath -> always chown
# else
#   if targetdir==installdir -> chown
#   else install
# endif
#ifeq ($(WITH_RPATH),1)
#INST_CMD	 = chown $(INST_USR).$(INST_GRP)
#else
TARGET_DIR	= $(dir $(TARGET))
rpTARGET	= $(realpath $(firstword $(TARGET_DIR)))
rpINSTDIR	= $(realpath $(prefix)/$(INSTDIR))
ifeq ($(rpTARGET),$(rpINSTDIR))
INST_CMD	 = chown $(INST_USR).$(INST_GRP)
else
INST_CMD	 = mkdir -p $(prefix)/$(INSTDIR); install -D -m 755 -g $(INST_GRP) -t $(prefix)/$(INSTDIR)
endif
#endif

# Definitions concerning build environment
CPPFLAGS	+= -DBRH_BUILD_ENV="\"$(shell uname -s -r -v -m -i -p -o)\""
CPPFLAGS	+= -DBRH_BUILD_TIME="\"$(shell date)\""

# Default pre-processor flags
CPPFLAGS	+= -DBASE_DIR="\"$(BASE_DIR)\""
CPPFLAGS	+= -D_REENTRANT
CPPFLAGS	+= -D_XOPEN_SOURCE

# c++ standard
CPPFLAGS	+= -std=c++11
CXXFLAGS	+= -std=c++11

# debug flag
ifeq ($(USE_DEBUG),3)
CXXFLAGS	+= -fno-inline -fbounds-check
else ifeq ($(USE_DEBUG),2)
CPPFLAGS	+= -DBRH_DEBUG
CXXFLAGS	+= -fno-inline -fbounds-check
else ifeq ($(USE_DEBUG),1)
CPPFLAGS	+= -DBRH_CHECK_IT
CXXFLAGS	+= -finline-functions -fbounds-check
else
CXXFLAGS	+= -finline-functions
endif

# Default compiler flags
CXXFLAGS	+= -fPIC
CXXFLAGS	+= -MMD
CXXFLAGS	+= -g
CXXFLAGS	+= -Wall -Werror
CXXFLAGS	+= -fexceptions
#CXXFLAGS	+= -fno-inline
CXXFLAGS	+= -O3
#CXXFLAGS	+= -O0
CXXFLAGS	+= -I$(BASE_DIR)/src
#CXXFLAGS	+= -std=c++11
#CXXFLAGS	+= -pg
#LDFLAGS		+= -pg

# BRH flags
ifeq ($(USE_BRH),1)
BRH_DIR		= $(BASE_DIR)
CPPFLAGS	+= -I$(BRH_DIR)/src
ifeq ($(WITH_RPATH),1)
LDFLAGS		+= -L$(LIB_DIR) -Wl,-rpath,$(LIB_INSTDIR) -lboost_system -lboost_thread -lpthread
else
LDFLAGS		+= -L$(LIB_DIR) -lboost_system -lboost_thread -lpthread
endif
LDLIBS		+= -lbrh -lboost_system -lboost_thread -lpthread
endif

ifeq ($(USE_BOOST_THREAD),1)
CPPFLAGS	+= -pthread -DHAS_BOOST_THREADS
LDFLAGS		+= -pthread -L$(DIR_BOOST_LIB) -lboost_system -lboost_thread -lpthread -L$(DIR_BOOST_THREAD) -l$(LIB_BOOST_THREAD)
endif

ifeq ($(USE_THREADPOOL),1)
CPPFLAGS	+= -DHAS_BOOST_THREADPOOL
CPPFLAGS	+= -I$(DIR_THREADPOOL)
endif

ifeq ($(USE_BOOST),1)
CPPFLAGS	+= -I$(DIR_BOOST_INC)
endif

# TCLAP flags
ifeq ($(USE_TCLAP),1)
CPPFLAGS	+= -I$(DIR_TCLAP)
endif

# BOOST flags
ifeq ($(USE_THREAD),1)
endif

# Check file type
ifeq ($(suffix $(firstword $(SOURCES))),.cpp)
IS_CPP		= 1
else
IS_CPP		= 0
endif

# List of objects
ifeq ($(IS_CPP),1)
OBJECTS		= $(SOURCES:%.cpp=$(OBJ_DIR)/%.o)
endif

# Default build
default:	all

ifeq ($(IS_CPP),1)
# Build a binary
$(BIN_DIR)/%: $(OBJECTS) Makefile $(BASE_DIR)/Makefile.rules
	@echo "  Linking $(@F)"
	@mkdir -p $(@D)
	@$(LINK.cc) $(LDFLAGS) -o $@ $(OBJECTS) $(LDLIBS)

# Build a shared library
$(LIB_DIR)/%.so: $(OBJECTS) Makefile $(BASE_DIR)/Makefile.rules
	@echo "  Linking $(@F)"
	@mkdir -p $(@D)
	@$(LINK.cc) -shared -o $@ $(OBJECTS) $(LDLIBS)

# Build a static library
$(LIB_DIR)/%.a: $(OBJECTS) Makefile $(BASE_DIR)/Makefile.rules
	@echo "  Linking $(@F)"
	@mkdir -p $(@D)
	@ar rcs $@ $(OBJECTS)

# Include dependency files
-include $(OBJECTS:%.o=%.d)

# Build an object file
$(OBJ_DIR)/%.o: %.cpp Makefile
	@echo "  Compiling $< "
	@mkdir -p $(@D)
	@$(COMPILE.cc) -o $@ $<
else
$(BIN_DIR)/%.py: %.py
	@echo "  Copying $< to target dir"
	@cp -f $< $@
endif



# Main targets are phony
.PHONY: default all clean_target clean_obj doc clean_doc clean install static

# Build target
all:	$(TARGET)

static:	$(TARGET)

# Clean objects
clean_obj:
	@$(RM) -r $(OBJ_DIR)

# Clean build
clean_target:
	@$(RM) -r $(OBJ_DIR) $(TARGET)

# Build documentation
doc:
	@$(MAKE) -C $(BASE_DIR) $@

# Clean build
clean: clean_target

# Install
install:
#	@echo "rpT   : $(rpTARGET)"
#	@echo "rpI   : $(rpINSTDIR)"
#	@echo "rpath : $(WITH_RPATH)"
#	@echo "inst  : $(INST_CMD) $(TARGET)"
	@target="$(TARGET)"; \
	[ $${#target} -gt 0 ] || target="$(OUTPUT)"; \
	for f in $${target}; do \
	    fname=`basename $$f`; \
	    if [ -f $$f ]; then \
	       echo "Installing in $(prefix)/$(INSTDIR)    <$$fname>"; \
	       $(INST_CMD) $$f; \
	    fi; \
	done

The configure step runs fine…

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make clean
username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo ./configure.sh 
OK - boost libraries
OK - boost thread libraries
OK - boost include dir
OK - tclap include dir
OK - thread pool include dir
Config file (/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/Makefile.config) successfully created

…and the build itself runs fine.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make all
  Compiling brh_version.cpp
  Compiling brh_common.cpp
  Compiling pairdataset.cpp
  Compiling brh_functions.cpp
  Compiling keymap.cpp
  Compiling inparalog.cpp
  Compiling clusterdata.cpp
  Compiling cluster.cpp
  Compiling clusterize.cpp
  Compiling cluster_functions.cpp
  Compiling modularity.cpp
  Compiling tri4algo.cpp
  Linking libbrh.so
  Compiling brhclus.cpp
  Linking brhclus
  Compiling evolrate.cpp
  Linking evolrate
  Compiling statbrh.cpp
  Linking statbrh
  Compiling brh2x.cpp
  Linking brh2x
  Compiling statclus.cpp
  Linking statclus
  Compiling cmpclus.cpp
  Linking cmpclus
  Compiling getclus.cpp
  Linking getclus
  Compiling cmpbrh.cpp
  Linking cmpbrh
  Compiling cmpinpar.cpp
  Linking cmpinpar
  Compiling renumclus.cpp
  Linking renumclus
  Compiling getinpar.cpp
  Linking getinpar
  Compiling mergeinpar.cpp
  Linking mergeinpar
  Compiling align2brh.cpp
  Linking align2brh
  Compiling findbrh.cpp
  Linking findbrh
  Copying parsedump.py to target dir
  Copying getbrh.py to target dir
  Copying plot_brhclus_progress.py to target dir

Stage 8 – Build The Tests

Just to make sure all is working, make globaltest.

username@ubuntu:/opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7$ sudo make globaltest 

-------------------------------------------------------------
Binary  dir : /opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/bin
Library dir : /opt/OrthoDB_soft_1.6/BRHCLUS-1.12.7/lib
-------------------------------------------------------------

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST MACHINE
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
uname [ Linux ubuntux 3.16.0-59-generic #79~14.04.1-Ubuntu SMP Mon Jan 18 15:41:27...]
cpu   [ Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz]
mem   [ MemTotal:        3949740 kB]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS
with option --single
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------
run default setup
------------------------------
------------------------------
run with harder e-value cuts
------------------------------
------------------------------
run with soft e-value cuts
------------------------------
------------------------------
Check output files
------------------------------
default                                [OK]
hard_e-val                             [OK]
soft_e-val                             [OK]
------------------------------
Check timing
------------------------------
default                                [16 %]
hard_e-val                             [-8 %]
soft_e-val                             [10 %]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS, resume function
with option --single
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
resume                                 [OK, contents same but clids or seqs differ]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS
with option --nostringent
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------
run default setup
------------------------------
------------------------------
run with harder e-value cuts
------------------------------
------------------------------
run with soft e-value cuts
------------------------------
------------------------------
Check output files
------------------------------
default                                [OK]
hard_e-val                             [OK]
soft_e-val                             [OK]
------------------------------
Check timing
------------------------------
default                                [18 %]
hard_e-val                             [10 %]
soft_e-val                             [2 %]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS, resume function
with option --nostringent
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
resume                                 [OK, contents same but clids or seqs differ]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------
run default setup
------------------------------
------------------------------
run with harder e-value cuts
------------------------------
------------------------------
run with soft e-value cuts
------------------------------
------------------------------
Check output files
------------------------------
default                                [OK]
hard_e-val                             [OK]
soft_e-val                             [OK]
------------------------------
Check timing
------------------------------
default                                [7 %]
hard_e-val                             [8 %]
soft_e-val                             [12 %]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS, resume function
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
resume                                 [OK, contents same but clids or seqs differ]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS
with option --lowmem
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
------------------------------
run default setup
------------------------------
------------------------------
run with harder e-value cuts
------------------------------
------------------------------
run with soft e-value cuts
------------------------------
------------------------------
Check output files
------------------------------
default                                [OK]
hard_e-val                             [OK]
soft_e-val                             [OK]
------------------------------
Check timing
------------------------------
default                                [6 %]
hard_e-val                             [8 %]
soft_e-val                             [7 %]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS, resume function
with option --lowmem
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
resume                                 [OK, contents same but clids or seqs differ]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST BRHCLUS
merge BRHs into an existing cluster configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mergebrh                               [OK]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST STATCLUS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
default                                [OK]
hard_e-val                             [OK]
soft_e-val                             [OK]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST CMPCLUS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
exact_match                            [OK]
mis-match                              [OK]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST CMPCLUS
related groups
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
default                                [OK]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TEST MERGEINPAR
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
clusters                               [OK]
unresolved                             [OK]


------------------------------------------
Warnings (4) were detected.
------------------------------------------

Most likely this is of not concern and the probable origin are messages like

  [OK, contents same but clids or seqs differ]

They may be caused by rounding errors on different architectures/versions of the software.

Private Internet Access, OpenVPN (2.3.2), and Ubuntu 14.04 (.2 LTS) – Yet Another Reported Way To Get Them Working (And The Only One That Works For Me)

Update: 17 May 2016 – Doesn’t happen often and is always nice to see. Thanks to Lucas Nell (on 26 April – that’s my bad) for taking one additional step out of the whole process with the script below. Simply replace “[put-your-password-here]” with your actual password (and no brackets), same the below as some_name.script (or whatever), chmod +x some_name.script to make it official, and you’re good to go to make the additional mod.

sudo -s

cd /etc/NetworkManager/system-connections

export pwd="\n[vpn-secrets]\npassword=[put-your-password-here]"

for f in PIA*
do
  sed -i 's/password-flags=1/password-flags=0/g' "${f}"
  echo -e "${pwd}" >> "${f}"
done

exit

If you sign up for an account with Private Internet Access (and this may go for some other VPN providers as well) and follow the only prominent Ubuntu link (12.04) in the Support Section (www.privateinternetaccess.com/pages/client-support/ubuntu-openvpn), you’ll be taken to a reasonably straightforward 9-step process that walks you through the OpenVPN setup – from the install_ubuntu.sh script download to the selection of PIA-points (I just made that up) in your Network Manager GUI (that radial wifi icon or arrows in the upper-right corner).

That is, for Ubuntu 12.04.

The Problem

If you try this in Ubuntu 14.04, everything more-or-less looks and runs the same way. That said, when you try to connect to a PIA-point in the Network Manager, nothing happens. Your wifi radial doesn’t change, flash, or provide any indication that something has gone right or wrong. More importantly (to the lack of feedback, anyway), you are not asked for your PIA password (having put in your username in the install process). This lack of password requesting turns out to be the real kicker (and diagnostic for the fix presented down below).

If you look in /etc/NetworkManager/system-connections, you’ll see that all of the PIA files have been successfully installed.

-rw——- 1 root root 326 Jul 16 16:14 PIA – AU Melbourne
-rw——- 1 root root 313 Jul 16 16:14 PIA – AU Sydney
-rw——- 1 root root 313 Jul 16 16:14 PIA – Brazil
-rw——- 1 root root 316 Jul 16 16:14 PIA – CA North York
-rw——- 1 root root 321 Jul 16 16:14 PIA – CA Toronto
-rw——- 1 root root 313 Jul 16 16:14 PIA – France
-rw——- 1 root root 315 Jul 16 16:14 PIA – Germany
-rw——- 1 root root 312 Jul 16 16:14 PIA – Hong Kong
-rw——- 1 root root 350 Jul 17 15:49 PIA – Ireland
-rw——- 1 root root 313 Jul 16 16:14 PIA – Israel
-rw——- 1 root root 311 Jul 16 16:14 PIA – Japan
-rw——- 1 root root 313 Jul 16 16:14 PIA – Mexico
-rw——- 1 root root 314 Jul 16 16:14 PIA – Netherlands
-rw——- 1 root root 310 Jul 16 16:14 PIA – Romania
-rw——- 1 root root 313 Jul 16 16:14 PIA – Russia
-rw——- 1 root root 312 Jul 16 16:14 PIA – Singapore
-rw——- 1 root root 313 Jul 16 16:14 PIA – Sweden
-rw——- 1 root root 317 Jul 16 16:14 PIA – Switzerland
-rw——- 1 root root 313 Jul 16 16:14 PIA – Turkey
-rw——- 1 root root 319 Jul 16 16:14 PIA – UK London
-rw——- 1 root root 329 Jul 16 16:14 PIA – UK Southampton
-rw——- 1 root root 327 Jul 16 16:14 PIA – US California
-rw——- 1 root root 315 Jul 16 16:14 PIA – US East
-rw——- 1 root root 321 Jul 16 16:14 PIA – US Florida
-rw——- 1 root root 321 Jul 16 16:14 PIA – US Midwest
-rw——- 1 root root 331 Jul 16 16:14 PIA – US New York City
-rw——- 1 root root 321 Jul 16 16:14 PIA – US Seattle
-rw——- 1 root root 334 Jul 16 16:14 PIA – US Silicon Valley
-rw——- 1 root root 317 Jul 16 16:14 PIA – US Texas
-rw——- 1 root root 315 Jul 16 16:14 PIA – US West

My first attempts at troubleshooting brought me to the installing privateinternetaccess on ubuntu 14.04 LTS page at askubuntu.com. The first response seems to be regurgitating the 12.04 installation process on the PIA site (which doesn’t work. For me, anyway), while the second response provides a list of installs that the install_ubuntu.sh script may or may not have successfully installed.

sudo apt-get install openvpn network-manager-openvpn network-manager-openvpn-gnome

The second commenter then walks through the install process as if the .ovpn config files didn’t exist (setting up from scratch, which can be laborious if you want to add all of the PIA points) but uses the contents of the openvpn.zip file downloaded by the question-asker.

The fix to the whole matter is partly in the questioner and second answer, but some additional work needs to be done. What’s described below is the process I used to figure out what was going on (showing all work), including using some alternatively-official .ovpn files (and the official ca.crt and crl.pem files provided by PIA).

The Diagnosing (What May Have Brought You Here)

With the failure to get any feedback from Network Manager (or the GUI) after the install, I went straight to the syslog to see if anything revealing appears (var/log/syslog). The error report for my VPN connection attempts reads as follows:

cd /var/log/syslog
more syslog

Jul 16 08:54:04 randommachine NetworkManager[13049]: Starting VPN service ‘openvpn’…
Jul 16 08:54:04 randommachine NetworkManager[13049]:
VPN service ‘openvpn’ started (org.freedesktop.NetworkManager.openvpn), PID 13164
Jul 16 08:54:04 randommachine NetworkManager[13049]:
VPN service ‘openvpn’ appeared; activating connections
Jul 16 08:54:04 randommachine NetworkManager[13049]: [1437051244.977042] [nm-vpn-connection.c:1374] get_secrets_cb(): Failed to request VPN secrets #2: (6) No agents were available for this request.
Jul 16 08:54:04 randommachine NetworkManager[13049]: Policy set ‘randomrouter’ (wlan0) as default for IPv4 routing and DNS.
Jul 16 08:54:10 randommachine NetworkManager[13049]:
VPN service ‘openvpn’ disappeared

A google search for “Failed to request VPN secrets #2“ (I can’t stress enough the value of quotes in troubleshooting Linux issues) dragged me to several pages that didn’t directly address my Network Manager issue, but indicated that one should consider running OpenPVN from the Terminal anyway. Extracting openvpn.zip (downloaded from the PIA website) and cd’ing into that folder (I assume you’re in Downloads), the following commands:

cd Downloads
unzip openvpn.zip
openvpn US\ East.ovpn 

Produces the following output – asking for username and password, but then failing to connect (and I include all the output below, assuming the error brought you here).

Thu Jul 16 09:06:55 2015 OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Dec 1 2014
Enter Auth Username:pXXXXXXX
Enter Auth Password:
Thu Jul 16 09:07:11 2015 UDPv4 link local: [undef]
Thu Jul 16 09:07:11 2015 UDPv4 link remote: [AF_INET]208.167.254.223:1194
Thu Jul 16 09:07:11 2015 WARNING: this configuration may cache passwords in memory — use the auth-nocache option to prevent this
Thu Jul 16 09:07:11 2015 [Private Internet Access] Peer Connection Initiated with [AF_INET]208.167.254.223:1194
Thu Jul 16 09:07:14 2015 ERROR: Cannot ioctl TUNSETIFF tun: Operation not permitted (errno=1)
Thu Jul 16 09:07:14 2015 Exiting due to fatal error

That said, when you apply root privileges:

sudo openvpn US\ East.ovpn

You get the following:

Thu Jul 16 09:07:28 2015 OpenVPN 2.3.2 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [eurephia] [MH] [IPv6] built on Dec 1 2014
Enter Auth Username:pXXXXXXX
Enter Auth Password:
Thu Jul 16 09:07:36 2015 UDPv4 link local: [undef]
Thu Jul 16 09:07:36 2015 UDPv4 link remote: [AF_INET]208.167.254.223:1194
Thu Jul 16 09:07:36 2015 WARNING: this configuration may cache passwords in memory — use the auth-nocache option to prevent this
Thu Jul 16 09:07:37 2015 [Private Internet Access] Peer Connection Initiated with [AF_INET]208.167.254.223:1194
Thu Jul 16 09:07:39 2015 TUN/TAP device tun0 opened
Thu Jul 16 09:07:39 2015 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Thu Jul 16 09:07:39 2015 /sbin/ip link set dev tun0 up mtu 1500
Thu Jul 16 09:07:39 2015 /sbin/ip addr add dev tun0 local 10.134.1.6 peer 10.134.1.5
Thu Jul 16 09:07:39 2015 Initialization Sequence Completed

Works great (can verify at whatsmyipaddress.com or other). And it asks for your username (because the .ovpn files haven’t been configured yet from the script) and password, so we were already ahead of the game from the Network Manager GUI.

OpenVPN seems to work fine and the .ovpn files work, so the problem is somewhere in Network Manager or how it and OpenVPN are interacting (which I’ve not yet found the answer to). Now, you’re supposed to be asked for the password when you try to establish the VPN connection with Network Manager. To see if that was the only problem with the .ovpn files, I simply added my password to the US East.ovpn file as follows (in US /East.ovpn):

nano US /East.ovpn

And add the following somewhere in the file:

password=put-your-password-here

Then restart the Network Manager (and wait a few seconds)

sudo service network-manager restart

And that didn’t work. That said, there’s another password flag in the file (aptly names password_flags) to play with. A search for details lead me to a post at forums.kali.org that goes into some detail about Network Manager NOT strong the VPN password correctly because the user keyring isn’t root-accessible.

Changing password-flags from 1 to 0 and attempting to connect with Network Manager = success!

So, the problem is somewhere in the failed password negotiation between Network Manager and OpenVPN, and providing that info in the .ovpn files from openvpn.zip and a network-manager restart solves the problem.

Now then, the differences between the .ovpn files in openvpn.zip (download-able from https://www.privateinternetaccess.com/openvpn/openvpn.zip) and the PIA VPN files installed using install_ubuntu.sh are as follows:

US East.ovpn

client
dev tun
proto udp
remote us-east.privateinternetaccess.com 1194
resolv-retry infinite
nobind
persist-key
persist-tun
ca ca.crt
tls-client
remote-cert-tls server
auth-user-pass
comp-lzo
verb 1
reneg-sec 0
crl-verify crl.pem

PIA US East

[connection]
id=PIA – US East
uuid=856fc7ec-cd90-4ab1-96c1-2d827e46ea8f
type=vpn
autoconnect=false

[vpn]
service-type=org.freedesktop.NetworkManager.openvpn
username=p9681681
comp-lzo=yes
remote=us-east.privateinternetaccess.com
connection-type=password
password-flags=0
ca=/etc/openvpn/ca.crt

[ipv4]
method=auto

The new format is so clean! It’s also the format you get if you go through the New Connection process through Network Manager. The formatting seems to be important for the fix I’m going to propose below, so I’m going to modify the newer format files below.

The Solution

The solution is, after many hours, stupid-simple – run the install_ubuntu.sh as described on the PIA website (which will also make you install a few extra programs if you don’t have them already- and it places ca.crt into your OpenVPN folder, which is then called by the VPN files), modify all of the PIA files in your /etc/NetworkManager/system-connections folder by putting your password into each (in the format as below), and restart network-manager from the Terminal. That, in theory, should be it. You’ll have to have root access to do this, though, as the file permissions are all (or should be) 600.

1. https://www.privateinternetaccess.com/pages/client-support/ubuntu-openvpn

2. Open Terminal

3. Move to the system-connections folder:

cd /etc/NetworkManager/system-connections

4. Edit all the PIA files. To each of the PIA files, all you have to do is add the following:

[vpn-secrets]
password=put-your-password-here

The [vpn-secrets] is important! I would have thought this to be a comment block for organizational purposes, but adding thr password line alone won’t cut it.

NOTE: If you’re trying to connect through the GUI and the VPN Connections DO NOT appear in the list – provided your password is in the file, your problem is very likely that the file permissions are wrong. If they’re not -rw——, then Network-Manager will not read them.

5. Extra bookkeeping step: double-confirm the permissions on the PIA files:

chmod 600 PIA*

6. Restart network-manager

sudo service network-manager restart

I do not know if/when the fix will come in between OpenVPN and Network Manager (or something else in Ubuntu) that will obviate the need for this workaround. In the meantime, the procedure above works just fine (works at all) on a clean install of 14.04.2 LTS. The problem seems to be with OpenVPN as it plays with 14.04, a recurring theme I’ve found from lots of people (or, perhaps more specifically, the use of the GUI to call OpenVPN). Given several reports of PIA/14.04, I’m surprised there isn’t more, perhaps specifically on the PIA website, to address this issue. Hopefully a proper fix from PIA, OpenVPN, or Ubuntu developers in en route.

Happy more safe/more secure surfing. And if you’re so inclined, the Litecoin bubble has not, yet, right now, burst (scroll to the bottom of http://www.somewhereville.com/?p=1896).