bfast-0.6.5a, MUMmer-3.22, and Amos-3.0.0 Installs In Ubuntu 10.04 LTS (And Related)

Taking care of a DNA/RNA fragment alignment installation triple-threat with this post. These Ubuntu installs for largely problem-free, but one little trick is needed for Amos (this because of my use of “/opt” for my usual installation and compilation attempts and, more so, my not being interested in modifying the root PATH statement despite the constant use of sudo when building in “/opt”).

So, with the downloads of

bfast-0.6.5a (currently: sourceforge.net/apps/mediawiki/bfast/index.php?title=Main_Page)
MUMmer-3.22 (currently: mummer.sourceforge.net)
Amos-3.0.0 (currently: sourceforge.net/apps/mediawiki/amos/index.php?title=AMOS)

taken care of, the following process is performed.

user@machine:~sudo aptitude update
user@machine:~sudo aptitude upgrade

[POSSIBLE RESTART REQUIRED after this. You don’t need-need to update/upgrade, but I do it before all builds regardless.]

user@machine:~sudo apt-get install bison build-essential cmake csh doxygen flex fort77 freeglut3-dev g++ g++-multilib gcc gcc-multilib gettext gfortran gnuplot ia32-libs lib32asound2 lib32gcc1 lib32gcc1-dbg lib32gfortran3 lib32gomp1 lib32mudflap0 lib32ncurses5 lib32nss-mdns lib32z1 libavdevice52 libbz2-dev libc6-dev-i386 libc6-i386 libfreeimage-dev libglew1.5-dev libnetcdf-dev libopenal1 libopenexr-dev libopenmpi-dev libpng12-dev libqt4-dev libssl-dev libstdc++6-4.3-dbg libstdc++6-4.3-dev libstdc++6-4.3-doc libxext-dev libxi-dev libxml-simple-perl libxmu-dev libxt-dev mercurial nfs-common nfs-kernel-server openmpi-bin patch portmap python2.6-dev rpm ssh tcsh xorg-dev zlib1g-dev

The large apt-get above is my “default” additional install for a variety of programs, including Amber, Abinit, GAMESS, GROMAC, etc. Many of these may not be needed but hard drives are cheap and figuring out the minimum list is more work than simply installing everything. Do check the list, however, to make sure something won’t confuse any other installs on your machine (if you’re new to this, likely not. If you’ve done builds a few times, you may already know the difference).

user@machine:~$ sudo mv bfast-0.6.5a.tar.gz /opt
user@machine:~$ sudo mv MUMmer3.22.tar.gz /opt
user@machine:~$ sudo mv amos-3.0.0.tar.gz /opt
user@machine:~$ cd /opt

Move the three programs to /opt (or not). Specifically for bfast, two additional apt-get’s are required.

user@machine:/opt$ sudo apt-get install libstatistics-descriptive-perl libdbd-pg-perl

The build for bfast is straightforward.

user@machine:/opt$ sudo tar xvfz bfast-0.6.5a.tar.gz 
user@machine:/opt$ cd bfast-0.6.5a/
user@machine:/opt/bfast-0.6.5a$ sudo ./configure 
user@machine:/opt/bfast-0.6.5a$ sudo make
user@machine:/opt/bfast-0.6.5a$ sudo make install
user@machine:/opt/bfast-0.6.5a$ cd ..

bfast is officially built and you’ve returned to your “/opt” directory. MUMmer is also straightforward.

user@machine:/opt$ sudo tar xvfz MUMmer3.22.tar.gz 
user@machine:/opt$ cd MUMmer3.22/
user@machine:/opt/MUMmer3.22$ sudo make check
user@machine:/opt/MUMmer3.22$ sudo make install

MUMmer is officially built. If you intend to build Amos, you will need some of what you built in MUMmer. Specifically, nucmer, delta-filter, and show-coords are used by Amos and must be present in your PATH during the Amos build. As I am building in “/opt,” I’m using sudo. As I do not want to deal with setting a new PATH for root, the solution is simply to move these three programs to a universally accessible place.

user@machine:/opt/MUMmer3.22$ sudo cp nucmer /usr/local/bin/
user@machine:/opt/MUMmer3.22$ sudo cp delta-filter /usr/local/bin/
user@machine:/opt/MUMmer3.22$ sudo cp show-coords /usr/local/bin/
user@machine:/opt/MUMmer3.22$ cd ..

And, with that, you are ready for the Amos build procedure. You will need two more apt-get installs to complete the Amos build.

user@machine:/opt/amos-3.0.0$ sudo apt-get install libboost-all-dev libqt3-headers

To build Amos with no errors and all of the listed components, note the ./configure settings and run the list below.

user@machine:/opt$ sudo tar xvfz amos-3.0.0.tar.gz 
user@machine:/opt$ cd amos-3.0.0/
user@machine:/opt/amos-3.0.0$ sudo ./configure --with-Qt-dir=/usr/share/qt3 --prefix=/opt/amos-3.0.0
user@machine:/opt/amos-3.0.0$ sudo make
user@machine:/opt/amos-3.0.0$ sudo make check
user@machine:/opt/amos-3.0.0$ sudo make install

If nucmer, delta-filter, show-coords, and the qt3 libraries are not present, you’ll see the following error list after running ./configure:

-- AMOS Assembler 2.0.8 Configuration Results --
  C compiler:          gcc -g -O2
  C++ compiler:        g++ -g -O2
  GCC version:         gcc (Ubuntu 4.4.3-4ubuntu5) 4.4.3
  Host System type:    x86_64-unknown-linux-gnu
  Install prefix:      /opt/amos-3.0.0
  Install eprefix:     ${prefix}

  See config.h for further configuration information.
  Email  with questions and bug reports.

WARNING! nucmer was not found but is required to run AMOScmp
   install nucmer if planning on using AMOScmp
WARNING! delta-filter was not found but is required to run AMOScmp-shortReads-alignmentTrimmed
   install delta-filter if planning on using AMOScmp-shortReads-alignmentTrimmed
WARNING! show-coords was not found but is required to run minimus2
   install show-coords if planning on using minimus2
WARNING! Qt3 toolkit was not found but is required to run AMOS GUIs
   install Qt3 or locate Qt3 with configure to build GUIs
   see config.log for more information on what went wrong
WARNING! Boost graph toolkit was not found but is required to run parts of the AMOS Scaffolder (Bambus 2)
   install Boost or locate Boost with configure to build Scaffolder
   see config.log for more information on what went wrong

And, finally, add these directories to your PATH.

user@machine:~$ cd
user@machine:~$ pico .profile

Add the following to your PATH statement:

/opt/amos-3.0.0/bin/:/opt/MUMmer3.22/:/opt/bfast-0.6.5a/butil/:

Crtl-X, “Y”, and quit.

The “Stone Boulders” – All 12-Group (4,096) and 16-Group (65,536) L/R Sticking Combinations In PDF Format

George L. Stone's Stick Control

You should have a copy regardless (amazon.com direct link).

[Drafted as an article for somewhere, stuck here instead…]

You could spend your life on the first six pages of “Stick Control” and still not cover all the possibilities. Dynamics, accents, foot-hand, foot-foot, fast/slow, hands on top of foot patterns, feet on top of hand patterns, regroupings and accenting in 5-7-4 (regrouping of the 16 strokes per pattern), 7-5-4 (re-regrouping of the 16 strokes), yadda yadda. If you see the first six pages of Stick Control as just exercises, you miss the fantastic complexity YOU can introduce to constantly humble yourself while hovering over a practice pad.

As I look at the sets of exercises, I see what I assume most of us do – paradiddles, singles, doubles, multiple-hits of the same stick, some oddball patterns you start playing as written and then mess up without knowing, etc. The question I found myself asking was “What drove Stone to use this particular sequence?” I eventually turned that question around and decided to answer the question “What did Stone leave out?” The PDFs linked to this article are what I’ve affectionately come to call the “Stone Boulder,” providing EVERY sticking combination Stone included and every other combination he didn’t. Some intro to how and why is below, followed by a bit of explanation. I think the patterns themselves are self-explanatory.

While not the most cite-able examples in all of genomics, there have always been passing references to drumming “being in someone’s DNA.” As it happens, drumming and biology did overlap in general approach during the mid-80’s-to-early-90’s (or so) in the great heyday of linear drumming (go dig out your Murray Houllif and Gary Chaffee books). The idea is simple: no two drums/cymbals hit at the same time, producing an often staccato and generally (well, to my ears anyway) more melodic sound from the drums (and much easier to transcribe than some of the superhuman overlapping rhythms people are having fun playing today). Ignoring the complexities of 3.5 billions years of evolution, DNA works the same way as these linear patterns to convey a message. The four bases in your DNA, A (adenine), C (cytosine), G (guanine), and T (thymine), act as a code that is read like those old drum beats were played – one at a time with no doubling-up please. The identical three million base-long DNA sequences in each of your cells (see CSI) could be turned from seemingly random patterns of [A,C,G,T] into seemingly random patterns of [L,R,B,H] (that’s left hand, right hand, bass drum foot, hi-hat foot), then some experimental linear drumming composer could “play” your genome. Better still, if the transcriber was as good as your cellular machinery, the entire performance could be written down and reconverted into [A,C,G,T] format exactly so you could be cloned and double-drum with someone who rushes and slows down just as much as you do.

While most people think of a drum programmer as someone who generates patterns on a computer, I took the route of programming to generate patterns to drum. I most certainly did NOT put the pattern pages together by hand (I promise, no mistakes). A small script in the Perl programming language used to generate DNA sequences did all of the dirty work (including making sure all patterns only appear ONCE in each document). The math for figuring out the total number of left/right patterns is quite simple. The number of combinations of unique sticking patterns for a particular pattern length is 2^n, n being the number of beats. For a single beat, that’s 2^1, or just 2, that single hit being performed with either the left or right hand. For a four-stroke pattern, that’s 2^4 (2 * 2 * 2 * 2), or 16 total patterns. These are shown below out of academic interest (although I hope you could write them down from memory).

Now, consider the first six pattern pages of the Stone book. 16 beats per pattern. That’s 2^16, or 65536 total patterns. At 20 patterns per page, the complete Stone book of these first six pages would take up 3,277 pages. At 2 seconds per pattern, you could rip through all 3,277 pages in about 36 hours 30 minutes (about the perceived length of a society gig).

For you fellow jazzers out there looking for a more swingin’ set, I’ve also included the same sticking deal with a triplet-feel set (12 beats instead of 16, so you’ve only got 4096 patterns to contend with, meaning you could play through the whole set in about 2 hours 30 minutes).

4,096 patterns are bad enough. 65,536 is borderline something uncouth. On the one hand, that’s a lot of patterns either way. On the other, for the obsessive compulsive readers, these are IT. There are no other 12- or 16-stroke sticking combinations that have a stick hitting on each beat (that is, no rests). As Terry Bozzio has said in one form or another in his many clinics introducing his ostinato independence exercises “once you’ve played through the 16, you’ve played every 16th note pattern there is.”

And it could be worse! If you wanted every combination of left, right and rest, that’s 3^16, or 43,046,721 patterns. At 2 seconds per pattern, that’s 23,915 hours, or about 2 years and 8 months. I pondered doing the same thing for all 16-note linear drum patterns (L,R,B,F), which would produce 4^16, or 4,294,967,296 patterns. That’s 2,386,093 hours, or 272 years and 3 months (that’s approaching four reincarnations of “no life”).

Each full page has three columns of 40 patterns (120 per page), producing a document that’s only 547 pages long (but entirely green-friendly in PDF format). You will note that most of the pages look like the same stupid thing. This is because the mechanism of generation for the sequences involved making single changes at a single position and walking down the entire 16-stroke sets until all changes had been accounted for. I become bored to tears staying on a single page and generally scroll at random and point the stick at the screen to pick a pattern to play. Be as methodical or all-over-the-map as you will.

Is there a good reason for doing this? Not particularly. There are lots of patterns here that are a mechanical challenge for your arms, but many (many, many) of these patterns do not immediately lend themselves to the funk-ability of some of the Stone patterns (which tend to at least have groupings that, again, reflect rudiments or make you work one limb preferentially in a “usable” way). They are here mostly for completeness and, for when you want to confuse your limbs, picking a page or more at random and seeing how the patterns feel. As independence exercises teach us very early on, our brains are wired for preferential patterning (you hit the same foot as you would hand, you’re non-dominant hand sucks, your hi-hat foot is born useless, and other revelations). This document is simply another PDF you can lose on your machine somewhere or have in that hidden work folder that comes out and gets an intense few looks as you try to split your left and right hands apart more.

And, it should be obvious, the same applies for your feet.

Having fought through enough of the combinations, I began to notice something I’m sure all of us have encountered as we approach that hypnotic state of cruising through a pattern we “get.” Some patterns feel really good to play, but only after you’ve internalized them enough to “play something else,” like feeling an odd clave or taking the patterns with many doubles of one hand and ripping them into a bounce-driven frenzy (or, invariably, playing one pattern we love to play to find out it’s a pattern you heard and memorized in a more musical context on record). The one benefit I’ve found from having this PDF around is that I have all of the patterns in one place, which makes me think harder about the different ways to play the patterns (although that is only a fringe benefit). If you treat them like a journey and not just the first 5 minutes of your warm-up routine, I suspect you could spend your life on any one page and still not cover all of the bases.

So, without further ado:

2011july13_stone_boulder_sticking_series_V1.pdf
2011july13_stone_boulder_jazz_grouping_series_V1.pdf

UNAFold 3.8, MFold Utilities 4.5/4.6 And Additional Component Installation (Using XCode Tools 3 And Fink 0.29.21) For OSX 10.6.x

NOTE: The version numbers for everything are given specifically because aspects of the installation process may change with different versions and, in the event, I will not necessarily know the answer to subsequent problems if major version changes include major changes to the below (and that should clear up the “qualifications” section).

The UNAFold (UNified Nucleic Acid Fold(ing)) nucleic acid folding and hybridization prediction program set (here using version 3.8) can by itself be built with few (and not important) errors in OSX with Xcode Tools 3. The actual running of UNAFold.pl produces several errors that do not affect the run but do affect the amount/format of the output. It is my assumption that any OS running a less-than “kitchen sink” installation of Linux/Unix (Ubuntu, gentoo and Damn Small Linux come to mind) will have these errors and will require subsequent installations of programs/libraries that pieces of UNAFold rely on for processing output into, specifically, images and PDF files. OSX has the same issue that is easy to handle using Fink (and less so trying to install otherwise completely unrelated programs to make these “dependencies” (programs and libraries) available to UNAFold). Once Fink is installed, it is a few-step process to build UNAFold, move the Mfold Utilities contents to their proper folders (and there is a small trick here as well), and generate a UNAFold-complete install for all your DNA/RNA needs.

1. UNAFold 3.8 Installation

To begin, download (currently at mfold.rna.albany.edu/?q=DINAMelt/software), extract, open a terminal, cd into the unafold_3.8 directory (likely ~/Downloads/unafold_3.8), and run ./configure.

[prompt]$ cd ~/Downloads/unafold_3.8
[prompt]$ ./configure

On my machine (MacBook Pro, 10.6.x OSX + XCode Tools 3), this produces the output found in the local file 2011june_unafold_configure_output.txt.

You will likely note two sets of errors in the ./configure output:

./configure: line 8579: sort: No such file or directory
./configure: line 8576: sed: No such file or directory
./configure: line 10077: sort: No such file or directory
./configure: line 10074: sed: No such file or directory

The 10077 and 10074 errors are a bit odd because there are only 10039 lines in the configure file.

Are these errors important? No, you can build UNAFold just fine. I have run into these two “sort” and “sed” problems with a few other build attempts in OSX but have no good answer as to how to get around them (in case you’re wondering, sort and sed are most certainly installed on the machine. The “sort” error can be removed by specifying the path explicitly in the configure file (in line 8579, change “sort” the “/usr/bin/sort”), but the sed error persists in the few attempts I tried to work around it. It doesn’t appear to be a simple PATH issue. I’m not yet interested enough in finding a proper solution but, if you know, please post a comment or send a message. Is it just a character issue as discussed at itmercenary.livejournal.com/1585.html?).

running “make” produces the output found in the local file 2011june_unafold_make_output.txt.

[prompt]$ make

No issues. To install UNAFold, which will default to putting components into /usr/local/bin and /usr/local/share/, run sudo make install, which produces the output found in the local file 2011june_unafold_sudo_make_install_output.txt.

[prompt]$ sudo make install

Again, no issues. You will now have a populated /usr/local/bin folder.

2. MFold Utilities 4.5 (and, currently, the source for 4.6)

The next (optional) step is the inclusion of the mfold_util-4.5-Mac binaries (currently available at mfold.rna.albany.edu/?q=mfold/download-mfold), which I’ve also placed into the /usr/local/bin folder by extracting the contents of this file, them performing a cp * /usr/local/bin from within the MacBin directory.

[prompt]$ cd ~/Downloads/MacBin/
[prompt]$ sudo cp * /usr/local/bin

The processing of the data into plots with these programs requires that a set of *.col files be placed in the folder /usr/local/shared/mfold_util. Furthermore, these *.col are NOT provided in the mfold_util-4.5-Mac binary package. To get these files, you need only download the mfold_util-4.6.tar.gz file (currently at mfold.rna.albany.edu/?q=mfold/download-mfold), cd your way into src, make the /usr/local/shared/mfold_util folder, and copy the *.col files to /usr/local/shared/mfold_util.

[prompt]$ sudo mkdir /usr/local/shared/mfold_util
[prompt]$ cd Downloads/mfold_util-4.6/src
[prompt]$ sudo cp *.col /usr/local/shared/mfold_util

3. Fink 0.29.21 Install From Scratch

The first indication that other work was required came from trying to run mutplot randomly, which produced the following error:

dyld: Library not loaded: /sw/lib/libpng12.0.dylib
  Referenced from: /usr/local/bin/mutplot
  Reason: image not found
Trace/BPT trap

As digging around for libraries is not as straightforward as it would be for a Linux distro, I chose instead to solve the many problems by installing dependencies through the Fink program (currently fink-0.29.21). As 10.6.x users will find that there is no available Fink binary, you must build this from the source (which, with Xcode Tools 3 installed, occurs without error. If you don’t have Xcode Tools 3 installed, the new mechanism for buying a copy of XCode Tools 4 is less than ideal (to me, anyway. $4.99?) but now occurs through the App Store).

Download the fink source (fink 0.29.21), extract, cd into the fink-0.29.21 directory, and run bootstrap. Upon completion, you run pathsetup.sh, source your .profile, and update fink.

[prompt]$ cd ~/Downloads/fink-0.29.21
[prompt]$ ./bootstrap
[prompt]$ . /sw/bin/pathsetup.sh 
[prompt]$ cd ~/
[prompt]$ source .profile
[prompt]$ fink selfupdate-rsync
[prompt]$ fink update-all

The output for my installation can be found in 2011june_fink_install_output.txt. The rsync output can be found in 2011june_fink_selfupdate_rsync_output.txt. NOTE: You will be asked several questions about the installation process. Be prepared to blindly select the default settings with [enter], but don’t just walk away from the screen.

This completes the UNAFold install, MacBin install, and Fink install, meaning now we can walk through the dependencies.

4. Installing UNAFold (well, MFold Utils) Dependencies

First dependency-free UNAFold.pl run attempt produces the following error:

[prompt]$ UNAFold.pl seqtest.txt 
Checking for boxplot_ng... dyld: Library not loaded: /sw/lib/libpng12.0.dylib
  Referenced from: /usr/local/bin/boxplot_ng
  Reason: image not found
found, supports Postscript
Checking for hybrid-plot-ng... found, supports Postscript
Checking for sir_graph_ng or sir_graph... dyld: Library not loaded: /sw/lib/libpng12.0.dylib
  Referenced from: /usr/local/bin/sir_graph
  Reason: image not found
found, supports Postscript
Checking for ps2pdfwr... not found
Calculating for seqtest.txt, t = 37

As the UNAFold install page states, you need glut, the GD library, and gnuplot installed (and all of the many libraries therein).

[prompt]$ fink install libjpeg tetex gd2 gnuplot

For gnuplot, you will be required to make a few selections during the build process (blindly hitting the enter key at these questions will do, but this is not just a “type and go” install process. And it took about two hours on a MBP).

A final working error-free run looks as below, leaving you to process the data with the MFold Utilities as you like:

[prompt]$ UNAFold.pl seqtest.txt 
Checking for boxplot_ng... found, supports Postscript
Checking for hybrid-plot-ng... found, supports Postscript
Checking for sir_graph_ng or sir_graph... found, supports Postscript
Checking for ps2pdfwr... found
Calculating for seqtest.txt, t = 37