HOWTO: compiling nwchem-6.5 beta on Ubuntu 14.04 x86-64

From NWChem

Viewed 7105 times, With a total of 4 Posts
Jump to: navigation, search
Forum >> NWChem's corner >> Compiling NWChem

Gets Around
Threads 18
Posts 133
12:00:52 AM PDT - Fri, Aug 29th 2014
I'm assuming that you have used NWChem before but have not necessarily ever built it from source. These instructions basically work for Debian Wheezy also, though you will have to tweak the Python-related paths for Debian.

I am assuming that you already have an optimized BLAS that provides a 64 bit integer interface for use by NWChem. If that is not true, skip ahead to the next post about building OpenBLAS.

All following instructions assume that you are downloading code and compiling starting in the directory /opt/science/nwchem/. Adjust paths accordingly if you are working somewhere else, like $HOME/nwchem/.

You will need a few non-standard packages to obtain the 6.5 beta source code and build it with all bells and whistles enabled. I think this covers the packages you need that aren't installed by default:

sudo apt-get install subversion openmpi-bin libopenmpi-dev libopenmpi1.6 gfortran python-dev


You will need to get the beta code from the Subversion repository:
svn checkout https://svn.pnl.gov/svn/nwchem/branches/release-6-5-patches nwchem-6.5

You will also need to get the latest NWChem development snapshot: http://www.nwchem-sw.org/download.php?f=Nwchem-dev.revision26027-src.2014-08-28.tar.gz

You need both because the src/tools/ directory is not populated in the code you check out from subversion, and you will need to replace it from the corresponding subdirectory in the snapshot. If you have nwchem-6.5 and Nwchem-dev.revision26027-src.2014-08-28.tar.gz in the same directory, do it like this:

tar xzvf Nwchem-dev.revision26027-src.2014-08-28.tar.gz
cd nwchem-6.5
rm -rf src/tools
cp -r ../Nwchem-dev.revision26027-src.2014-08-28/src/tools src


Now set a bunch of environment variables to enable all optional features and prepare for building. We're even enabling so-new-they're-undocumented features like EA-CCSD and IP-CCSD. Note that contrary to the wiki build documentation, if you want multi-reference coupled cluster capabilites you set MRCC_METHODS, not MRCC_THEORY.

export USE_NOFSCHECK=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export PYTHONLIBTYPE=so
export PYTHONCONFIGDIR=config-x86_64-linux-gnu
export CC=gcc
export FC=gfortran
export FOPTIMIZE="-O3 -march=native -mtune=native -msse -msse2 -msse3 -mssse3 -msse4 -msse4.1 -msse4.2 -funroll-loops -fprefetch-loop-arrays -fvariable-expansion-in-unroller -ffast-math -mfpmath=sse"
export COPTIMIZE="-O3 -march=native -mtune=native -msse -msse2 -msse3 -mssse3 -msse4 -msse4.1 -msse4.2 -funroll-loops -ffast-math -mfpmath=sse"
export HAS_BLAS=yes
export BLASOPT="-L/opt/science/openblas/64/lib -lopenblas"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export ARMCI_NETWORK=SOCKETS
export MPI_LOC=/usr/lib/openmpi/
export MPI_LIB=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBMPI="-lpthread -L$MPI_LIB -lmpi_f90 -lmpi_f77 -lmpi"
export MRCC_METHODS=y
export CCSDTQ=y
export CCSDTLR=y
export IPCCSD=y
export EACCSD=y
cd $NWCHEM_TOP/src
make clean
make nwchem_config 2>&1 | tee make.nwchem_config.log
make
cd ../contrib
./getmem.nwchem


The compiler flags are those suggested by Jeff Hammond: https://github.com/jeffhammond/HPCInfo/wiki/NWChem

This takes a while when it all goes successfully. Note especially the BLASOPT setting: if you haven't already installed OpenBLAS, and in the exact same way I did (unlikely), you will probably need to change that line.

If all builds successfully, you will find a 120 MB binary file in nwchem-6.5/bin/LINUX64. If you already have a version of NWChem installed, and you want the new version you've just built to be found when you type nwchem in a shell, adjust your PATH to put it first.

Gets Around
Threads 18
Posts 133
12:09:15 AM PDT - Fri, Aug 29th 2014
Building OpenBLAS for NWChem
OpenBLAS is fast, free, and simple to build compared to e.g. ATLAS.

I'm going to assume that you are working in /opt/science/openblas/; adjust your paths accordingly where needed.

Get the latest stable source release: https://github.com/xianyi/OpenBLAS/archive/v0.2.11.tar.gz

Then we're going to compile it so that it supplies a 64 bit integer interface, the NWChem default, and disables internal threading so that parallelism is left up to NWChem and MPI.

wget https://github.com/xianyi/OpenBLAS/archive/v0.2.11.tar.gz
tar xzvf v0.2.11.tar.gz
cd OpenBLAS-0.2.11
make USE_THREAD=0 INTERFACE64=1
make install PREFIX=/opt/science/openblas/64


If you use other scientific software that links against a BLAS, you can build multiple versions depending on whether you want a 32 or 64 bit integer interface, and whether or not the BLAS itself should enable threading. I actually have all 4 variants installed on my system, under the directories 64, 64-threaded, 32, and 32-threaded. I use wrapper scripts to set the LD_LIBRARY_PATH according to the dependencies of each BLAS-using program. But for NWChem all you need is the non-threaded 64 bit integer interface version.

Forum Vet
Threads 9
Posts 1498
4:14:45 PM PDT - Fri, Aug 29th 2014
Quote:Mernst Aug 28th 11:00 pm

You will need to get the beta code from the Subversion repository:
svn checkout https://svn.pnl.gov/svn/nwchem/branches/release-6-5-patches nwchem-6.5

You will also need to get the latest NWChem development snapshot: http://www.nwchem-sw.org/download.php?f=Nwchem-dev.revision26027-src.2014-08-28.tar.gz

You need both because the src/tools/ directory is not populated in the code you check out from subversion, and you will need to replace it from the corresponding subdirectory in the snapshot. If you have nwchem-6.5 and Nwchem-dev.revision26027-src.2014-08-28.tar.gz in the same directory, do it like this:

tar xzvf Nwchem-dev.revision26027-src.2014-08-28.tar.gz
cd nwchem-6.5
rm -rf src/tools
cp -r ../Nwchem-dev.revision26027-src.2014-08-28/src/tools src



Mernst

Please do NOT suggest replace the tools directory with the tools directory NWChem development snapshot since it will overwrite important fixes. The get-tools script will download the required patched ga-5-3 tarball as explained in
http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id1400/Beta_version_of_NWChem_6.5_a...

Cheers, Edo

Forum Vet
Threads 9
Posts 1498
4:22:53 PM PDT - Fri, Aug 29th 2014
Quote:Mernst Aug 28th 11:00 pm

export FOPTIMIZE="-O3 -march=native -mtune=native -msse -msse2 -msse3 -mssse3 -msse4 -msse4.1 -msse4.2 -funroll-loops -fprefetch-loop-arrays -fvariable-expansion-in-unroller -ffast-math -mfpmath=sse"
export COPTIMIZE="-O3 -march=native -mtune=native -msse -msse2 -msse3 -mssse3 -msse4 -msse4.1 -msse4.2 -funroll-loops -ffast-math -mfpmath=sse"



I strongly discourage modifying the values of the FOPTIMIZE and COPTIMIZE variables.
The reason is that the default values for FOPTIMIZE and COPTIMIZE have been tested by the NWChem developers (using the internal QA suites, among others), while any modification might produce incorrect results.

Cheers, Edo

Gets Around
Threads 18
Posts 133
6:44:22 PM PDT - Fri, Aug 29th 2014
updated instructions
Incorporating feedback from above, to ensure that you get the latest Global Arrays and use default compiler settings:

svn checkout https://svn.pnl.gov/svn/nwchem/branches/release-6-5-patches nwchem-6.5
cd nwchem-6.5/src/tools
./get-tools
cd ../..
export USE_NOFSCHECK=TRUE
export TCGRSH=/usr/bin/ssh
export NWCHEM_TOP=`pwd`
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export LARGE_FILES=TRUE
export ENABLE_COMPONENT=yes
export PYTHONHOME=/usr
export PYTHONVERSION=2.7
export PYTHONLIBTYPE=so
export PYTHONCONFIGDIR=config-x86_64-linux-gnu
export CC=gcc
export FC=gfortran
export HAS_BLAS=yes
export BLASOPT="-L/opt/science/openblas/64/lib -lopenblas"
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export ARMCI_NETWORK=SOCKETS
export MPI_LOC=/usr/lib/openmpi/
export MPI_LIB=/usr/lib/openmpi/lib
export MPI_INCLUDE=/usr/lib/openmpi/include
export LIBMPI="-lpthread -L$MPI_LIB -lmpi_f90 -lmpi_f77 -lmpi"
export MRCC_METHODS=y
export CCSDTQ=y
export CCSDTLR=y
export IPCCSD=y
export EACCSD=y
cd $NWCHEM_TOP/src
make clean
make nwchem_config 2>&1 | tee make.nwchem_config.log
make
cd ../contrib
./getmem.nwchem
Edited On 12:55:59 AM PDT - Sat, Aug 30th 2014 by Mernst


Forum >> NWChem's corner >> Compiling NWChem



Who's here now Members 0 Guests 1 Bots/Crawler 0

AWC's: 2.5.10 MediaWiki - Stand Alone Forum Extension
Forum theme style by: AWC
Retrieved from "http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id1401"