6.3 openib build problems?

From NWChem

Viewed 1502 times, With a total of 4 Posts
Jump to: navigation, search
Forum >> NWChem's corner >> Compiling NWChem

Clicked A Few Times
Threads 3
Posts 13
11:17:33 AM PDT - Mon, Jul 1st 2013
hmmm,

the build script below works when i build 6.1.1 but not with the current 6.3 release. if fails here:

ifort -i8 -align -w -g -vec-report1 -O2 -g -L/shared/build/nwchem/nwchem-6.3-src.2013-05-28/lib/LINUX64 -L/shared/build/nwchem/nwchem-6.3-src.2013-05-28/src/tools/install/lib -o /shared/build/nwchem/nwchem-6.3-src.2013-05-28/bin/LINUX64/nwchem nwchem.o stubs.o -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver -loptim -lnwdft -lgradients -lcphf -lesp -lddscf -ldangchang -lguess -lhessian -lvib -lnwcutil -lrimp2 -lproperty -lnwints -lprepar -lnwmd -lnwpw -lofpw -lpaw -lpspw -lband -lnwpwlib -lnwxc -lcafe -lspace -lanalyze -lqhop -lpfft -ldplot -ldrdy -lvscf -lqmmm -lqmd -letrans -lpspw -ltce -lbq -lcons -lperfm -ldntmc -lccca -lnwcutil -lga -larmci -lpeigs -lperfm -lcons -lbq -lnwcutil -Wl,--start-group /shared/apps/intel/composer_xe_2013.3.163/mkl/lib/intel64/libmkl_intel_ilp64.a /shared/apps/intel/composer_xe_2013.3.163/mkl/lib/intel64/libmkl_sequential.a /shared/apps/intel/composer_xe_2013.3.163/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -llapack -lblas -L/shared/apps/openmpi-1.6.4/intel/lib -lpthread -L/shared/apps/openmpi-1.6.4/intel/lib -lmpi_f90 -lmpi_f77 -lmpi -lrdmacm -libumad -libverbs -ldl -lpthread -lm -lrt -lnsl -lutil
ld: cannot find -lrdmacm
make: *** [all] Error 1

the build script is:

#!/bin/bash
export NWCHEM_TOP=/shared/build/nwchem/nwchem-6.3-src.2013-05-28
export NWCHEM_TARGET=LINUX64
echo "NWCHEM_TOP=$NWCHEM_TOP"
echo "NWCHEM_TARGET=$NWCHEM_TARGET"

export ARMCI_NETWORK=OPENIB
export IB_HOME=/opt/ofed
export IB_INCLUDE=/opt/ofed/include
export IB_LIB=/opt/ofed/lib64
export IB_LIB_NAME="-lrdmacm -libumad -libverbs -ldl -lpthread -lm -Wl,--export-dynamic -lrt -lnsl -lutil"
export MSG_COMMS=MPI
export TCGRSH=/usr/bin/ssh
echo "ARMCI_NETWORK=$ARMCI_NETWORK"
echo "IB_HOME=$IB_HOME"
echo "IB_INCLUDE=$IB_INCLUDE"
echo "IB_LIB=$IB_LIB"
echo "IB_LIB_NAME=$IB_LIB_NAME"
echo "MSG_COMMS=$MSG_COMMS"
echo "TCGRSH=$TCGRSH"

export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export MPI_LOC=/shared/apps/openmpi-1.6.4/intel
export MPI_LIB=$MPI_LOC/lib
export MPI_INCLUDE=$MPI_LOC/include
export LIBMPI="-lpthread -L$MPI_LIB -lmpi_f90 -lmpi_f77 -lmpi"
echo "USE_MPI=$USE_MPI"
echo "USE_MPIF=$USE_MPIF"
echo "USE_MPIF4=$USE_MPIF4"
echo "MPI_LOC=$MPI_LOC"
echo "MPI_LIB=$MPI_LIB"
echo "export MPI_INCLUDE=$export MPI_INCLUDE"
echo "LIBMPI=$LIBMPI"

export NWCHEM_MODULES="all"
export LARGE_FILES=TRUE
export USE_NOFSCHECK=TRUE
export LIB_DEFINES=-DDFLT_TOT_MEM=16777216
echo "NWCHEM_MODULES=$NWCHEM_MODULES"
echo "LARGE_FILES=$LARGE_FILES"
echo "LARGE_FILES=$LARGE_FILES"
echo "LIB_DEFINES=$LIB_DEFINES"

export PYTHONHOME=/usr
export PYTHONVERSION=2.6
export USE_PYTHON64=y
export PYTHONLIBTYPE=so
echo "PYTHONHOME=$PYTHONHOME"
echo "PYTHONVERSION=$PYTHONVERSION"
echo "USE_PYTHON64=$USE_PYTHON64"
echo "PYTHONLIBTYPE=$PYTHONLIBTYPE"


export MKLROOT=/shared/apps/intel/composer_xe_2013.3.163/mkl
export BLASOPT="-Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_ilp64.a $MKLROOT/lib/intel64/libmkl_sequential.a $MKLROOT/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm
"
echo "BLASOPT=$BLASOPT"

export FC=ifort
export CC=icc
echo "FC=$FC"
echo "CC=$CC"

echo "### START MAKING####"

echo "make realclean"
make realclean

echo "cd $NWCHEM_TOP/src"
cd $NWCHEM_TOP/src
pwd
echo "make nwchem_config"
make nwchem_config
echo " "
echo "########"
echo " "
echo "make FC=$FC CC=$CC"
make FC=$FC CC=$CC</nowiki>

do i need to modify my IB_LIB_NAMES variable?

--- michael

  • Huub Forum:Admin, Forum:Mod, NWChemDeveloper, bureaucrat, sysop
    Profile
    Send PM
Forum Regular
Threads 1
Posts 185
1:40:19 PM PDT - Mon, Jul 1st 2013
Hi Michael,

I have noticed that the value of IB_LIB does not appear on your link line. At the moment I don't know why that happens (as you said with nwchem-6.1.1 this used to work). As a work around could you try modifying IB_LIB_NAME to

export IB_LIB_NAME="-L$(IB_LIB) $(IB_LIB_NAME)"

In the meantime I trying to figure out why the value of IB_LIB is no longer picked up.

Huub

Clicked A Few Times
Threads 3
Posts 13
8:41:10 AM PDT - Tue, Jul 2nd 2013
i modified this line to:

export IB_LIB_NAME="-L/opt/ofed/lib64 -lrdmacm -libumad -libverbs -ldl -lpthread -lm -Wl,--export-dynamic -lrt -lnsl -lutil"

still fails at:

ifort -i8 -align -w -g -vec-report1 -O2 -g -L/shared/build/nwchem/nwchem-6.3-src.2013-05-28/lib/LINUX64 -L/shared/build/nwchem/nwchem-6.3-src.2013-05-28/src/tools/install/lib -o /shared/build/nwchem/nwchem-6.3-src.2013-05-28/bin/LINUX64/nwchem nwchem.o stubs.o -lnwctask -lccsd -lmcscf -lselci -lmp2 -lmoints -lstepper -ldriver -loptim -lnwdft -lgradients -lcphf -lesp -lddscf -ldangchang -lguess -lhessian -lvib -lnwcutil -lrimp2 -lproperty -lnwints -lprepar -lnwmd -lnwpw -lofpw -lpaw -lpspw -lband -lnwpwlib -lnwxc -lcafe -lspace -lanalyze -lqhop -lpfft -ldplot -ldrdy -lvscf -lqmmm -lqmd -letrans -lpspw -ltce -lbq -lcons -lperfm -ldntmc -lccca -lnwcutil -lga -larmci -lpeigs -lperfm -lcons -lbq -lnwcutil -Wl,--start-group /shared/apps/intel/composer_xe_2013.3.163/mkl/lib/intel64/libmkl_intel_ilp64.a /shared/apps/intel/composer_xe_2013.3.163/mkl/lib/intel64/libmkl_sequential.a /shared/apps/intel/composer_xe_2013.3.163/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -lpthread -lm -llapack -lblas -L/shared/apps/openmpi-1.6.4/intel/lib -lpthread -L/shared/apps/openmpi-1.6.4/intel/lib -lmpi_f90 -lmpi_f77 -lmpi -lrdmacm -libumad -libverbs -ldl -lpthread -lm -lrt -lnsl -lutil
ld: cannot find -lrdmacm
make: *** [all] Error 1

  • Huub Forum:Admin, Forum:Mod, NWChemDeveloper, bureaucrat, sysop
    Profile
    Send PM
Forum Regular
Threads 1
Posts 185
8:32:39 AM PDT - Tue, Jul 9th 2013
Hi Michael,

From the link line it seems that actually neither IB_LIB nor IB_LIB_NAME are picked up. Hence I would like to suggest setting the following:

export LIBMPI="$(LIBMPI) -L$(IB_LIB) $(IB_LIB_NAME)"

I am still at a loss why the OpenIB environment variables are not picked up but hopefully this will fix the link line at least as a workaround.

Huub

Forum Vet
Threads 3
Posts 855
8:59:37 AM PDT - Tue, Jul 9th 2013
Michael
I got it work with the following settings

IB_LIB=/opt/ofed/lib64
IB_INCLUDE=/opt/ofed/include

Cheers, Edo


Forum >> NWChem's corner >> Compiling NWChem



Who's here now Members 0 Guests 1 Bots/Crawler 0

AWC's: 2.5.10 MediaWiki - Stand Alone Forum Extension
Forum theme style by: AWC
Retrieved from "http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id890/"