Trouble compiling NWChem with CUDA enabled

From NWChem

Viewed 3504 times, With a total of 11 Posts
Jump to: navigation, search

Gets Around
Threads 18
Posts 60
I am trying to compile NWChem 6.3 (2014-01-28) on a CentOS 6.5-x86_64 system with CUDA-5.5 installed. The 2.08MB make.log file ends with the following 8 lines:
/usr/local/nwchem-6.3/lib/LINUX64/libtce.a(hybrid.o): In function `check_device_':
hybrid.c:(.text+0x2): undefined reference to `armci_me'
hybrid.c:(.text+0x8): undefined reference to `armci_master'
/usr/local/nwchem-6.3/lib/LINUX64/libtce.a(hybrid.o): In function `device_init_':
hybrid.c:(.text+0x2f): undefined reference to `armci_me'
hybrid.c:(.text+0x35): undefined reference to `armci_master'
collect2: ld returned 1 exit status
make: *** [all] Error 1

This version of CentOS/NWChem compiles correctly, and runs, w/o CUDA. Cuda 5.5 installed correctly and runs cuda samples so I assume that is working.
Here are the environment variables that were set:
export NWCHEM_TOP=/usr/local/nwchem-6.3
export NWCHEM_TARGET=LINUX64
export USE_MPI=y
export USE_MPIF=y
export LIBMPI="-pthread -m64 -lmpi_f90 -lmpi_f77 -lmpi -ldl"
export MPI_LOC=/usr/lib64/openmpi
export MPI_LIB=/usr/lib64/openmpi/lib
export MPI_INCLUDE=/usr/include/openmpi-x86_64
export NWCHEM_MODULES=all
export LARGE_FILES=TRUE
export USE_NOFSCHECK=TRUE
export TCE_CUDA=y
export CUDA_LIBS="-L/usr/local/cuda-5.5/lib64 -lcudart"
export CUDA_FLAGS="-arch sm_20 "
export CUDA_INCLUDE="-I. -I/usr/local/cuda-5.5/include"
export CUDA=nvcc
export PATH=$PATH:/usr/local/cuda-5.5/bin
export FC=gfortran
export CC=gcc

Then compiled with make nwchem_config, and make >& make.log
I guess the big question is, what are the references to "armci" ?

Forum Vet
Threads 9
Posts 1576
Jkweller,
This is a know problem of the Cuda part of NWChem (it works straight out of the box only for certain values of ARMCI_NETWORK)
The next release will contain a fix for it.
If you are interested in testing the patch for this, I can pass it to you in a few days.
Edo

Gets Around
Threads 18
Posts 60
Edo,
Thanks for the offer. I would be glad to test a CUDA patch on the system I mentioned. This is a fairly typical setup, so others may also find it useful if it works.
John Keller

Gets Around
Threads 18
Posts 60
Not having received any further information on a patch, can you suggest values for ARMCI_NETWORK that, if set, will give a successful compilation along the lines of my original post?

  • Huub Forum:Admin, Forum:Mod, NWChemDeveloper, bureaucrat, sysop
    Profile
    Send PM
Forum Regular
Threads 1
Posts 185
In your original settings I didn't see any ARMCI_NETWORK setting at all (when building with the build_nwchem script that is fine but otherwise it isn't). In any case you seem to have MPI so you could try ARMCI_NETWORK=MPI-TS. I have no way of testing whether that will actually work in your setup, but it asks ARMCI to build using MPI as the communication fabric. I would be surprised if that wouldn't work.

Huub

Forum Vet
Threads 9
Posts 1576
Quote:Jwkeller Apr 17th 4:44 pm
Not having received any further information on a patch, can you suggest values for ARMCI_NETWORK that, if set, will give a successful compilation along the lines of my original post?


Jwkeller

I am not quite ready with the patch yet

I might be able to pass it to you in few days.

Cheers, Edo

Gets Around
Threads 18
Posts 60
Huub,
Adding the line "export ARMCI_NETWORK=MPI-TS " to list in the first box above, and using the same make commands, gives exactly the same make.log ouput and no nwchem executable.
John Keller

Gets Around
Threads 18
Posts 60
...and using the build_nwchem script also give the same error messages.
John K.

Forum Vet
Threads 9
Posts 1576
Quote:Jwkeller Apr 28th 11:07 pm
...and using the build_nwchem script also give the same error messages.
John K.

Yes, This is the expected behavior.
If you can wait a few more days, I should be able to issue a patch before the end of next week.
Edo

Forum Vet
Threads 9
Posts 1576
John
I have just uploaded a patch that should allow to use the Cuda part of NWChem (TCE/CCSD(T))

http://www.nwchem-sw.org/images/Cuda.patch.gz

To patch your NWChem 6.3 source, you may use the following steps

1) cd $NWCHEM_TOP/src
2) wget http://www.nwchem-sw.org/images/Cuda.patch.gz
3) gzip -d Cuda.patch
4) patch -p0 < Cuda.patch
5) recompile the directories util and tce/ccsd_t
6) relink

Gets Around
Threads 18
Posts 60
Thank you, Edo. I will try this. John Keller

Just Got Here
Threads 0
Posts 1
This worked great for me. Thanks!


Forum >> NWChem's corner >> Compiling NWChem



Who's here now Members 0 Guests 1 Bots/Crawler 0


AWC's: 2.5.10 MediaWiki - Stand Alone Forum Extension
Forum theme style by: AWC