Intel E5-2640v3 Haswell: No such instruction

From NWChem

Viewed 883 times, With a total of 4 Posts
Jump to: navigation, search
Forum >> NWChem's corner >> Compiling NWChem

Just Got Here
Threads 1
Posts 3
5:40:21 AM PST - Thu, Dec 8th 2016
Hi,

I am trying to compile nwchem-6.6 on a HPC cluster running CentOS Linux version 6 on Intel E5-2640v3 Haswells.

I intent to use gcc 4.9.2, openmpi 1.6.5, and intel-mkl 2017 and have employed all patches from the web page.

Unfortunately, I run into the following make error suggesting that the CPU is erroneously detected:

make[1]: Entering directory `/panfs/pan01/vol011/home/dkr/nwchem/nwchem-6.6/src/basis'
gfortran  -c -m64 -ffast-math  -Warray-bounds -fdefault-integer-8 -march=native -mtune=native -finline-functions -O2 -g -fno-aggressive-loop-optimizations -g -O   -I.  -I/home/dkr/nwchem/nwchem-6.6/src/include -I/home/dkr/nwchem/nwchem-6.6/src/tools/install/include -DGFORTRAN -DCHKUNDFLW -DGCC4 -DGCC46 -DEXT_INT -DLINUX -DLINUX64 -DPARALLEL_DIAG   basis.F
/tmp/ccpY0ZUm.s: Assembler messages:
/tmp/ccpY0ZUm.s:8542: Error: no such instruction: `shlx %rax,968(%rsp),%rax'
/tmp/ccpY0ZUm.s:8585: Error: no such instruction: `shlx %rax,968(%rsp),%rax'
/tmp/ccpY0ZUm.s:8602: Error: no such instruction: `shlx %rax,968(%rsp),%rax'
/tmp/ccpY0ZUm.s:8613: Error: no such instruction: `shlx %rax,968(%rsp),%rax'
/tmp/ccpY0ZUm.s:8961: Error: no such instruction: `shlx %rax,968(%rsp),%rax'
/tmp/ccpY0ZUm.s:11072: Error: no such instruction: `shlx %rax,968(%rsp),%rax'
/tmp/ccpY0ZUm.s:12009: Error: no such instruction: `shlx %rax,616(%rsp),%rdi'
/tmp/ccpY0ZUm.s:12019: Error: no such instruction: `shlx %rax,616(%rsp),%rsi'
/tmp/ccpY0ZUm.s:12806: Error: no such instruction: `shlx %rax,616(%rsp),%rdi'
/tmp/ccpY0ZUm.s:12816: Error: no such instruction: `shlx %rax,616(%rsp),%rsi'
/tmp/ccpY0ZUm.s:13603: Error: no such instruction: `shlx %rax,632(%rsp),%rdi'
/tmp/ccpY0ZUm.s:13613: Error: no such instruction: `shlx %rax,632(%rsp),%rsi'
make[1]: *** [/home/dkr/nwchem/nwchem-6.6/lib/LINUX64/libnwcutil.a(basis.o)] Error 1
make[1]: Leaving directory `/panfs/pan01/vol011/home/dkr/nwchem/nwchem-6.6/src/basis'
make: *** [libraries] Error 1


I have set the environmental variables as listed below. This includes FOPTIMIZE, which I have added in the hope to tame the optimizer and prevent it from using -march=native. Unfortunately, to no avail.

export NWCHEM_TOP=/home/dkr/nwchem/nwchem-6.6
export NWCHEM_TARGET=LINUX64
export NWCHEM_MODULES="all python"
export NWCHEM_LONG_PATHS=Y
export USE_NOFSCHECK=Y
export USE_MPI=y
export USE_MPIF=y
export USE_MPIF4=y
export FC=gfortran
export CC=gcc
export ARMCI_NETWORK=OPENIB
export MSG_COMMS=MPI
export IB_INCLUDE="/usr/include"
export IB_LIB="/usr/lib64"
export IB_LIB_NAME="-libverbs"
export SLURMOPT=-lslurm
export SLURM=y
export BLASOPT="-L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_gf_ilp64 -lmkl_intel_    thread -lmkl_core -lmkl_blacs_openmpi_ilp64 -liomp5 -lpthread -lm -ldl"
export BLAS_SIZE=8
export BLAS_LIB="-L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_gf_ilp64 -lmkl_intel    _thread -lmkl_core -lmkl_blacs_openmpi_ilp64 -liomp5 -lpthread -lm -ldl"
export LAPACK_SIZE=8
export LAPACK_LIB="-L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_gf_ilp64 -lmkl_int    el_thread -lmkl_core -lmkl_blacs_openmpi_ilp64 -liomp5 -lpthread -lm -ldl"
export SCALAPACK_SIZE=8
export SCALAPACK_LIB="-L${MKLROOT}/lib/intel64 -Wl,--no-as-needed -lmkl_scalapack_ilp64 -lmkl_gf_ilp64 -lmkl_    intel_thread -lmkl_core -lmkl_blacs_openmpi_ilp64 -liomp5 -lpthread -lm -ldl"
export FFLAGS="  -fdefault-integer-8 -m64 -I${MKLROOT}/include"
export CFLAGS="  -fdefault-integer-8 -m64 -I${MKLROOT}/include"
export FOPTIMIZE="-O3  -mfpmath=sse -ffast-math -fno-aggressive-loop-optimizations"
export COPTIMIZE="-O3  -mfpmath=sse -ffast-math -fno-aggressive-loop-optimizations"
export PYTHON_EXE=/usr/bin/python
export PYTHONVERSION=2.6
export USE_PYTHON64=yes
export PYTHONPATH=./:/home/dkr/nwchem-6.6/contrib/python/
export PYTHONHOME=/usr
export PYTHONLIBTYPE=so


I would be very thankful for hints as to how to resolve this issue.

Thank you,
Daniel

Forum Vet
Threads 8
Posts 1388
9:48:58 AM PST - Thu, Dec 8th 2016
1) Your failure is likely due to your assembler (/usr/bin/as) not being in sync with the GNU compiler version 4.9; in other words, GNU 4.9 supports Haswell instructions, while as does not.
What is the output of the command
/usr/bin/as -v

2) please do not set FFLAGS,CFLAGS, FOPTIMIZE, COPTIMIZE since it is likely to lead to unexpected results

Just Got Here
Threads 1
Posts 3
12:05:49 PM PST - Thu, Dec 8th 2016
Hi Edoapra,

Quote:Edoapra Dec 8th 9:48 am
1) Your failure is likely due to your assembler (/usr/bin/as) not being in sync with the GNU compiler version 4.9; in other words, GNU 4.9 supports Haswell instructions, while as does not.
What is the output of the command
/usr/bin/as -v

Thank you for your insights. Your guess is spot on:

"as --version" gives:
GNU assembler version 2.20.51.0.2-5.42.el6 20100205

i.e. it's the system's as (i.e. /usr/bin/as). Unfortunately, the compiles (which I pull in via the module system) do not provide their own assemblers (well, at least no as as binaries). Is there a way to have the assembly code pulled in as inline assembly via gcc/gfortran or do I actually have to compile my private gcc including as? Obviously, I would prefer the least invasive approach. Could I possibly rely on a non-optimized reference (i.e. non-ASM) implementation instead? I could give the Intel compilers a try. However, I am facing another issue there ...

Quote:username

2) please do not set FFLAGS,CFLAGS, FOPTIMIZE, COPTIMIZE since it is likely to lead to unexpected results

I am aware of this recommendation. I have added the FOPTIMIZE flag in the hope to avoid this problem as per a similar issue/post on the forum. As for FFLAGS, I tried to live up to the suggestions of Intel's mkl link line advisor. Which alternative way do you suggest to get the recommended compiler options passed on? Is there another environment variable to do the job?

Thank you for your help,
Daniel

Forum Vet
Threads 8
Posts 1388
1:06:50 PM PST - Thu, Dec 8th 2016
Could you try to use -- as a baseline -- the GNU compiler that comes on your RedHat (or Centos) 6 box (out of /usr/bin)?
Have you already try to use this option and found problem with it?

Just Got Here
Threads 1
Posts 3
12:04:07 AM PST - Mon, Dec 12th 2016
Quote:Edoapra Dec 8th 1:06 pm
Could you try to use -- as a baseline -- the GNU compiler that comes on your RedHat (or Centos) 6 box (out of /usr/bin)?
Have you already try to use this option and found problem with it?


Brilliant! Using the system compiler (gfortran 4.4.7) and an older version of the mpich2 libraries (I did not have the openmpi libraries for the older compiler readily available) everything went smoothly. However, this issue left me puzzled as to the functionality of the 4.9.2 compilers. Given that these are supposedly heavily utilized, I am surprised that this problem did not reveal itself previously. Anyway, thank you very much!


Forum >> NWChem's corner >> Compiling NWChem



Who's here now Members 0 Guests 1 Bots/Crawler 0

AWC's: 2.5.10 MediaWiki - Stand Alone Forum Extension
Forum theme style by: AWC
Retrieved from "http://www.nwchem-sw.org/index.php/Special:AWCforum/st/id2207/Intel_E5-2640v3_Haswell:_No_such....html"