Looks like the 6.0 version was compiled over TCP/IP sockets and is run using "parallel.x". 6.1 seems to be a serial version somehow.
Some more info is needed:
1. What kind of platform are you compiling on?
2. We need more of the output to understand here it fails. Also, search for nproc in the output, how many procs is it using?
Bert
Hi, Bert. Thank you for the prompt reply.
1. Platform information:
OS- CentOS 5.7
Intel Dual Socket Motherboard with Intel 5520 Chipset
Intel Xeon E5645 / 6 Core Processors / 2.4GHz / 5.86 QPI GT/sec
48GB DDR3 1066 Mhz ECC Registered System Memory
10/100/1000 BaseT Gigabit Network Adapter on Motherboard
Single Port DDR InfiniBand HCA Mellanox Technologies MT25204 [InfiniHost III Lx HCA]
Custom installation of CBeST v. 3.0 Beowulf cluster software including:
Setup of OS including Custom Kernel for Performance Optimization
Message Passing Libraries (MPI CH, MPI CH2, Open MPI)
TORQUE Batch Scheduling System
Ganglia Cluster Monitoring Utility and Custom Scripting
System Imager for Remote Node Installation, Updating and Addition
PowerScripts for Remote Cluster or Node Reboot, Remote Cluster or Node Shutdown
Security Patches (Port Mapper, IP Chains/Tables) and Latest Security Updates)
Scientific Libraries (LAPACK & BLAS)
GNU Compilers (C & Fortran)
2. JOb-log files contatins nproc= to the number defined in PBS. (nproc > 1)
3. I get this from configuration output (probably it could be useful):
Aggregate Remote Memory Copy Interface (ARMCI) configured as follows:
configure: **************************************************************
configure:
configure: TARGET=LINUX64
configure: MSG_COMMS=TCGMSGMPI
configure: GA_MP_LIBS= -lmpich -lopa -lmpl -lpthread -lrt
configure: GA_MP_LDFLAGS= -L/home/morozov/mpich2/lib -L//home/morozov/mpich2/lib -L/home/morozov/mpich2/lib
configure: GA_MP_CPPFLAGS= -I/home/morozov/mpich2/include -I/home/morozov/mpich2/include
configure: ARMCI_NETWORK=SOCKETS
configure: ARMCI_NETWORK_LDFLAGS=
configure: ARMCI_NETWORK_LIBS=
configure: ARMCI_NETWORK_CPPFLAGS=
configure: F77=/opt/intel/fce/11.1/bin/intel64/ifort
configure: FFLAGS=
configure: FFLAG_INT=-integer-size 64
configure: ARMCI_FOPT=-O3 -w -cm -xW -tpp7
configure: CC=/opt/intel/cce/11.1/bin/intel64/icc
configure: CFLAGS=
configure: ARMCI_COPT=
configure: CPP=/opt/intel/cce/11.1/bin/intel64/icc -E
configure: CPPFLAGS=
configure: LDFLAGS=
configure: LIBS=
configure: FLIBS= -L/opt/intel/fce/11.1/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../.. -L/lib64 -L/lib -L/usr/lib64 -L/usr/lib -lifport -lifcore -limf -lsvml -lm -lipgo -lirc -lpthread -lirc_s -ldl
configure: BLAS_LDFLAGS=
configure: BLAS_LIBS=-lblas
configure: BLAS_CPPFLAGS=
configure: AR=ar
configure: AR_FLAGS=cru
configure: CCAS=/opt/intel/cce/11.1/bin/intel64/icc
configure: CCAS_FLAGS=
configure: DEFS=-DHAVE_CONFIG_H
configure: SHELL=/bin/sh
configure: MPIEXEC=/opt/mpich2/ch3_mrail_gen2-intel11/bin/mpirun -n %NP%
configure: NPROCS=4
configure:
configure:
configure: **************************************************************
configure: Global Arrays (GA) configured as follows:
configure: **************************************************************
configure:
configure: TARGET=LINUX64
configure: MSG_COMMS=TCGMSGMPI
configure: GA_MP_LIBS= -lmpich -lopa -lmpl -lpthread -lrt
configure: GA_MP_LDFLAGS= -L/home/morozov/mpich2/lib -L//home/morozov/mpich2/lib -L/home/morozov/mpich2/lib
configure: GA_MP_CPPFLAGS= -I/home/morozov/mpich2/include -I/home/morozov/mpich2/include
configure: ARMCI_NETWORK=SOCKETS
configure: ARMCI_NETWORK_LDFLAGS=
configure: ARMCI_NETWORK_LIBS=
configure: ARMCI_NETWORK_CPPFLAGS=
configure: F77=/opt/intel/fce/11.1/bin/intel64/ifort
configure: FFLAGS=
configure: FFLAG_INT=-integer-size 64
configure: GA_FOPT=-O3 -w -cm -xW -tpp7
configure: CC=/opt/intel/cce/11.1/bin/intel64/icc
configure: CFLAGS=
configure: GA_COPT=
configure: CPP=/opt/intel/cce/11.1/bin/intel64/icc -E
configure: CPPFLAGS=
configure: LDFLAGS=
configure: LIBS=
configure: FLIBS= -L/opt/intel/fce/11.1/lib/intel64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../.. -L/lib64 -L/lib -L/usr/lib64 -L/usr/lib -lifport -lifcore -limf -lsvml -lm -lipgo -lirc -lpthread -lirc_s -ldl
configure: BLAS_LDFLAGS=
configure: BLAS_LIBS=-lblas
configure: BLAS_CPPFLAGS=
configure: AR=ar
configure: AR_FLAGS=cru
configure: CCAS=/opt/intel/cce/11.1/bin/intel64/icc
configure: CCAS_FLAGS=
configure: DEFS=-DHAVE_CONFIG_H
configure: SHELL=/bin/sh
configure: MPIEXEC=/opt/mpich2/ch3_mrail_gen2-intel11/bin/mpirun -n %NP%
configure: NPROCS=4
Regards,
Alex
