openmpi problem

[zwwpku]

Hi all,
I am using mac os x 10.6.8 (x86_64), and have gfortran as fortran compiler. I downloaded openmpi and installed it.
When I try to compile ROMS with MPI, I got the error message as follows:
========================================================================
/usr/local/gfortran/bin/gfortran -c -frepack-arrays -O3 -I/usr/include master.f90
/usr/local/gfortran/bin/gfortran -frepack-arrays -O3 -I/usr/include Build/esmf_roms.o Build/master.o Build/ocean_control.o Build/ocean_coupler.o Build/propagator.o Build/roms_export.o Build/roms_import.o -o oceanM Build/libUTIL.a Build/libNLM.a Build/libNLM_bio.a Build/libNLM_sed.a Build/libANA.a Build/libMODS.a Build/libUTIL.a Build/libMODS.a -L/opt/gfortran/serial/netcdf3/lib -lnetcdf -L/usr/lib -lfmpi -lmpi
ld: library not found for -lfmpi
collect2: ld returned 1 exit status
make: *** [oceanM] Error 1
========================================================================
I found under the library directory I specified for MPI there are:
libmpi.0.dylib libmpi_cxx.0.dylib libmpi_f77.0.dylib libmpi_f90.a libvt.mpi.a
libmpi.dylib libmpi_cxx.dylib libmpi_f77.dylib libmpi_f90.la libvt.ompi.a
libmpi.la libmpi_cxx.la libmpi_f77.la libvt.fmpi.a

Can anyone help me or tell me how to solve the problem?

Thanks,
Vivian

[kate]

What is the directory where these libraries live? Is it one of the -L locations in that link statement?

Many mpi implementations ask that you compile and link with the mpif90 command, which is a shell wrapper around your compiler. This is invoked with USE_MPIF90 in the Makefile/build script. Have you tried that? It will find the correct libraries and include files (hopefully).

[zwwpku]

Hi Kate,
The mpi related libraries are located in the /usr/lib directory as I specified.

And I did use mpif90 to compile and link "successfully" without errors. However when I use 'mpirun -np 20 ./oceanM < ocean.in', the following errors occur:
mca: base: component_find: unable to open /usr/lib/openmpi/mca_ras_dash_host: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_ras_gridengine: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_ras_localhost: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_ras_xgrid: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_errmgr_hnp: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_errmgr_orted: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_errmgr_proxy: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_iof_proxy: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)
[zhareng.ceoe.udel.edu:38184] mca: base: component_find: unable to open /usr/lib/openmpi/mca_iof_svc: perhaps a missing symbol, or compiled for a different version of Open MPI? (ignored)


========================================================================
And I checked under directory of /usr/lib/openmpi/, and found that those mca_* files that cannot be found are dated long ago when the machine was purchased, while other mc_*files are dated when I installed the openmpi.


Thanks,
Vivian

[kate]

Each MPI is going to have its own runmpi and mpif90. You need to be sure these are consistent with each other and the ones you want. That's why our system manager use a "module" system to keep them separate - and to have only one visible at a time (ideally). Try "which runmpi" and "which mpif90". If you are invoking runmpi from a script, make sure the script's environment is what you want it to be.

'mpirun -np 20 ./oceanM < ocean.in'

This should be without the "<" so that ocean.in is an argument to oceanM.

[zwwpku]

Hi Kate,
Thank you for your reply. I also tried without the '<', but the result is just the same.

I don't know about runmpi, I do which runmpi, nothing returned.

[kate]

Sorry, I meant mpirun. I have a script named runmpi which I submit with "qsub runmpi". So for me, the script needs to have an environment which is consistent with the build environment. A "which mpirun" should return the same thing both on the command line and in the script - and should point to the same installation as "which mpif90".

[zwwpku]

Hi, I just found out the problem is actually hardware related somehow.
I cannot use all the two threads of each core to run the model, I can only employ 10 cores at the most.

Thanks, Kate!