PYROMS error

Discussion about analysis, visualization, and collaboration tools and techniques

Moderators: arango, robertson

Post Reply
Message
Author
dhirendra.cub
Posts: 2
Joined: Mon Dec 19, 2016 8:04 pm
Location: JNU

PYROMS error

#1 Unread post by dhirendra.cub »

I have installed pyroms using python 2.7 in conda. The installation was successful but I am not being able to import pyroms to check whether it is installed or not. It give the error as following:

>>> import pyroms
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ROMS/anaconda2/lib/python2.7/site-packages/pyroms/__init__.py", line 13, in <module>
from . import extern
File "/home/ROMS/anaconda2/lib/python2.7/site-packages/pyroms/extern/__init__.py", line 4, in <module>
from .greatcircle import GreatCircle
File "/home/ROMS/anaconda2/lib/python2.7/site-packages/pyroms/extern/greatcircle.py", line 415
print(" Flinders Peak = %3i\xF8%3i\' %6.3f\", " % ( deg, minn, sec ), end=' ')
^
SyntaxError: invalid syntax

Again, I tried the same with python 3.5 (as mentioned on pyroms github page) in conda, but import pyroms ended up with segmentation fault repeatedly.

Any insight in overcoming the error is appreciated.

Thanks in anticipation

Dhirendra

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#2 Unread post by kate »

The "master" branch of pyroms should work with Python 2.7. The code diffs for that line are:

Code: Select all

-   print " Flinders Peak =   %3i\xF8%3i\' %6.3f\",  " % ( deg, minn, sec ),
+   print(" Flinders Peak =   %3i\xF8%3i\' %6.3f\",  " % ( deg, minn, sec ), end=' ')
I have the new branch working in Python version 3.6.2.

bangdt
Posts: 30
Joined: Sun Mar 10, 2019 2:34 pm
Location: University of Tsukuba

Re: PYROMS error

#3 Unread post by bangdt »

kate wrote:The "master" branch of pyroms should work with Python 2.7. The code diffs for that line are:

Code: Select all

-   print " Flinders Peak =   %3i\xF8%3i\' %6.3f\",  " % ( deg, minn, sec ),
+   print(" Flinders Peak =   %3i\xF8%3i\' %6.3f\",  " % ( deg, minn, sec ), end=' ')
I have the new branch working in Python version 3.6.2.
Dear Kate

Could you please kindly share your new branch working in Python version 3.6.2 and some guide to install and using it?

I have ubuntu 18.04 with anaconda3, python 3.6 installed. I have already try to install pyroms with no success. The installed was run smoothly with no error but when i try to import module I have some error because of lacking lpsolve55.so, Script.so and libgu.so.

Thank you.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#4 Unread post by kate »

Type "git branch" to see which branch you have. The "python3" branch is the current default, so that is probably what you have already.

As for those libraries, you need to make sure your LD_LIBRARY_PATH environment variable includes the directory they live in.

bangdt
Posts: 30
Joined: Sun Mar 10, 2019 2:34 pm
Location: University of Tsukuba

Re: PYROMS error

#5 Unread post by bangdt »

kate wrote:Type "git branch" to see which branch you have. The "python3" branch is the current default, so that is probably what you have already.

As for those libraries, you need to make sure your LD_LIBRARY_PATH environment variable includes the directory they live in.
Dear Kate
Thank you so much for your reply, I have tried to reinstall anaconda an python to python 2.6.8 in order to match the pyroms-master requiremenet. Now I have compiled all the lib.
but when I try to install bathy_smoother I got some error and failth.

for pyroms and pyroms-toolbox which completed. I try to import them but still got error at this step with
----------------------------------------------------------------------------------------------------
Traceback (most recent call last):
File "/work/apps/anaconda3/lib/python2.7/site-packages/pyroms/cf.py", line 20, in <module>
import pyroms.io
File "/work/apps/anaconda3/lib/python2.7/site-packages/pyroms/__init__.py", line 14, in <module>
import hgrid
File "/work/apps/anaconda3/lib/python2.7/site-packages/pyroms/hgrid.py", line 19, in <module>
from matplotlib.nxutils import points_inside_poly
ImportError: No module named nxutils
---------------------------------------------------------------------------------------------------

I think that new matplotlib does not have nxutils so
I already tried to rolback by

matplotlib pkgs/main::matplotlib-2.2.3-py27hb69d~ --> uvcdat/label/old::matplotlib-1.5.0-py27_0

but the error still remain

Please help me the link to the right dependencies to download, I am totally messed up now. sorrry!!

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#6 Unread post by kate »

No, don't go backwards! We always had problems with python finding those libraries, so going backwards is not the answer. Stick with Python 3 and read the README where I tell you what I have to do for python to find the libraries - I just have copies in three different places, with hopes that at least one will be found. Your LD_LIBRARY_PATH needs to point to one of them.

bangdt
Posts: 30
Joined: Sun Mar 10, 2019 2:34 pm
Location: University of Tsukuba

Re: PYROMS error

#7 Unread post by bangdt »

kate wrote:No, don't go backwards! We always had problems with python finding those libraries, so going backwards is not the answer. Stick with Python 3 and read the README where I tell you what I have to do for python to find the libraries - I just have copies in three different places, with hopes that at least one will be found. Your LD_LIBRARY_PATH needs to point to one of them.
Dear Kate

Now I come back to anaconda3 with python 3.6 and try to install pyroms-python3 again
I have all netcdf and netcdf-fortran installed but with nf-config --all I've got
"nf-config not yet implemented for cmake builds"

so with makefile2 under ~/pyroms-python3/pyroms/external/scrip/source I have

NC_CONFIG = nf-config
LIBDIR = $(shell $(NC_CONFIG) --prefix)/lib
INCDIR = $(shell $(NC_CONFIG) --prefix)/include
SRCDIR = .
PREFIX = /work/apps/anaconda3/lib

these code can not be pass because of nf-config problem.

the README file in ~/pyroms-python3/pyroms suggest that
----------------------------------------------------------------------
"I had to hack the scrip makefile for the fortran90 netcdf stuff. The
anaconda netcdf-fortran reports:
nf-config not yet implemented for cmake builds

A note on the .so files from fortran: They might now end up with names like:

scrip.cpython-35m-x86_64-linux-gnu.so

It's OK - they'll load as long as they are in your PYTHONPATH. You might
also need libgu.so to be in your LD_LIBRARY_PATH. I'm getting inconsistent
results with where the .so files need to be. Best results for me are if I go
to the site-packages directory where pyroms got installed and copy all the
.so files:

cp pyroms/*.so .
cp pyroms_toolbox/*.so .
cp bathy_smoother/*.so .
"
-----------------------------------------------------------------------
Now I still can not pass the nf-config problem, I got these errors

(base) root@C2:/media/bangdt/Ext4/0Setup/Python/pyroms-python3/pyroms/external/scrip/source# make -f makefile2
gfortran -g -fdefault-real-8 -ffixed-form -O2 -fPIC -Inf-config not yet implemented for cmake builds/include -c ./kinds_mod.f
gfortran: error: not: No such file or directory
gfortran: error: yet: No such file or directory
gfortran: error: implemented: No such file or directory
gfortran: error: for: No such file or directory
gfortran: error: cmake: No such file or directory
gfortran: error: builds/include: No such file or directory
makefile2:84: recipe for target 'kinds_mod.o' failed
make: *** [kinds_mod.o] Error 1


I still do not understand how to fix and run the makefile2. Please give me more instructions to solve this.

Thank you so much for your great works.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#8 Unread post by kate »

This troublesome bit:

Code: Select all

 -Inf-config not yet implemented for cmake builds/include
is coming from:

Code: Select all

INCDIR = $(shell $(NC_CONFIG) --prefix)/include
which as you point out is from nf-config. It is executing "nf-config --prefix" which on my machine gives:

Code: Select all

chinook04.rcs.alaska.edu 543% nf-config --prefix
/home/kshedstrom
Do you get a better result from "nc-config --prefix"?

Alternatively, you need to somehow provide makefile2 with the path to 'netcdf.inc', the Fortran include file for netCDF. You can change:

Code: Select all

LIBDIR = $(shell $(NC_CONFIG) --prefix)/lib
INCDIR = $(shell $(NC_CONFIG) --prefix)/include
to something like:

Code: Select all

LIBDIR = /usr/local/netcdf/lib
INCDIR = /usr/local/netcdf/include
assuming that is where your files live.

bangdt
Posts: 30
Joined: Sun Mar 10, 2019 2:34 pm
Location: University of Tsukuba

Re: PYROMS error

#9 Unread post by bangdt »

kate wrote:This troublesome bit:
Alternatively, you need to somehow provide makefile2 with the path to 'netcdf.inc', the Fortran include file for netCDF. You can change:

Code: Select all

LIBDIR = $(shell $(NC_CONFIG) --prefix)/lib
INCDIR = $(shell $(NC_CONFIG) --prefix)/include
to something like:

Code: Select all

LIBDIR = /usr/local/netcdf/lib
INCDIR = /usr/local/netcdf/include
assuming that is where your files live.
Dear Kate

Thank you so much for your useful advice. I finally have pyroms installed. But there is still some more problems

1st when I try to load pyroms_toolbox I have
---------------------------------------------------------------
>>> import pyroms_toolbox

Traceback (most recent call last):
File "<pyshell#4>", line 1, in <module>
import pyroms_toolbox
File "/work/apps/anaconda3/envs/PYROMS/lib/python2.7/site-packages/pyroms_toolbox/__init__.py", line 52, in <module>
from TS_diagram import TS_diagram
File "/work/apps/anaconda3/envs/PYROMS/lib/python2.7/site-packages/pyroms_toolbox/TS_diagram.py", line 8, in <module>
def TS_diagram(temp, salt, depth=None, dens_lev=None, marker_size=2, fmt='%2.2f', pal=cm.spectral, \
AttributeError: 'module' object has no attribute 'spectral'
---------------------------------------------------------------------------
by comment out the error line I can import pyroms_toolbox
...
from _move_runoff import move_runoff
from _move_river_t import move_river_t
#from TS_diagram import TS_diagram
from date2jday import date2jday
from jday2date import jday2date
...

2nd I have tried to copy lpsolve55.so and liblpsolve55.so to all PATh by following
export LD_LIBRARY_PATH=/work/apps/anaconda3/envs/PYROMS/lib
export PYTHONPATH=/work/apps/anaconda3/envs/PYROMS/lib

but i still have the not found error of lpsolve55.so

But is there any solutions to fix them?

Thank you.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#10 Unread post by kate »

Spectral has a capital "S". See for example this Stackoverflow thread. I don't know how you ended up with a lower case spectral.

As for the other, my solution is that I don't use lpsolve. I think I once copied the thing to my current directory and that worked.

bangdt
Posts: 30
Joined: Sun Mar 10, 2019 2:34 pm
Location: University of Tsukuba

Re: PYROMS error

#11 Unread post by bangdt »

kate wrote:Spectral has a capital "S". See for example this Stackoverflow thread. I don't know how you ended up with a lower case spectral.

As for the other, my solution is that I don't use lpsolve. I think I once copied the thing to my current directory and that worked.
Dear Kate

Thank you for correct me about the "S", I change it and it run smoothly. I dont know why but my install come with the "Spectra"

Could you please give me more instructions about how to pass or not use lpsolve?

Thank you so much.

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#12 Unread post by kate »

The only use I know of for lpsolve is to smooth the bathymetry. I usually use pyroms for other tasks, not grid generation. What do you need pyroms to do for you?

bangdt
Posts: 30
Joined: Sun Mar 10, 2019 2:34 pm
Location: University of Tsukuba

Re: PYROMS error

#13 Unread post by bangdt »

kate wrote:The only use I know of for lpsolve is to smooth the bathymetry. I usually use pyroms for other tasks, not grid generation. What do you need pyroms to do for you?
Dear Kate
Thank you so much for your very quick and useful instructions. I am going to make boundary and initial file for r4dvar case test.
I am newbie in roms. Up to now I can run wc13 test and restart it from his file.
Also I can make grid file. My plan is run a real case for my domain and study mct run with wrf (I can run wrf real case for my domain already).
Thank again and have a nice weakend.

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error in grid generation

#14 Unread post by reba1 »

Hi everyone,
I am facing the error of segmentation fault while executing the make_remap_weights_file.py.
Assuming spherical is integer b'T' <class 'numpy.ma.core.MaskedArray'>
Load geographical grid from file
grid shape 338 640
grid shape 338 639
grid shape 337 640
Segmentation fault

Any help with this issue....

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#15 Unread post by kate »

What exactly are you running? It has been a few years since I got scrip to work for me, so I'm switching over to ESMPy. There's one HYCOM example of it.

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#16 Unread post by reba1 »

Hi Kate,
Thanks for the reply,I am running Palau_HYCOM example for another study domain.Using gridbuilder ,I created grid for the study region,once I run the make_remap_weights_file.py ,I am facing above error

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#17 Unread post by reba1 »

Hi Kate,
I am getting following error while running the get_nc_Grid_HYCOM2.py


z_t = np.tile(depth,(mask_t.shape[2],mask_t.shape[1],1)).T
IndexError: tuple index out of range

How to overcome this problem

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#18 Unread post by kate »

I don't know. What are the shapes of all the things?

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#19 Unread post by reba1 »

Thanks Kate, for the reply
I think some issues with my src_gridfile!!
The src file is as follows:-

netcdf HYCOM_GLBy0.08_2022_065 {
dimensions:
lon = 438 ;
lat = 751 ;
ocean_time = UNLIMITED ; // (1 currently)
z = 40 ;
variables:
float lon(lon) ;
lon:long_name = "longitude" ;
lon:units = "degrees_east" ;
float lat(lat) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
float ocean_time(ocean_time) ;
ocean_time:units = "days since 1900-01-01 00:00:00" ;
ocean_time:calendar = "gregorian" ;
float ssh(ocean_time, lat, lon) ;
ssh:_FillValue = -30000.f ;
ssh:long_name = "Water Surface Elevation" ;
ssh:units = "m" ;
ssh:coordinates = "lon lat" ;
float z(z) ;
z:long_name = "depth" ;
z:units = "meter" ;
float temp(ocean_time, z, lat, lon) ;
temp:_FillValue = -30000.f ;
temp:long_name = "Water Temperature" ;
temp:units = "degC" ;
temp:coordinates = "lon lat" ;
float salt(ocean_time, z, lat, lon) ;
salt:_FillValue = -30000.f ;
salt:long_name = "Salinity" ;
salt:units = "psu" ;
salt:coordinates = "lon lat" ;
float u(ocean_time, z, lat, lon) ;
u:_FillValue = -30000.f ;
u:long_name = "Eastward Water Velocity" ;
u:units = "m/s" ;
u:coordinates = "lon lat" ;
float v(ocean_time, z, lat, lon) ;
v:_FillValue = -30000.f ;
v:long_name = "Northward Water Velocity" ;
v:units = "m/s" ;
v:coordinates = "lon lat" ;

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#20 Unread post by kate »

I meant inside Python, print out the shapes of the objects in Python.

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#21 Unread post by reba1 »

Hi Kate,
The variables list along with error

>>> print(lon.shape)
(438,)
>>> print(depth.shape)
(40,)
>>> print(lat.shape)
(751,)
>>> print(ssh.shape)
(751, 438)
>>> print(var.shape)
(40, 751, 438)
>>> mask_t = np.array(~var[:].mask, dtype='int')
>>> print(mask_t)
1
>>> z_t = np.tile(depth,(mask_t.shape[2],mask_t.shape[1],1)).T
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: tuple index out of range
>>>

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#22 Unread post by kate »

So the shape of mask_t is wrong for some reason. What does that "~" do?

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#23 Unread post by reba1 »

Hi Kate,

Thanks for reply,actually i havnt changed the code, ~is present in the new version of pyroms.
"~" returns true,false,1,0
Can you help with the 'mask' variable,based on what condition ,masking happens
mask_t = np.array(~var[:].mask, dtype='int')

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#24 Unread post by kate »

It's not working correctly and my husband has no idea why either. mask_t should have three dimensions, just like var.
He says the "~" should be returning valid entries. We want to know which points are masked out. For me, ncview
looking at the temp variable shows some white patches where it's land or under the sea bottom.

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#25 Unread post by reba1 »

Hi Kate,
Thanks for the reply.
My input hycom netcdf file is having negative fill values,it created problem in masking .Now I am facing another error while running another script remap_bdry.py
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (103,) and requested shape (1,30)
Can you help me with this problem

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#26 Unread post by kate »

remap_bdry.py should not be run stand-alone, but rather is called by make_bdry_file.py.

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#27 Unread post by reba1 »

Hi Kate,

Actually once I run the make_bdry.py ,this error came.
zeta = remap_bdry(file, 'ssh', src_grd, dst_grd, dst_dir=dst_dir)


Can you help me to understand what is causing this kind of error
src grid=hycom_glby grid
dimensions:
ocean_time = UNLIMITED ; // (1 currently)
lon = 438 ;
lat = 751 ;
z = 40 ;

dest grid=roms_grd
dimensions:
xi_rho = 103 ;
eta_rho = 53 ;
xi_u = 102 ;
eta_u = 53 ;
xi_v = 103 ;
eta_v = 52 ;
xi_psi = 102 ;
eta_psi = 52 ;
one = 1 ;
s_rho = 15 ;
s_w = 16 ;

User avatar
kate
Posts: 4088
Joined: Wed Jul 02, 2003 5:29 pm
Location: CFOS/UAF, USA

Re: PYROMS error

#28 Unread post by kate »

No, sorry, packing for a trip, not able to help right now.

reba1
Posts: 11
Joined: Thu Feb 24, 2022 9:24 pm
Location: india

Re: PYROMS error

#29 Unread post by reba1 »

Hi Kate,
Thanks for the help provided during our previous discussion .Happy journey!!

Post Reply