Dealing with FES 2014's all junk value

[maouyami]

Hello, I'm new to SCHISM and getting the FES 2014 data. This question might be a bit of a beginner's problem but...
I was trying the code on the online guide (https://schism-dev.github.io/schism/master/getting-started/pre-processing-with-pyschism/bctides.html), modifying it a bit for FES 2014 values.

import numpy as np

from pyschism.mesh import Hgrid
from pyschism.forcing.bctides import Bctides

if __name__ == '__main__':
    start_date = datetime(2017, 12, 1)
    rnday = 396
    bctypes = [[5, 5, 4, 4]]
    constituents = 'major'
    database = 'fes2014'
    earth_tidal_potential = True
    sthconst = [np.nan, np.nan, 0]
    tobc = [0.5, 0.5, 1]
    sobc = [0.5, 0.5, 1]
    outdir = './'
    hgrid = Hgrid.open("F:/Model/schism/hgrid.gr3", crs="epsg:4326")

    bctides = Bctides(
        hgrid = hgrid,
        flags = bctypes,
        constituents = constituents,
        database = database,
        add_earth_tidal = earth_tidal_potential,
        sthconst = sthconst,
        #tobc = tobc,
        #sobc = sobc,
    )

    bctides.write(
        outdir, 
        start_date=start_date, 
        rnday=rnday, 
        overwrite=True,
    )

Im sure i downloaded all of the FES 2014's data (eastward velocity, northward velocity, ocean tide extrapolated) from their ftp server. However when i run the code, it took an error

runfile('F:/Model/schism/get_bctides.py', wdir='F:/Model/schism')
C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\hycom\gofs.py:8: UserWarning: The seawater library is deprecated! Please use gsw instead.
  import seawater as sw
C:\ProgramData\anaconda3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm
Combination of 3 and 4, time history of elevation is read in from elev2D.th.nc!
Combination of 3 and 4, time history of velocity is read in from uv.3D.th.nc!
Traceback (most recent call last):

  File C:\ProgramData\anaconda3\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File f:\model\schism\get_bctides.py:31
    bctides.write(

  File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\bctides.py:265 in write
    f.write(str(self))

  File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\bctides.py:162 in __str__
    uamp, uphase, vamp, vphase = self.tides.get_velocity(constituent, vertices)

  File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\tides.py:160 in get_velocity
    return self.tidal_database.get_velocity(constituent, vertices)

  File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\fes2014.py:53 in get_velocity
    uamp = self._get_interpolation(

  File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\fes2014.py:135 in _get_interpolation
    raise ValueError('All junk values for {phys_var} {constituent}')

ValueError: All junk values for {phys_var} {constituent}

I assumed it was about FES 2014's velocity files. Any way to deal with this?

[quynhmoc95]

I have the same problem as you had. Did you solve it?
If you did, how can you do it? I am new to SCHISM. So I hope I can find some answer about this matter.

Thanks,

[maouyami]

I have the same problem as you had. Did you solve it? If you did, how can you do it? I am new to SCHISM. So I hope I can find some answer about this matter.

Thanks,

Unfortunately I did not solved it yet,
However, I modified the fes2014.py file to shows which tidal components is different (or faulty) from the others.
Heres the result:

runfile('F:/Model/schism/get_bctides.py',```` wdir='F:/Model/schism')
C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\hycom\gofs.py:8: UserWarning: The seawater library is deprecated! Please use gsw instead.
import seawater as sw
C:\ProgramData\anaconda3\Lib\site-packages\tqdm\auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
Current working directory: F:\Model\schism
Kinh độ của lưới: 105.6199918505 - 112.3298940722
Vĩ độ của lưới: 14.9675933298 - 21.6610012503
Combination of 3 and 4, time history of elevation is read in from elev2D.th.nc!
Combination of 3 and 4, time history of velocity is read in from uv.3D.th.nc!
Traceback (most recent call last):

File C:\ProgramData\anaconda3\Lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
exec(code, globals, locals)

File f:\model\schism\get_bctides.py:38
bctides.write(

File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\bctides.py:344 in write
f.write(str(self))

File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\bctides.py:164 in str
uamp, uphase, vamp, vphase = self.tides.get_velocity(constituent, vertices)

File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\tides.py:160 in get_velocity
return self.tidal_database.get_velocity(constituent, vertices)

File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\fes2014.py:53 in get_velocity
uamp = self._get_interpolation(

File C:\ProgramData\anaconda3\Lib\site-packages\pyschism\forcing\bctides\fes2014.py:148 in _get_interpolation
raise ValueError(error_message)

ValueError: eastward_vel Ua M2.nc
Invalid indices: (array([ 0, 27], dtype=int64),)
vmax: [1.8446744e+19 8.4512513e-03 5.5163861e-03 3.9384118e-03 5.2882191e-03
6.0025798e-03 4.9228058e-03 6.1235116e-03 5.3264354e-03 7.5838496e-03
1.3425043e-02 9.4353696e-03 9.5735937e-03 1.1473537e-02 1.1305908e-02
1.1489985e-02 1.2566685e-02 1.7248653e-02 2.1635173e-02 3.6730990e-02
6.4911604e-02 6.1305098e-02 6.6692084e-02 7.3599495e-02 8.7373629e-02
9.3458414e-02 1.0062890e-01 1.8446744e+19]
vmin: [1.8446744e+19 7.8424215e-03 5.3467820e-03 3.8721431e-03 4.7905520e-03
4.8059374e-03 4.6890783e-03 5.5117430e-03 4.7293580e-03 5.6979363e-03
8.9595979e-03 8.8991886e-03 8.9974543e-03 1.1003058e-02 1.0865458e-02
1.1203669e-02 1.1839091e-02 1.4861172e-02 2.0431550e-02 2.7470373e-02
5.5983342e-02 5.8051080e-02 6.3898563e-02 7.0579819e-02 8.4584258e-02
8.8988565e-02 8.8855051e-02 1.8446744e+19]
values: [1.8446744073709552e+19 1.8446744073709552e+19]

Maybe it has to do something to convert these numbers to the appropiate one...

[quynhmoc95]

Thank you so much. I also found the problem came from fill_value in the FES2014 data. We have to deal with the missing data here. I have been fixing this Python code. If I did it, I will update the file here.

By the way, I used this matlab code to create bctide.in "https://github.com/schism-dev/schism/blob/master/src/Utility/Tides/one_click_bctides_schism_fes2014.m", but I modified it as this:
At the line 566
amp0= netcdf.getVar(ncid, vid);
we need to add 2 lines after this:
indx = find(amp0>(1.8447e+19)/10)) %noted: fill_value = 1.8447e+19
amp0(indx) = amp0(indx)*0.0

That's what i did in the matlab code.
Thanks,

[quynhmoc95]

Hello,

This is the new version of my fes2014.py. I added some lines for dealing with the missing data and it worked.
If you are still not find the solution, you can replace my code with your old one.

Thanks,
fes2014.py.txt