observation data rejected!
-
- Posts: 60
- Joined: Tue Sep 29, 2009 3:50 pm
- Location: School of Environment System Engineering,UWA
observation data rejected!
Number of State Observations Processed: ObsTime = 40124.0000
Variable IstrObs IendObs Count Rejected
temp 1 33 33 32
Total 33 32
Obs Tally 33 32
Wrote NLM state at observation locations, datum = 0000001 - 0000033
Almost all of my data were rejected.Does anyone have the similar problems?
Variable IstrObs IendObs Count Rejected
temp 1 33 33 32
Total 33 32
Obs Tally 33 32
Wrote NLM state at observation locations, datum = 0000001 - 0000033
Almost all of my data were rejected.Does anyone have the similar problems?
Re: observation data rejected!
Have you checked to make sure that your obs file variables are all in the correct format? The model is very particular in how it reads in the data.
- arango
- Site Admin
- Posts: 1361
- Joined: Wed Feb 26, 2003 4:41 pm
- Location: DMCS, Rutgers University
- Contact:
Re: observation data rejected!
The 4D-Var algorithms reject data if the input observations locations are not in inside the application grid or inside the assimilation time window. Notice the the spatial locations (obs_Xgrid, obs_Ygrid, obs_Zgrid) are in in grid fractional coordinates (0:L, 0:M, 1:N) and the time coordinate (survey_time, obs_time) is in days with the same calendar as the model time and forcing (for example, days since 1968-05-23 00:00:00 GMT).
Users need to check the observations before assimilating. There is not quality control on the observations inside ROMS. This needs to be done during pre-processing.
Users need to check the observations before assimilating. There is not quality control on the observations inside ROMS. This needs to be done during pre-processing.
-
- Posts: 28
- Joined: Fri Dec 28, 2012 5:44 am
- Location: National Marine Environmental Forecasting Center
Re: observation data rejected!
hi,
i come with the same question. Gauge data is assimilated to the model and i check every point to make sure
Anyone know why?
i come with the same question. Gauge data is assimilated to the model and i check every point to make sure
. The cycle runs 1 day and data per hour has prepared for the poried. Only one time level data is accepted。the spatial locations (obs_Xgrid, obs_Ygrid, obs_Zgrid) are in in grid fractional coordinates (0:L, 0:M, 1:N) and the time coordinate (survey_time, obs_time) is in days with the same calendar as the model time and forcing
Anyone know why?
-
- Posts: 28
- Joined: Fri Dec 28, 2012 5:44 am
- Location: National Marine Environmental Forecasting Center
Re: observation data rejected!
And also,
i prepare 24 gauge data along the coast, and restore it in form of grid.Then 'd_ssh_obs.m' is changed to process the grid data. The result is, only 7 diffetent gauge data writes to the obs.nc.Why?
i prepare 24 gauge data along the coast, and restore it in form of grid.Then 'd_ssh_obs.m' is changed to process the grid data. The result is, only 7 diffetent gauge data writes to the obs.nc.Why?
-
- Posts: 12
- Joined: Wed Jun 04, 2014 1:45 pm
- Location: UFBA and REMO
Re: observation data rejected!
I'm also having SST gridded satellite data getting rejected.
About 195 out of 204 SST data points are being rejected. And only 4 out of 204 SSH data points were rejected.
I'm setting the SST (temp) depth to 2 cm and the minimum error to 0.2oC.
Please let me know if you have any ideas.
Fragments of the log file below:
About 195 out of 204 SST data points are being rejected. And only 4 out of 204 SSH data points were rejected.
I'm setting the SST (temp) depth to 2 cm and the minimum error to 0.2oC.
Please let me know if you have any ideas.
Fragments of the log file below:
Code: Select all
" Number of State Observations Processed: ObsTime = 9585.0000, 2016-03-30 00:00:00.00
Variable IstrObs IendObs Count Rejected
zeta 1 204 204 4
Total 204 4
Obs Tally 204 4
...
Number of State Observations Processed: ObsTime = 9585.5000, 2016-03-30 12:00:00.00
Variable IstrObs IendObs Count Rejected
temp 205 408 204 195
Total 204 195
Obs Tally 408 199
...
Number of State Observations Processed: ObsTime = 9586.0000, 2016-03-31 00:00:00.00
Variable IstrObs IendObs Count Rejected
zeta 409 612 204 4
Total 204 4
Obs Tally 612 203
...
Number of State Observations Processed: ObsTime = 9586.5000, 2016-03-31 12:00:00.00
Variable IstrObs IendObs Count Rejected
temp 613 816 204 189
Total 204 189
Obs Tally 816 392
"
Re: observation data rejected!
Hi Raf.
Did you check your obs_Zgrid?
Another thing to check is if you have BGQC activated. This will reject data based on the parameters set in your s4dvar.in
Cheers
Joao
Did you check your obs_Zgrid?
Another thing to check is if you have BGQC activated. This will reject data based on the parameters set in your s4dvar.in
Cheers
Joao
Re: observation data rejected!
Rafael,
Look in the mod file. It reports all the variables that ROMS uses to make a decision as to which observations are rejected.
There you will see reported the innovations, obs and background errors. If you have BGQC active you can determine whether it is configured as you intended for your specified error hypotheses. Be careful to note whether values are standard deviations or variances.
Data outside the domain will be rejected, so make sure your obs coordinates are correct. Mishandling of the vertical grid is a common mistake.
John.
Look in the mod file. It reports all the variables that ROMS uses to make a decision as to which observations are rejected.
There you will see reported the innovations, obs and background errors. If you have BGQC active you can determine whether it is configured as you intended for your specified error hypotheses. Be careful to note whether values are standard deviations or variances.
Data outside the domain will be rejected, so make sure your obs coordinates are correct. Mishandling of the vertical grid is a common mistake.
John.
John Wilkin: DMCS Rutgers University
71 Dudley Rd, New Brunswick, NJ 08901-8521, USA. ph: 609-630-0559 jwilkin@rutgers.edu
71 Dudley Rd, New Brunswick, NJ 08901-8521, USA. ph: 609-630-0559 jwilkin@rutgers.edu
-
- Posts: 12
- Joined: Wed Jun 04, 2014 1:45 pm
- Location: UFBA and REMO
Re: observation data rejected!
Thanks Joao and John for your replies.
The problem was in obs_Zgrid. I've changed the depth of temp from 0.02 to 0.5 m in the scripts I'm using and I got better values for obs_Zgrid (below).
Before:
obs_Zgrid = -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, 29.9982020984498, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, 29.9931272988072, 29.9914863532605,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, 0, 0, 0, 0, -0, -0, -0, -0
After:
obs_Zgrid = 29.7888158381515, 29.5984096106621, 29.6626000352451,
29.756141155388, 29.6407162246107, 29.8618950435596, 29.7500118677251,
29.7599251774095, 29.6392872744959, 29.7943965092529, 29.762106521323,
29.8524734031373, 29.7938844741204, 29.809034937104, 29.7090891536804,
29.644895458076, 29.5965694954022, 29.627895167986, 29.6355442815496 ...
The BGQC is on and (bgqc_type == 1) and applied to tracers: S_bgqc(isTvar) == 4.0d0 4.0d0 ! 1:NT tracers
I've started to check the mod.nc files to have a summary of each DA cycle.
Thanks,
Rafael
The problem was in obs_Zgrid. I've changed the depth of temp from 0.02 to 0.5 m in the scripts I'm using and I got better values for obs_Zgrid (below).
Before:
obs_Zgrid = -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, 29.9982020984498, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, 29.9931272988072, 29.9914863532605,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0, -0,
-0, -0, -0, -0, -0, -0, -0, -0, -0, -0, 0, 0, 0, 0, -0, -0, -0, -0
After:
obs_Zgrid = 29.7888158381515, 29.5984096106621, 29.6626000352451,
29.756141155388, 29.6407162246107, 29.8618950435596, 29.7500118677251,
29.7599251774095, 29.6392872744959, 29.7943965092529, 29.762106521323,
29.8524734031373, 29.7938844741204, 29.809034937104, 29.7090891536804,
29.644895458076, 29.5965694954022, 29.627895167986, 29.6355442815496 ...
The BGQC is on and (bgqc_type == 1) and applied to tracers: S_bgqc(isTvar) == 4.0d0 4.0d0 ! 1:NT tracers
I've started to check the mod.nc files to have a summary of each DA cycle.
Thanks,
Rafael
Re: observation data rejected!
Hello,
I am seeing some unusual behaviour with Zgrid when running VERIFICATION.
In my input obs.nc file I have Zgrid=50 and depth=-2 for SST obs (my model has 50 layers).
When I run the VERIFICATION, the obs file is written to and altered. For the first day (of 6 daily times for which I have SST data) the Zgrid is altered from 50 to zero in some places and to range from 30-50 in others. At all following days Zgrid is left at 50.
The SST values that are output to the mod file are incorrect when the newly written Zgrid=50 and Zgrid=0, and only make ‘some’ sense when Zgrid is between 30-50 i.e. on day 1.
I tried writing Zgrid as 49.8 rather than 50 but I still get the same error. I also tried leaving Zgrid at 50 and changed depth to 0 for SST, same error. I also tried swapping Zgrid and depth in the obs file, as I have done in 4D-Var applications in the past, but still get the same error.
For SSH I have Zgrid=50 and depth=0 and the data is output correctly. Likewise I have some subsurface obs that are output correctly.
I am seeing some unusual behaviour with Zgrid when running VERIFICATION.
In my input obs.nc file I have Zgrid=50 and depth=-2 for SST obs (my model has 50 layers).
When I run the VERIFICATION, the obs file is written to and altered. For the first day (of 6 daily times for which I have SST data) the Zgrid is altered from 50 to zero in some places and to range from 30-50 in others. At all following days Zgrid is left at 50.
The SST values that are output to the mod file are incorrect when the newly written Zgrid=50 and Zgrid=0, and only make ‘some’ sense when Zgrid is between 30-50 i.e. on day 1.
I tried writing Zgrid as 49.8 rather than 50 but I still get the same error. I also tried leaving Zgrid at 50 and changed depth to 0 for SST, same error. I also tried swapping Zgrid and depth in the obs file, as I have done in 4D-Var applications in the past, but still get the same error.
For SSH I have Zgrid=50 and depth=0 and the data is output correctly. Likewise I have some subsurface obs that are output correctly.