Hi all,
The biggest headache I've been running into is the normalization step necessary for simulations using 4DVAR, specifically that it doesn't write any output until the normalization for all variables over the entire domain is finished. It would be wonderful if the normalization could output its progress once in awhile so that in case it does not finish (crashes, etc.) it can be restarted from some point without losing everything. With so many more people using ROMS with data assimilation, especially post-docs, graduate students, and others who may not have access to computational resources capable of performing a full normalization during a single runtime, I think this is a worthy endeavor.
I understand I can run the normalization for one variable at a time, but our domain's resolution is so high (1000x1100x15) that I can't get the normalization to finish using my system's maximum allowed processors (256), memory (2TB), and walltime (10 days).
I've tried thirding the resolution (I can't go any lower), which resulted in the normalization for a single 2D variable (zeta) taking ~6 days using the randomization method (N=5000). Maybe its possible to scale this normalization to the higher resolution? Or maybe it's feasible to use a lower N and do some smoothing afterwards? I'll begin testing these soon, but some input from others with more experience would be greatly appreciated.
I've started learning FORTRAN and I've tried editing the code myself (so far I've only tried changing the nesting of the for-loops, but I haven't gotten it working yet), but it seems like something that someone more experienced with FORTRAN would be able to do much more quickly & efficiently.
Cheers, and thanks for listening,
Arin N.