Inflation not turned off while using obs_impact tool #777
Comments
braczka
added
Documentation
|
Shout out to @XueliHuo for identifying this issue. |
|
Thanks for submitting this report which clearly outlines the concern.
In my opinion, this is not a bug nor is it an undocumented feature. The
obs_impact tool is designed to limit the impact of observations in updating
the prior distribution of a state variable and its inflation. This is done
consistently. The function cov_and_impact_factor in assim_tools_mod
computes the product of the localization and the obs_impact and it is named
‘final_factor’ when returned. This product is used consistently in both
subroutines obs_update_ens and update_varying_state_space_inflation to
limit the impact of the observation.
This mechanism is not supposed to limit the application of inflation to
state variables during an assimilation step and it does not. This
functionality would not be well-defined since there could be observations
of multiple kinds of observations being assimilated and each could have a
different setting for no_obs_impact.
If I understand correctly, the problem here is that this application has
two distinct assimilation processes going on. One occurs at high frequency,
maybe daily, while the other is at low frequency, maybe monthly. The desire
is to not change the low frequency state variables with the high frequency
observations. I think that the solution already in place for this, making
these variables no_copy_back when they are not to be updated in any way, is
the correct approach.
…On Fri, Nov 22, 2024 at 3:21 PM Brett Raczka ***@***.***> wrote:
Shout out to @XueliHuo <https://github.com/XueliHuo> for identifying this
issue.
—
Reply to this email directly, view it on GitHub
<#777 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANDHUIVNMMWHB2BZVSNH2DT2B6U5RAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJUHE3DQNBVHA>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
Yes -- I think you're understanding is correct, although I don't see where in the documentation a description of how the obs_impact tool is intended to interact with inflation. For example, inflation is not mentioned at all within the obs_impact tool section. I may be revealing my ignorance of how the code is supposed to work, but I don't see how the existing code limits the impact of inflation on state variables that are completely localized from the observations -- it appears the inflation is just tapered down according to the inflation 'damping' factor. Is this the intended functionality? When a state variable experiences no increment from an observation because of either spatial or variable localization, it seems like inflation should be prescribed a value = 1, and should not have memory of the prior inflation state. Maybe applying a different inflation flavor (different than 5) for this application is preferred. |
|
The key distinction here is between the application of inflation (inflating
state variable priors) and the updating of the adaptive inflation (the
impact of the observations on the inflation prior estimate). The second is
controlled by obs_impact, the first has basically nothing to do with
obs_impact. Perhaps the solution is to update the obs_impact documentation
to be explicit about it only affecting the impact of an observation on the
prior state ensemble and the prior inflation distribution.
…On Mon, Nov 25, 2024 at 9:36 AM Brett Raczka ***@***.***> wrote:
Yes -- I think you're understanding is correct, although I don't see where
in the documentation a description of how the obs_impact tool is intended
to interact with inflation. For example, inflation is not mentioned at all
within the obs_impact tool section. I may be revealing my ignorance of how
the code is supposed to work, but I don't see how the existing code limits
the impact of inflation on state variables that are completely localized
from the observations -- it appears the inflation is just tapered down
according to the inflation 'damping' factor. Is this the intended
functionality? When a state variable experiences no increment from an
observation because of either spatial or variable localization, it seems
like inflation should be prescribed a value = 1, and should not have memory
of the prior inflation state. Maybe applying a different inflation flavor
(different than 5) for this application is preferred.
—
Reply to this email directly, view it on GitHub
<#777 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANDHUIUUV3XCGCE4AE335X32CNGXTAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYGUYDGNZQG4>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Brett, if the state variable experiences no increment, the inflation is not updated either. When the state is not updated, the prior remains unchanged. The inflation algorithm is Bayesian and recursive in time, so if there is no update then we are left with the prior inflation distribution. Right now, the only way to reduce the impact of inflation in case of no update is through damping. Setting the inflation to 1 breaks the recursive nature of the scheme. I think you'll need to adapt the damping to your observation network frequency. |
|
OK -- so my interpretation is the obs_impact tool is completely unrelated to the application of inflation, it simply turns off the adaptive nature of the inflation update (Bayesian, recursive nature) but still applies the inflation. The use of damping in this case to address this issue is not ideal -- because it is my understanding the damping factor is applied universally across all DART state variables, so, in this case, that would not be appropriate, because of the different frequency of the observations. In the case where we want to turn off the recursive nature of the inflation update, toggling to 'NO_COPY_BACK' to immediately set the inflation = 1 seems preferred. |
|
Another question -- in cases of spatial localization, where the state variable increments are downscaled based on the Gaspari-Cohn radius, how does inflation interact with the application of spatial localization? My impression is that the inflation would also be downscaled? |
|
Brett, yes we apply prior inflation before we update its value (mean and sd)! |
|
The same reducing factor, which is a product of the Gaspari Cohn and the
standard localization, is applied to the increments for both the prior
state_variable ensemble and the prior inflation parameters for the state
variable. See my earlier comment about the routine where this final_factor
is computed.
…On Mon, Nov 25, 2024 at 10:46 AM Brett Raczka ***@***.***> wrote:
Another question -- in cases of spatial localization, where the state
variable increments are downscaled based on the Gaspari-Cohn radius, how
does inflation interact with the application of spatial localization? My
impression is that the inflation would also be downscaled?
—
Reply to this email directly, view it on GitHub
<#777 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANDHUITDQF4ZLUB7PU7CMW32CNO6HAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYGY3DCOBXGM>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
OK -- my final question. Is the reducing factor (localization of inflation) applied the same way to spatial localization and variable localization? In other words, is any localization (reducing factor) applied to inflation when using variable localization -- which is controlled through the obs_impact tool ? . |
|
Localization ( |
|
Again, see my original reply above which I think addresses your question.
The reducing factor that incorporates all modifications to the regression
coefficient is computed once and passed to the routine that updates the
prior state ensemble and also to the routine that updates the prior
inflation parameters. You can follow this with a peek inside
assim_tools_mod with a search for final_factor.
…On Mon, Nov 25, 2024 at 11:07 AM Brett Raczka ***@***.***> wrote:
OK -- my final question. Is the reducing factor (localization of
inflation) applied the same way to *spatial* localization and *variable*
localization? In other words, is any localization (reducing factor) applied
to inflation when using *variable* localization -- which is controlled
through the obs_impact tool ? .
—
Reply to this email directly, view it on GitHub
<#777 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANDHUISPPVOQRS5JYL4IZX32CNRNHAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYG4YDONJWGI>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
|
Thanks @jlaucar and @mgharamti. Even though variable localization is accounted for when updating inflation, it seems to me there is nothing in the code that enforces the condition that prior inflation = 1, when we know the increment = 0. This condition can only be enforced through assigning variables as 'NO_COPY_BACK'. |
|
The original theoretical motivation for the adaptive inflation is
predicated on the idea of an observing system that is homogeneous in time,
but not necessarily in space. One can argue that this can be extended to
cases where the observing system is slowly-varying in time. The inflation
damping, which has never been adequately studied and documented to the best
of my knowledge, was introduced to deal with the case where the observing
system is inhomogeneous in time, in particular when observations can
quickly disappear in time. Everything else being equal, the damping would
be the implemented solution to the original problem here I think. However,
if I understand correctly, the damping settings are not quick enough to
avoid blowing up and some inflation may be needed when the infrequent obs
are next available.
Automating a general solution to this could be challenging. For instance,
how does one know that no observations are impacting a given type of state
variable. Requiring a completely filled out obs_impact table that required
all possible obs and their impact on all possible state variables is the
only robust solution that I can think of, but I'm not sure we want to
require that since the vast majority of applications don't use a table at
all. It may be that your current solution is pretty much the best available.
…On Mon, Nov 25, 2024 at 1:26 PM Brett Raczka ***@***.***> wrote:
Thanks @jlaucar <https://github.com/jlaucar> and @mgharamti
<https://github.com/mgharamti>. Even though variable localization is
accounted for when updating inflation, it seems to me there is nothing in
the code that enforces the condition that prior inflation = 1, when we know
the increment = 0. This condition can only be enforced through assigning
variables as 'NO_COPY_BACK'.
—
Reply to this email directly, view it on GitHub
<#777 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ANDHUIV6PPPNMXZGAO3F3Z32COBVZAVCNFSM6AAAAABSKIVDCKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOJYHE3DGOBSGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Describe the bug
CLM-DART is used to assimilate observations of leaf area (monthly frequency), SIF (monthly frequency) and SWE (daily frequency). We use the obs impact tool to isolate the impact of the leaf area (LAI) observations onto leaf carbon only, the impact of SIF observation onto leaf nitrogen only, and the impact of SWE observations on snow only. During the daily assimilation time step where there is only SWE observations the ensemble spread of the leaf carbon and nitrogen gets exceedingly large because inflation is still applied to these variables. The inflation damping value is set to 0.9 so the inflation >1 is applied for all days of the month. There are no observations to counteract this increased spread for leaf carbon and leaf nitrogen. This spurious spread causes poor forecast performance.
This bug was already mentioned in a separate and concurrent UPDATE and NO_COPY_BACK inflation issue #775
Because we use the obs impact tool such that leaf carbon and leaf nitrogen are not influenced by SWE observations (and daily obs only include SWE observations) the expectation was that no inflation would be applied during these times to leaf carbon and leaf nitrogen (inflation =1). In other words, when variable localization is applied such that observation A does not influence state variable B, if observation A is the only observation available during a particular timestep, inflation should not be applied (inflation =1) for that state variable B.
What actually happens is that inflation is applied for all timesteps for all variables in the DART state regardless of the obs impact tool specifications. This causes huge ensemble spread for leaf nitrogen, and also for leaf area (shown below), because no observation constraint is available to counteract the effect. This produces spurious behavior including: 1) high leaf nitrogen generates high photosynthesis, and 2) large spread in leaf area also promotes the dieoff (LAI =0) of certain ensemble members which destroys the background error covariance.
Screenshots
In below figures the 'LAI SIF DA' simulation gives the expected ensemble spread, whereas 'LAI SIF SWE DA' simulations give the spuriously large ensemble spread. Inflation damping value set to 0.9.
Which model(s) are you working with?
CLM5
Version of DART
v11.8.5-14
Have you modified the DART code?
Quick fixes: Manually adjust the input.nml such that for assimilation time steps which only have SWE observations, the leaf carbon and leaf nitrogen state variables are set as 'NO_COPY_BACK' (or these variables are removed from the DART state) so that inflation is not applied. This requires custom/dynamic changes to the CLM shell scripting which must be aware of the time/date.
Bad fix: Perhaps we could adjust the inflation settings such that the damping value is much lower (0.1) thus dramatically decreasing the inflation, however, we would prefer the damping values for the leaf carbon and leaf nitrogen to be different than for SWE. Currently there is no way to apply state variable specific inflation settings so this solution is not ideal.
Permanent fixes: The DART code should recognize the obs_impact tool by automatically setting inflation =1 if there is no relevant observation at that time step on a per variable basis. I think this is the same behavior for spatial localization, where inflation is dampened/removed to reflect the localization distance. At the very least, the documentation has to improve on this -- there is no mention of how inflation interacts with the obs_impact_tool. It's very easy to miss this behavior, unless the user is looking at the ensemble member specific behavior.
The text was updated successfully, but these errors were encountered: