Log of yearly Iowa summaries

[akrherz]

The purpose of this issue is to track the yearly summary values and denote what changes between each iteration. So this is the first entry, 23 June 2017

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2007	38.85	5.72	1.86	1.99
2008	40.59	6.48	5.51	6.12
2009	36.53	3.50	2.37	2.66
2010	42.91	7.26	5.79	6.42
2011	29.57	2.79	3.04	3.42
2012	23.90	1.36	1.39	1.58
2013	33.58	5.57	4.95	5.49
2014	35.91	7.05	12.03	13.13
2015	42.91	5.98	6.93	7.53
2016	37.52	6.21	5.08	5.54

[akrherz]

A bunch of precipitation was reprocessed for the 2007-2014 period. The primary bugs fixed were to include sub 1mm/day precip and not have those all be zero. Also, some prism via stage IV bias correction was done, which helped boost precip totals over poor RADAR coverage locations in the state.

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2007	41.06	7.14	2.17	2.32
2008	39.85	6.36	5.26	5.85
2009	38.12	3.77	2.50	2.80
2010	43.22	7.87	6.10	6.75
2011	30.78	3.14	3.21	3.62
2012	24.71	1.61	1.44	1.62
2013	35.06	6.56	5.32	5.89
2014	36.60	7.03	11.81	12.87
2015	42.91	6.01	6.90	7.50
2016	37.52	6.21	5.07	5.53

[akrherz]

Checked this again now and found no change.

[akrherz]

A current report with 2017 and 2018 now included

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	39.85	6.36	5.26	5.85
2009	38.12	3.77	2.50	2.80
2010	43.22	7.87	6.10	6.75
2011	30.78	3.14	3.21	3.62
2012	24.71	1.61	1.44	1.62
2013	35.06	6.56	5.32	5.89
2014	36.60	7.03	11.81	12.87
2015	42.91	6.01	6.90	7.50
2016	37.52	6.21	5.07	5.53
2017	33.31	4.07	4.40	4.78
2018	48.22	10.83	6.70	7.14

[akrherz]

After reprocessing all the precip and reingesting all results, we have some lower numbers now

yr	precip_in	runoff_in	delivery_ta	detatchment_ta
2008	42.11	7.13	3.41	3.70
2009	38.45	3.82	1.23	1.35
2010	43.88	7.83	3.19	3.48
2011	30.71	2.95	1.72	1.90
2012	25.92	1.34	0.89	1.00
2013	34.80	6.08	3.56	3.87
2014	39.44	7.30	6.82	7.26
2015	44.44	6.40	4.60	4.89
2016	39.70	6.76	4.35	4.67
2017	33.72	3.81	3.18	3.41
2018	46.74	9.67	5.72	6.09

Of concern at the moment is if I have some issue with soils database being incorrectly applied. Will likely be reprocessing that next.

[akrherz]

Did a spot check just now and found the numbers last printed unchanged.

[akrherz]

So, we'd like to know why the largish change in yearly erosion happened earlier this year. So lets focus in on the large change during 2014 and filter for Iowa only HUCs. We find 15 Jun 2014 to be a current interesting case. Some current delivery numbers per HUC12

HUC12	VALID	PRECIP mm	DELIVERY T/a
102300031504	2014-06-15	115.56	22.55

So reprocessing the old env data is trouble as I don't have baseline flowpath metadata, uffties. We need to find a database level backup, I think.... dead end.

We have a backup of the individual output files from circa late 2016. So an initial difference is that the present version has 67 flowpaths and the older version had 75. So for example, flowpath 104 no longer exists. I was able to redo the import of the flowpaths for this HUC12 and found 75 flowpaths in the orginal provided version. I ran ia_grid_order_dbfs_2016_12_01.7z against sol_input_fy2014.2 and got a good match. Only soil 401987 was not found. I was able to jam all these through the preprocessing steps and make a new run in development for this HUC12. The resulting delivery was 24.08 T/a, that's interesting and a positive lead to this investigation.

So now we can explictly compare the two:

So one of the largest negative (present day is much less) is flowpath 52. The slopes are identical between the two... Small difference in soil files

$ diff /i/0/sol/10230003/1504/102300031504_52.sol  /i/1000/sol/10230003/1504/102300031504_52.sol
4c4
< 'Ida' 'SIL' 2 0.230000 0.750000 4868705.000000 0.009000 3.500000 1.510000
---
> 'Ida' 'SIL' 2 0.230000 0.750000 4868705.000000 0.010000 3.500000 1.510000
8c8
< 'Monona' 'SIL' 3 0.230000 0.750000 4758445.000000 0.008000 3.500000 0.975000
---
> 'Monona' 'SIL' 3 0.230000 0.750000 4758445.000000 0.010000 3.500000 0.980000
13c13
< 'Napier' 'SIL' 4 0.230000 0.750000 4758445.000000 0.008000 3.500000 1.130000
---
> 'Napier' 'SIL' 4 0.230000 0.750000 4758445.000000 0.010000 3.500000 1.130000

So the rotation difference here is rather wild:

CURRENT: PPBCBCBCPPPPP
OLD ROT: PPPPPBCBCBCBC

Hmmm, does the database give any clues about that? current FID=1517320 past FID=3118635. No clues there...

So that rotation difference was a variance in how I am presently interrupting the Management code vs what it previously was assumed to be. 83afb4d was one source of the difference. The raw input files have a different 6 year or 8 year rotation string too. This gets very complicated now.

[akrherz]

Yadda Yadda Yadda, and it turns out a significant issue was having a loss of decimal precision with the inbound soil files. The rill factor was truncated at 0.01 instead of having at least three places of precision.

[akrherz]

Snapshot after a large turnover of Iowa data related to DEM reprocessing. Another turnover will happen later this week.

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.16	7.45	3.40	3.67
2009	38.50	4.14	1.51	1.65
2010	43.93	8.17	2.87	3.09
2011	30.76	3.08	1.86	2.05
2012	25.96	1.36	0.79	0.89
2013	34.85	6.19	3.10	3.38
2014	39.50	7.47	5.93	6.32
2015	44.49	6.62	4.58	4.89
2016	39.75	6.75	3.79	4.08
2017	33.78	3.70	2.70	2.91
2018	46.80	9.63	4.40	4.69

[akrherz]

Snapshot after the complete inputs turnover of Jan 2022

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.21	7.66	3.64	2.92
2009	38.52	4.14	1.71	1.64
2010	43.90	7.97	3.09	2.74
2011	30.78	2.76	2.20	2.07
2012	25.97	1.37	0.90	0.82
2013	34.89	6.52	3.77	3.23
2014	39.52	7.09	6.05	5.44
2015	44.52	6.86	4.82	4.31
2016	39.80	6.56	3.94	3.59
2017	33.83	3.73	2.92	2.71
2018	46.84	9.55	4.87	4.18
2019	44.87	9.01	5.08	4.44
2020	30.84	3.41	1.87	1.60
2021	30.52	3.68	2.45	2.05

[akrherz]

Snapshot after the second complete input turnover of Feb 2022

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.15	7.51	2.41	2.42
2009	38.48	4.14	1.36	1.36
2010	43.91	7.92	2.27	2.27
2011	30.77	2.77	1.77	1.79
2012	25.95	1.38	0.69	0.69
2013	34.87	6.43	2.67	2.69
2014	39.49	7.01	4.69	4.71
2015	44.47	6.77	3.75	3.77
2016	39.77	6.44	3.08	3.10
2017	33.79	3.69	2.29	2.30
2018	46.82	9.35	3.54	3.55
2019	44.84	8.89	3.69	3.71
2020	30.81	3.39	1.32	1.33
2021	30.48	3.67	1.77	1.78

[akrherz]

An update before a full re-ingest

year	precip_in	runoff_in	delivery_ta	detachment
2008	42.15	7.72	2.74	2.84
2009	38.48	4.24	1.00	1.04
2010	43.91	8.17	2.17	2.26
2011	30.77	2.82	1.11	1.17
2012	25.95	1.38	0.66	0.71
2013	34.87	6.56	1.92	2.01
2014	39.49	7.20	3.94	4.05
2015	44.47	6.96	3.02	3.12
2016	39.77	6.53	3.16	3.29
2017	33.79	3.96	1.81	1.88
2018	46.84	9.60	3.34	3.46
2019	43.64	8.75	2.20	2.30
2020	30.94	3.66	1.24	1.30
2021	31.22	3.72	1.25	1.29
2022	28.30	2.29	1.20	1.26

[akrherz]

And our results now after the turnover and updated OFE logic code

year	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.15	7.78	2.56	2.85
2009	38.48	4.34	0.84	0.93
2010	43.91	8.26	1.94	2.15
2011	30.77	2.92	0.88	0.99
2012	25.96	1.48	0.57	0.67
2013	34.87	6.59	1.71	1.92
2014	39.49	7.11	3.33	3.64
2015	44.47	6.85	2.51	2.71
2016	39.77	6.84	2.59	2.85
2017	33.79	4.12	1.42	1.55
2018	46.84	10.03	2.83	3.03
2019	43.64	8.69	1.76	1.93
2020	30.95	3.53	1.08	1.21
2021	31.23	3.54	1.04	1.13
2022	28.30	2.21	0.99	1.11

[akrherz]

Reingest after an issue was found with tillage code 2 for soybeans (extra field cultivation operation)

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.15	7.81	1.93	2.12
2009	38.48	4.35	0.68	0.74
2010	43.91	8.28	1.53	1.67
2011	30.77	2.87	0.69	0.77
2012	25.96	1.48	0.43	0.50
2013	34.83	6.59	1.47	1.64
2014	39.49	7.04	2.67	2.89
2015	44.47	6.91	1.94	2.07
2016	39.77	6.84	2.13	2.31
2017	33.79	4.15	1.18	1.28
2018	46.84	10.04	2.36	2.50
2019	43.54	8.65	1.53	1.66
2020	30.60	3.39	0.83	0.93
2021	31.18	3.52	0.81	0.87
2022	28.23	2.20	0.75	0.83

[akrherz]

version: 20230510 (migration to WEPP20230117)

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.15	7.54	1.65	1.79
2009	38.49	4.40	0.56	0.61
2010	43.91	8.18	1.28	1.37
2011	30.77	3.14	0.59	0.65
2012	25.96	1.37	0.35	0.40
2013	34.83	6.23	1.22	1.34
2014	39.49	7.19	2.23	2.38
2015	44.47	6.59	1.69	1.79
2016	39.77	6.81	1.81	1.93
2017	33.79	3.87	1.02	1.09
2018	46.84	9.83	2.03	2.13
2019	43.53	8.66	1.32	1.43
2020	30.57	3.08	0.67	0.74
2021	31.30	3.72	0.73	0.78
2022	28.28	2.07	0.60	0.65

[akrherz]

DEP Version 20230518 after some tillage adjustments

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.15	7.56	2.16	2.35
2009	38.49	4.42	0.66	0.72
2010	43.91	8.26	1.60	1.72
2011	30.77	3.22	0.67	0.74
2012	25.96	1.41	0.48	0.54
2013	34.83	6.25	1.51	1.66
2014	39.49	7.28	2.71	2.92
2015	44.47	6.60	1.88	1.99
2016	39.77	6.96	2.26	2.44
2017	33.79	3.89	1.15	1.24
2018	46.84	9.89	2.46	2.60
2019	43.49	8.73	1.50	1.63
2020	30.57	3.13	0.95	1.05
2021	31.30	3.79	0.79	0.84
2022	28.28	2.12	0.84	0.93

[akrherz]

DEP version 230720

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.13	7.85	0.97	1.04
2009	38.48	4.67	0.45	0.48
2010	43.90	8.35	0.90	0.96
2011	30.76	3.45	0.42	0.45
2012	25.95	1.42	0.25	0.28
2013	34.83	6.51	1.04	1.13
2014	39.48	7.28	1.36	1.44
2015	44.45	6.80	1.28	1.35
2016	39.76	6.86	1.20	1.27
2017	33.78	4.02	0.79	0.84
2018	46.81	10.04	1.35	1.40
2019	43.18	8.98	1.08	1.17
2020	30.33	2.85	0.41	0.44
2021	31.46	4.05	0.60	0.64
2022	28.40	2.12	0.36	0.39

[akrherz]

DEP version 230927

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.13	7.90	2.40	2.63
2009	38.48	4.47	0.72	0.78
2010	43.90	8.80	1.89	2.04
2011	30.76	3.36	0.74	0.81
2012	25.95	1.43	0.51	0.59
2013	34.83	6.52	1.75	1.93
2014	39.48	7.23	2.79	3.03
2015	44.45	6.60	1.91	2.04
2016	39.76	6.96	2.26	2.45
2017	33.78	3.90	1.16	1.25
2018	46.99	9.95	2.46	2.62
2019	43.00	8.74	1.49	1.62
2020	30.16	2.87	0.94	1.05
2021	31.37	3.81	0.79	0.85
2022	28.23	2.05	0.82	0.92

[akrherz]

DEP version 20231123

yr	precip_in	runoff_in	delivery_ta	detachment_ta
2008	42.13	7.82	1.71	1.86
2009	38.48	4.46	0.60	0.65
2010	43.90	8.75	1.45	1.57
2011	30.76	3.33	0.59	0.64
2012	25.95	1.42	0.38	0.44
2013	34.83	6.52	1.46	1.60
2014	39.48	7.17	2.08	2.24
2015	44.45	6.60	1.60	1.71
2016	39.76	6.93	1.76	1.89
2017	33.78	3.88	0.96	1.03
2018	47.12	10.04	1.96	2.06
2019	42.96	8.74	1.30	1.42
2020	30.00	2.75	0.66	0.73
2021	31.24	3.83	0.72	0.77
2022	28.18	2.01	0.59	0.65

[akrherz]

DEP version 20231208

year	precip_in	runoff_in	delivery_ta	detachment
2008	42.13	8.04	1.88	2.06
2009	38.48	4.61	0.62	0.67
2010	43.90	8.84	1.46	1.57
2011	30.76	3.51	0.60	0.65
2012	25.95	1.49	0.41	0.46
2013	34.83	6.65	1.43	1.57
2014	39.48	7.27	2.13	2.29
2015	44.45	6.75	1.66	1.77
2016	39.76	7.01	1.83	1.97
2017	33.78	4.00	0.99	1.06
2018	47.12	10.26	2.03	2.14
2019	42.96	8.87	1.37	1.50
2020	30.00	2.90	0.68	0.74
2021	31.24	3.96	0.73	0.78
2022	28.18	2.12	0.60	0.66

[akrherz]

DEP version 20241118

year	precip_in	runoff_in	delivery_ta	detachment
2008	42.16	8.71	2.43	2.76
2009	38.46	4.82	0.92	1.04
2010	43.92	9.81	1.97	2.20
2011	30.77	3.96	0.86	0.97
2012	25.96	1.62	0.73	0.87
2013	34.83	7.21	2.60	3.12
2014	39.49	7.88	3.22	3.58
2015	44.47	7.35	2.07	2.28
2016	39.75	7.38	2.03	2.23
2017	33.78	4.55	1.34	1.49
2018	48.23	11.41	2.81	3.07
2019	42.86	9.22	1.66	1.87
2020	30.00	3.15	0.87	1.00
2021	31.24	4.32	0.87	0.96
2022	28.22	2.36	0.82	0.94
2023	28.36	3.68	0.85	0.99