Add a specialized function for calculating rolling averages

[nmdefries]

Create a variant of epi_slide that only calculates rolling averages and is much faster doing so. It is backed by data.table::frollmean (RcppRoll::roll_mean is somewhat faster but due to licensing issues, we are not using it). Users can modify frollmean behavior by passing args to it via ....

Because frollmean performs calculations strictly by index, i.e. it does not check that the last n obs correspond to the last n dates, epi_slide_mean cannot be used to aggregate across groups. Any group containing duplicate time values will raise an error. So, e.g., trying to compute an average across all geos in a dataset that contains multiple geos but is ungrouped will fail. This is in line with common use cases.

[nmdefries]

I'll be adding unit tests tomorrow. Results from epi_slide_mean match epi_slide(mean) on a variety of manual tests (time types, missing dates, etc) -- tests will codify those comparisons.

The new function is 4-20 times faster than epi_slide (may be faster on larger datasets than I tested). epi_slide_mean scales much better as the input data gets bigger. It looks like all those time_types in epi_df actually work in epi_slide, so I've implemented them here, too.

[nmdefries]

thought: to make this more general, we could let the user pass a data.table rolling fn (frollmean, frollsum, frollapply). Or if we want to use slider's slide_sum, etc, those too. (Intend to add this in a separate PR.)

Edit 04/01/2024: Added in #433

[dsweber2]

So I was trying to actually test this to see if I had anything useful to say, and the tests won't run; for whatever reason my environment can't find a function called Start; what confuses me is that it seems to run fine in the check?

Edit: substituting Start -> min and End -> max allowed the tests to pass, at least. looking at the context it seems like the intended effect

[nmdefries]

Looks like Start and End were removed in #418. Maybe they weren't being used anywhere else. I replaced them, so this should work again.

[dsweber2]

So this is just going to be my commentary on the tests for now (I'll do a closer readthrough of the actual code eventually). Generally covers most of the bases I can think of. I'm not sure I saw any examples with NA's instead of gaps, which would be worth checking.

minor thing: there's a number of test functions that are all named the same thing (generate_special_date_data and test_time_type_mean) that its hard to tell if they actually are.

[dsweber2]

comparing this with the same calculation on the ungrouped version (epi_slide_mean(small_x %>% ungroup(), "value", before = 50, names_sep = NULL, na.rm = TRUE)), it looks like epi_slide_mean doesn't work for ungrouped tables?

This seems inconsistent with epi_slide though? E.g. epi_slide(small_x, cases_7dav = mean(value), before = 50) auto-groups by geo_value.

What was the motivation for the change?

[brookslogan]

E.g. epi_slide(small_x, cases_7dav = mean(value), before = 50) auto-groups by geo_value.

(Pretending you used before = 6 here.) It actually does something else... it calculates averages across usually-7-day x "number of geos" row windows, then broadcasts (repeats) those slide+aggregation values back to the ref_time_value and every geo appearing at that ref_time_value. (Technically, "number of geos" isn't quite right. If a new geo or geos start reporting midway through, then we might have 3 x {old number of geos} + 4 x {new number of geos} going into the computation.)

Rant: this simple average is probably not what we want almost ever. With count data, we've divided by "number of geos"; we probably want either the national 7dav of counts, or maybe national 7dav * current geo pop / total pop, "distributing" across states proportionally to population. With rate data, we almost surely want to get a 7dav of the weighted mean of the rates by population (equivalent to converting to cases, getting national 7dav, converting to national rate, broadcasting). --- So the plan to split off the aggregating+broadcasting slide + add geo/epigroup aggregation helpers might save us from thinking we're getting by-geo or getting good cross-geo results here.

[dsweber2]

Ok, I misunderstood what was going on here; the epi_slide call should've been epi_slide(small_x %>% ungroup, cases_7dav = mean(value), before = 50), which misled me about epi_slide's base behavior.

Depending on how the rework goes we may want to switch epi_slide to just the grouped behavior and leave the ungrouped versions to whatever we call the aggregating versions (like Logan is getting at).

[dsweber2]

I went through the actual code this time too, generally lgtm.