empty lists retrieved from cache as SpatVectors

[CeresBarros]

Empty lists nested in another list are being retrieved as a SpatVector after caching:

Browse[2]>     speciesGAMMs <- Cache(makePSPgamms,
+                           studyAreaANPP = sim$studyAreaANPP,
+                           PSPperiod = P(sim)$PSPperiod,
+                           PSPgis = sim$PSPgis_sppParams,
+                           PSPmeasure = sim$PSPmeasure_sppParams,
+                           PSPplot = sim$PSPplot_sppParams,
+                           useHeight = P(sim)$useHeight,
+                           biomassModel = P(sim)$biomassModel,
+                           speciesCol = P(sim)$sppEquivCol,
+                           sppEquiv = sim$sppEquiv,
+                           NoOfIterations = P(sim)$GAMMiterations,
+                           knots = P(sim)$GAMMknots,
+                           minimumSampleSize = P(sim)$minimumPlotsPerGamm,
+                           quantileAgeSubset = P(sim)$quantileAgeSubset,
+                           minDBH = P(sim)$minDBH,
+                           speciesFittingApproach = P(sim)$speciesFittingApproach,
+                           userTags = c(currentModule(sim), "makePSPgamms"))
Browse[2]> speciesGAMMs$Abie_Bal
$speciesGamm
named list()

$originalData
Empty data.table (0 rows and 7 cols): speciesTemp,MeasureYear,OrigPlotID1,PlotSize,standAge,biomass...

$simData
    speciesTemp MeasureYear OrigPlotID1 PlotSize standAge biomass spDom Weights
 1:        <NA>        1650     random1        1        0       0     1       1
 2:        <NA>         174     random2        1        0       0     1       1
 3:        <NA>        1024     random3        1        0       0     1       1
 4:        <NA>         822     random4        1        0       0     1       1
(...)
    speciesTemp MeasureYear OrigPlotID1 PlotSize standAge biomass spDom Weights

$NonLinearModel
named list()

Browse[2]>     #prepare PSPdata
Browse[2]>     speciesGAMMs <- Cache(makePSPgamms,
+                           studyAreaANPP = sim$studyAreaANPP,
+                           PSPperiod = P(sim)$PSPperiod,
+                           PSPgis = sim$PSPgis_sppParams,
+                           PSPmeasure = sim$PSPmeasure_sppParams,
+                           PSPplot = sim$PSPplot_sppParams,
+                           useHeight = P(sim)$useHeight,
+                           biomassModel = P(sim)$biomassModel,
+                           speciesCol = P(sim)$sppEquivCol,
+                           sppEquiv = sim$sppEquiv,
+                           NoOfIterations = P(sim)$GAMMiterations,
+                           knots = P(sim)$GAMMknots,
+                           minimumSampleSize = P(sim)$minimumPlotsPerGamm,
+                           quantileAgeSubset = P(sim)$quantileAgeSubset,
+                           minDBH = P(sim)$minDBH,
+                           speciesFittingApproach = P(sim)$speciesFittingApproach,
+                           userTags = c(currentModule(sim), "makePSPgamms"))
Oct07 00:29:36 Bmss_spcsP   ...(Object to retrieve (f5e8143ef26263b7.rds))
Oct07 00:29:36 Bmss_spcsP      loaded cached result from previous makePSPgamms call
Browse[2]> speciesGAMMs$Abie_Bal
$speciesGamm
 class       : SpatVector
 geometry    : none
 dimensions  : 0, 0  (geometries, attributes)
 extent      : 0, 0, 0, 0  (xmin, xmax, ymin, ymax)
 coord. ref. :

$originalData
Empty data.table (0 rows and 7 cols): speciesTemp,MeasureYear,OrigPlotID1,PlotSize,standAge,biomass...

$simData
    speciesTemp MeasureYear OrigPlotID1 PlotSize standAge biomass spDom Weights
 1:        <NA>        1650     random1        1        0       0     1       1
 2:        <NA>         174     random2        1        0       0     1       1
 3:        <NA>        1024     random3        1        0       0     1       1
 4:        <NA>         822     random4        1        0       0     1       1
 (...)
    speciesTemp MeasureYear OrigPlotID1 PlotSize standAge biomass spDom Weights

$NonLinearModel
 class       : SpatVector
 geometry    : none
 dimensions  : 0, 0  (geometries, attributes)
 extent      : 0, 0, 0, 0  (xmin, xmax, ymin, ymax)
 coord. ref. :

[CeresBarros]

reprex:

Require::Require("PredictiveEcology/SpaDES.project@504b3bda335b7df93c2a7ec285322f3e067db009 (HEAD)",)

baseDir <- "~/"

if (SpaDES.project::user("cbarros")) {
  baseDir <- "D:/GitHub/"
}

if (SpaDES.project::user("CBARROS")) {
  baseDir <- "~/GitHub/"
}

## add your own user's baseDir here if different from $HOME.
## to verify your user's name run: SpaDES.project::user()

out <- setupProject(
  ## NECESSARY OBJECTS -----------------------------------
  ## these need to be evaluated first and are thus listed before any formalArg
  sppEquivCol = "Boreal",
  vegLeadingProportion = 0,
  successionTimestep = 10L,
  eventCaching = c(".inputObjects", "init"),
  useParallel = FALSE,
  name = "LandRDemo_coreVeg",
  paths = list(
    projectPath = normalizePath(file.path(baseDir, "LandR-tutorials/LandRDemo_coreVeg"), winslash = "/")
    , packagePath = "../packages"  ## relative to projectPath
  ),
  packages = c("PredictiveEcology/reproducible@fileBackendFails (HEAD)",
               "PredictiveEcology/SpaDES.experiment@development (HEAD)"),
  # packages = c("PredictiveEcology/SpaDES.core@development (HEAD)"),
  options = list(
    "reproducible.destinationPath" = paths$inputPath   ## gets paths$inputPath from argument above
    , "reproducible.useCache" = TRUE
    , "reproducible.useNewDigestAlgorithm" = 2
    , "reproducible.useTerra" = TRUE
    , "spades.moduleCodeChecks" = TRUE
    , "spades.inputPath" = paths$inputPath
    , "spades.useRequire" = FALSE
  ),
  modules = c(
    "PredictiveEcology/Biomass_speciesData@terra-migration"
    , "CeresBarros/Biomass_borealDataPrep@allowRTMLfromsim"
    , "PredictiveEcology/Biomass_speciesParameters@development"
    , "PredictiveEcology/Biomass_core@terra-migration"
    , "PredictiveEcology/Biomass_validationKNN@development"
  ),
  # Restart = TRUE, ## restart RStudio from project
  ##
  ## STUDY AREA(S)
  studyAreaLarge = {
    originalcrs <- "+proj=longlat +datum=NAD83 +no_defs +ellps=GRS80 +towgs84=0,0,0"
    Biomass_corecrs <- "+proj=lcc +lat_1=49 +lat_2=77 +lat_0=0 +lon_0=-95 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs +ellps=GRS80 +towgs84=0,0,0"

    largeExtent <- c(xmin = -104.757, xmax = -104.2197, ymin = 55.68663, ymax = 56.20319)
    studyAreaL <- terra::vect(terra::ext(largeExtent))
    studyAreaL <-  terra::vect(terra::geom(studyAreaL), "polygons", crs = originalcrs, atts = data.frame(id = 1:length(studyAreaL)))
    studyAreaL <- terra::project(studyAreaL, Biomass_corecrs)
    studyAreaL
  },
  studyArea = {
    originalcrs <- "+proj=longlat +datum=NAD83 +no_defs +ellps=GRS80 +towgs84=0,0,0"
    Biomass_corecrs <- "+proj=lcc +lat_1=49 +lat_2=77 +lat_0=0 +lon_0=-95 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs +ellps=GRS80 +towgs84=0,0,0"

    smallExtent <- c(xmin = -104.757, xmax = -104.48835, ymin = 55.68663, ymax = 55.94491)
    studyAreaS <- terra::vect(terra::ext(smallExtent))
    studyAreaS <-  terra::vect(terra::geom(studyAreaS), "polygons", crs = originalcrs, atts = data.frame(id = 1:length(studyAreaS)))
    studyAreaS <- terra::project(studyAreaS, Biomass_corecrs)
    studyAreaS
  },
  ##
  ## SPECIES LISTS
  sppEquiv = {
    data("sppEquivalencies_CA", package = "LandR")
    sppEquivalencies_CA[grep("Pin", LandR), `:=`(EN_generic_short = "Pine",
                                                 EN_generic_full = "Pine",
                                                 Leading = "Pine leading")]
    sppEquivalencies_CA[grep("Betu_pap", LandR), `:=`(EN_generic_short = "Birch",
                                                      EN_generic_full = "Birch",
                                                      Leading = "Birch leading")]
    ## all Popu will be merged
    sppEquivalencies_CA[grep("Popu_", LandR), `:=`(EN_generic_short = "Poplar",
                                                   EN_generic_full = "Poplar",
                                                   Leading = "Poplar leading")]
    sppEquivalencies_CA[grep("Popu_", LandR), Boreal := "Popu_Spp"]

    ## define spp column to use for model
    # sppEquivCol <- "Boreal"
    sppEquivalencies_CA <- sppEquivalencies_CA[!"", on = sppEquivCol]
    sppEquivalencies_CA <- na.omit(sppEquivalencies_CA, sppEquivCol)
    sppEquivalencies_CA
  },
  sppColorVect = {
    LandR::sppColors(sppEquiv, sppEquivCol, newVals = "Mixed", palette = "Accent")
  },
  speciesParams = {
    list(
      "shadetolerance" = list(
        Betu_Pap = 1
        , Lari_Lar = 1
        , Pice_Gla = 2
        , Pice_Mar = 3
        , Pinu_Ban = 1.5
        , Popu_Spp = 1
      )
    )
  },
  ##
  ## SIMULATION SETUP ------------------------------------
  times = list(start = 2001, end = 2031),
  params = c("CeresBarros/LandR-tutorials@main/LandRDemo_coreVeg/params.R")
)

out$modules <- out$modules[out$modules != "Biomass_validationKNN"]  ## remove validation module, can only be run after simulation's done
simInitOut <- do.call(SpaDES.core::simInit, out) |>
  reproducible::Cache()
spades(simInitOut)

[eliotmcintire]

Having run this, I see some issues that are not related to Cache, I think:

sim$studyAreaANPP
# NULL

# AND if I switch to sim$studyArea ... 
# Oct08 14:42:33 Bmss_spcsP Filtering PSPs for ANPP to study Area...
# Oct08 14:42:33 Bmss_spcsP There are  0  PSPs in your study area
# Oct08 14:42:33 Bmss_spcsP Filtering PSPs for ANPP by study period...

I don't know how to debug this

[eliotmcintire]

This appears to be a data problem. The study area I have access to your reprex is so small, there are no data points that are in the PSP dataset. So it returns an empty spatial object. I believe that the sim$studyAreaANPP needs to be present and it needs to be some "minimum size".

@ianmseddy is it possible that this should have a .inputObjects entry to create the studyAreaANPP if it is missing?

[CeresBarros]

The study area I have access to your reprex is so small

That shouldn't be the case. This is the same study area as in Barros et al 2023 MEE , which had some PSP data ... something's a miss or the data changed?

[eliotmcintire]

@CeresBarros Can you try to confirm this is still happening, and that it isn't because it is "just a small area"

[CeresBarros]

I was having other issues that prevented me from executing the full workflow. I'll try to update when possible.

In any case, this shouldn't be study area size-related problem. speciesGAMMs$NonLinearModel is initially a list, but becomes a SpatVector when retrived from Cache. Even if the study area is small the object class shouldn't have changed.

[CeresBarros]

@eliotmcintire I finally had time to come back to this, but am hitting separate issues and having trouble running the reprex.
See PredictiveEcology/SpaDES.project#51