Slow load times for map with ~600k river segments

[johnForne]

Kia ora
I've been testing whether we can use leafgl to report on river water quality data for New Zealand. From my testing, I've discovered that leafgl seems to work really fast and nicely for a subset river segments in New Zealand. However, once I try mapping all river segments using leafgl it becomes too slow to load the map. I would really appreciate your thoughts and insights into whether it is possible to speed up the time it takes to load the maps.

I have created a repository in Github to provide an example of the type of map that I'm trying to publish and for people to see the slow load times I'm getting (takes me approximately 2 minutes:20 seconds to load the shiny app. Note that the data used in this example is just the river lines data and doesn't include the river water quality attributes.

For context, we regularly publish environmental indicators using Shiny.io (see for example River water quality: clarity and turbidity). Currently we rasterize() the vector riverlines in order to get the load times and performance we need to make it useable.

From the README.md it is apparent that leafgl

allows rendering of a large amount of features on a leaflet map.

I suspect that the issue is that while I'm trying to plot ~600k river features (less that the 1 million points which the readme suggests should be ok) - that these are slower because there are effectively many more than a million points joined up to form the lines.

Just wanted to check if anyone had an ideas about how I might be able to publish the river lines as vector?

Thanks in advance,

John

[tim-salabim]

Hi, two things you could try (you will need the latest github version):

1. set src = TRUE in the addGlPolylines call
2. set digits = 5 (or 6 depending on the level of accuracy you want) in case your coordinates have many digits.

Both of these should reduce load times, but as you said, it's very specific to the actual complexity of the features.

If that is not helping enough, I would suggest to use vector tiles. Though I don't think there is any R package to put vector tiles on a leaflet map, there are mapdeck and rdeck that both should be able to handle vector tiles.

Also, is it possible to get "data.RDS" so I can trial some things myself?

[johnForne]

Awesome - thanks heaps Tim.
I'll have a look into mapdeck and rdeck, both look promising. I'll have a look into these...

I tried publishing data.RDS to git - but ran into some error about the files being too big to upload.

I hope that you'll be able to access the data by first downloading, loading and saving as data.RDS (as per hashed code lines #11-17 in app.R and copied below)?

Please let me know if this doesn't work for you and we'll try and find a "plan b".

thanks again,

John

# #  First, download the data (as file geodatabase) from https://data-niwa.opendata.arcgis.com/datasets/NIWA::river-lines/about and upload to project directory folder. Then load data and prepare the data for mapping and save. Once this chunk has been run and data.RDS saved, hash it and then click "run App". And publish app to shiny.io (refer https://statisticsnz.shinyapps.io/Leafgl_performance_test/)

# st_read("./f836a785-194b-4d2b-bfcd-a00849d29a43.gdb/") %>%
#   # head(500000) %>%
#   st_transform(4236) %>%
#   st_cast("LINESTRING") %>%
#   saveRDS("data.RDS")

d_ <- readRDS("data.RDS")

[johnForne]

Kia ora Tim
Apologies for the slow turn-around.... I've finally had a go at your suggestion re adding the src = TRUE argument.

m <- leaflet() %>%
  addProviderTiles(provider = providers$CartoDB.DarkMatter) %>%
  # addGlPolylines(data = d_, group = "glpolylines", color = cols) %>%
  addGlPolylines(data = d_, group = "glpolylines", color = cols, src = TRUE) %>%
  setView(lng = 173.5, lat = -40, zoom = 6)
m

The map (m) draws fine when I run the code above. But when I then pass to a shiny app and then run the app, unfortunately, it seems to show only the base map/provider tiles (refer screen shot below).

# create shiny app
ui <- fluidPage(
  leafglOutput("mymap")
)

server <- function(input, output, session) {
  output$mymap <- renderLeaflet(m)
}

Am I missing something with how to add the src = TRUE argument?

Thanks

[tim-salabim]

As I said you will need the latest github version... This was only fixed recently, see #71