[Feature]: Avoid dependencies not compatible with webr

[wincowgerDEV]

Guidelines

• I agree to follow this project's Contributing Guidelines.

Description

I really want to make Open Specy compatible with webR it will allow us to scale the use of the package on the web by so much! There are 3 dependencies that we currently don't have supported.

Problem

Open Specy support on webr

Proposed Solution

I think the easiest way to get around this is to avoid these dependencies. Another way is to try and figure out how to get them all supported but the fact that they are all written in lowlevel code that I don't know makes me think avoiding is easier.

Alternatives Considered

Get the dependencies included in webr

[wincowgerDEV]

Looks like us and plotly are depending on curl

https://repo.r-wasm.org/

It is kind of weird though because above it shows that plotly is all good but then when you look at it, it says it isn't good. Also none of these dependencies seem like they are actual dependencies of any of our apps dependencies, they are just suggestions, maybe we are already good to go.

[wincowgerDEV]

What else is interesting is if I try to put Open Specy into the code editor it will say that its good except for saying that glmnet isn't supported.

https://shinylive.io/r/editor/

Looks like you can dowload a tar for it though, not sure how to use it.
OpenSpecy_1.0.8.zip

[wincowgerDEV]

Seems like running this in a shiny app allows it to run without error, adding the install was key.

webr::install("OpenSpecy")
library(OpenSpecy)

[wincowgerDEV]

Working with the app a bit now, looks like we may need to restructure how the app grabs data. Might try reconfiguring the app with just a small test dataset for the library to get around this issue in the meantime. posit-dev/r-shinylive#55

[wincowgerDEV]

This is helpful: https://stackoverflow.com/questions/78160039/using-shinylive-to-allow-deployment-of-r-shiny-apps-from-a-static-webserver-yiel?rq=2

[wincowgerDEV]

I think most of the dependencies currently listed as suggests could be removed. I think these are the only ones actually used in the package.

testthat (≥ 3.1.9), qs, ggplot2, aws.s3, loggit

[wincowgerDEV]

Also remove shiny from depends. @zsteinmetz, what do you think about this shift, any potential for it to crash and burn when people want to run run_app or similar opperations?

[zsteinmetz]

Hey @wincowgerDEV, this looks great! I'll need some more time to dig into it though. Hard to say how much would break right now. Would you like to touch this asap or rather have it planned for 2.0?

[wincowgerDEV]

Hey @zsteinmetz, I will probably want to get to this in the next couple of months. I recently got some funds to make Open Specy maintenance cheeper,  the website is now double the cost as it was last year, its getting used like crazy. I think this will let us scale cheaply. 

[zsteinmetz]

Oh wow! Yeah, let's try it.

[wincowgerDEV]

Just pushed an update without CURL.

[zsteinmetz]

I quickly screened through this article: https://hypebright.nl/index.php/en/2023/07/25/building-serverless-shiny-apps-with-webr-a-step-by-step-guide.

In their verdict, they don't seem overly enthusiastic about using webR for a fully fledged Shiny app just yet. Making a Shiny app webR-ready reads like a different story than just making the package compatible with webR and running a web-based RStudio like session. They also say it's quite slow which might be the deal breaker here. If I understand it right, all packages need to be made available on one's local computer before the web app starts, so do the spectral libraries. This will take a while.

What's your take on this?

One alternative would be to move away from Shiny as the GUI and use a JavaScript framework instead. This is beyond my capabilities though.

[zsteinmetz]

For some reason, I can't run the Shiny app locally anymore; neither from the repo nor from run_app(). I just get this rather generic error:

Warning: Error in !: invalid argument type
  64: set_sidebar_skin
  63: dashboardSidebar
   1: shiny::runApp

Is this related to removing curl?

[wincowgerDEV]

Good find on that article. I think the speed thing isn't a deal breaker, people are generally fine with the current speed for Open Specy which is somewhat slow too. There will be some speed boosts for people who with poor internet because after the app loads it is basically in offline mode and I think we can remove a lot of the data sharing functionality so that this is the default for folks. There should also in theory be a speed boost for people with high performance computers because they wont be running open specy on our 8GB ram sever with everyone else any more, they will be running it on their desktop just in the browser. On the flip side there is latency on the first load up that takes a little while and this will be true for everyone across the board but personally I think the 1 min it takes to load is worth all these benefits. What do you think?

Huh, I don't see the error on my end. I think I have seen it before though and I remedied it by updating my packages or reverting them to a previous verison. There were some other weird errors I had to version around for Open Specy online and set these dependencies:

RUN R -e "devtools::install_version('config', version = '0.3.1', dependencies= T)"
RUN R -e "devtools::install_version('yaml', version = '2.3.7', dependencies= T)"
RUN R -e "devtools::install_version('bs4Dash', version = '2.2.1', dependencies= T)"

Your error sounds like it is from bs4dash. What version are you using? I am using 2.2.1, here is my session info:

R version 4.3.3 (2024-02-29 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 11 x64 (build 22631)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/Los_Angeles
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] aws.s3_0.3.21      bs4Dash_2.2.1      OpenSpecy_1.0.8    loggit_2.1.1      
 [5] mongolite_2.7.2    config_0.3.1       digest_0.6.31      DT_0.28           
 [9] data.table_1.14.8  plotly_4.10.2      ggplot2_3.4.2      dplyr_1.1.2       
[13] shinyWidgets_0.7.6 shinyjs_2.1.0      shiny_1.7.4       

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.0    viridisLite_0.4.2   urltools_1.7.3     
 [4] bitops_1.0-7        fastmap_1.1.1       lazyeval_0.2.2     
 [7] promises_1.2.0.1    mime_0.12           lifecycle_1.0.3    
[10] ellipsis_0.3.2      survival_3.5-8      magrittr_2.0.3     
[13] compiler_4.3.3      rlang_1.1.1         sass_0.4.6         
[16] tools_4.3.3         utf8_1.2.3          yaml_2.3.7         
[19] askpass_1.1         htmlwidgets_1.6.2   interp_1.1-4       
[22] curl_5.2.1          aws.signature_0.6.0 xml2_1.3.4         
[25] RColorBrewer_1.1-3  rsconnect_0.8.29    httpcode_0.3.0     
[28] withr_2.5.0         purrr_1.0.1         triebeard_0.4.1    
[31] grid_4.3.3          fansi_1.0.4         latticeExtra_0.6-30
[34] caTools_1.18.2      xtable_1.8-4        colorspace_2.1-0   
[37] scales_1.2.1        iterators_1.0.14    MASS_7.3-60.0.1    
[40] signal_0.7-7        crul_1.4.0          cli_3.6.1          
[43] generics_0.1.3      rstudioapi_0.14     httr_1.4.6         
[46] cachem_1.0.8        splines_4.3.3       base64enc_0.1-3    
[49] vctrs_0.6.5         glmnet_4.1-7        Matrix_1.6-5       
[52] jsonlite_1.8.5      crosstalk_1.2.0     jpeg_0.1-10        
[55] testthat_3.1.10     foreach_1.5.2       fontawesome_0.5.1  
[58] tidyr_1.3.0         jquerylib_0.1.4     glue_1.6.2         
[61] codetools_0.2-19    stringi_1.7.12      shape_1.4.6        
[64] gtable_0.3.3        later_1.3.1         deldir_1.0-9       
[67] munsell_0.5.0       tibble_3.2.1        pillar_1.9.0       
[70] htmltools_0.5.5     brio_1.1.3          openssl_2.0.6      
[73] R6_2.5.1            lattice_0.22-5      osfr_0.2.9         
[76] png_0.1-8           hyperSpec_0.100.0   mmand_1.6.3        
[79] memoise_2.0.1       httpuv_1.6.15       bslib_0.4.2        
[82] Rcpp_1.0.10         fs_1.6.2            pkgconfig_2.0.3    
> 

[zsteinmetz]

Good find on that article. I think the speed thing isn't a deal breaker, people are generally fine with the current speed for Open Specy which is somewhat slow too. There will be some speed boosts for people who with poor internet because after the app loads it is basically in offline mode and I think we can remove a lot of the data sharing functionality so that this is the default for folks. There should also in theory be a speed boost for people with high performance computers because they wont be running open specy on our 8GB ram sever with everyone else any more, they will be running it on their desktop just in the browser. On the flip side there is latency on the first load up that takes a little while and this will be true for everyone across the board but personally I think the 1 min it takes to load is worth all these benefits. What do you think?

I was a bit shocked that this very simple Shiny app already takes about 12 secs to load from my end. Hard to say how long it'll take for Open Specy and for people with a poorer internet connection. If it's less than a minute and we implement a loading bar or something, it might be okay. Everything beyond that .. huh .. don't know.

This is my session info running the current R version with bs4Dash 2.3.3:

R version 4.4.0 (2024-04-24)
Platform: x86_64-pc-linux-gnu
Running under: Manjaro Linux

Matrix products: default
BLAS:   /usr/lib/libblas.so.3.12.0 
LAPACK: /usr/lib/liblapack.so.3.12.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Berlin
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] bs4Dash_2.3.3      OpenSpecy_1.0.8    loggit_2.1.1       mongolite_2.8.0    config_0.3.2      
 [6] digest_0.6.35      DT_0.33            data.table_1.15.4  plotly_4.10.4      ggplot2_3.5.1     
[11] dplyr_1.1.4        shinyWidgets_0.8.6 shinyjs_2.1.0      shiny_1.8.1.1     

loaded via a namespace (and not attached):
 [1] gtable_0.3.5        shape_1.4.6.1       bslib_0.7.0         htmlwidgets_1.6.4   caTools_1.18.2     
 [6] latticeExtra_0.6-30 lattice_0.22-6      vctrs_0.6.5         tools_4.4.0         hyperSpec_0.100.2  
[11] bitops_1.0-7        generics_0.1.3      curl_5.2.1          tibble_3.2.1        fansi_1.0.6        
[16] pkgconfig_2.0.3     mmand_1.6.3         Matrix_1.7-0        RColorBrewer_1.1-3  lifecycle_1.0.4    
[21] deldir_2.0-4        compiler_4.4.0      brio_1.1.5          munsell_0.5.1       fontawesome_0.5.2  
[26] codetools_0.2-20    httpuv_1.6.15       htmltools_0.5.8.1   sass_0.4.9          yaml_2.3.8         
[31] lazyeval_0.2.2      glmnet_4.1-8        crayon_1.5.2        later_1.3.2         pillar_1.9.0       
[36] jquerylib_0.1.4     tidyr_1.3.1         MASS_7.3-60.2       cachem_1.0.8        iterators_1.0.14   
[41] foreach_1.5.2       mime_0.12           tidyselect_1.2.1    stringi_1.8.4       purrr_1.0.2        
[46] splines_4.4.0       fastmap_1.1.1       grid_4.4.0          colorspace_2.1-0    cli_3.6.2          
[51] magrittr_2.0.3      crul_1.4.2          survival_3.5-8      utf8_1.2.4          osfr_0.2.9         
[56] withr_3.0.0         scales_1.3.0        promises_1.3.0      httr_1.4.7          jpeg_0.1-10        
[61] signal_1.8-0        interp_1.1-6        png_0.1-8           memoise_2.0.1       testthat_3.2.1.1   
[66] viridisLite_0.4.2   rlang_1.1.3         Rcpp_1.0.12         httpcode_0.3.0      xtable_1.8-4       
[71] glue_1.7.0          xml2_1.3.6          rstudioapi_0.16.0   jsonlite_1.8.8      R6_2.5.1           
[76] fs_1.6.4          

[wincowgerDEV]

Yeah, I get where you are coming from. I think in the long run, this type of app deployment will be the default but for now there are some latency issues.

As for bs4dash, I think you may need to depreciate your bs4dash to 2.2.1 not sure why the most up to date one throws an error.

[zsteinmetz]

Yeah, I get where you are coming from. I think in the long run, this type of app deployment will be the default but for now there are some latency issues.

Don't get me wrong. I totally agree that we should switch to client-side computing in the long run. I'm just skeptical if it might be too early. Maybe we should start testing with an OS version isolated from mainstream development, maybe in a separate branch or locally.

As for bs4dash, I think you may need to depreciate your bs4dash to 2.2.1 not sure why the most up to date one throws an error.

I'll try to track down that error. Regular users might find it too confusing to use older versions of specific packages to make the Shiny app run locally.

[wincowgerDEV]

I was able to get the run_app working again. Unfortuantely though even after removing curl we still have curl imported by another package. Looks like osfr is dependant on it. Maybe we can download the libraries without curl?

[wincowgerDEV]

Have these dependencies left

Here are the dependency errors for each of our suggests and imports.


I think they built an exception for this already.

[wincowgerDEV]

We can produce a special version of Open Specy and host it through a Github action. Right now following these instructions and have a simple version of the package with just the make_rel function going.

https://r-wasm.github.io/rwasm/articles/github-actions.html
https://webr.r-wasm.org/latest/

webr::install("OpenSpecyWebr", repos = "https://moore-institute-4-plastic-pollution-res.github.io/OSWebR/")
library(OpenSpecyWebr)
make_rel(1:100)

[wincowgerDEV]

We are basically done with this just need the most recent Open Specy library to be fully working.

[wincowgerDEV]

I now have the package being simultaneously deployed on the same repo with a github action.
can run this here to test: https://webr.r-wasm.org/latest/

library(caTools)
library(data.table)
library(jsonlite)
library(yaml)
library(hyperSpec)
library(mmand)
library(plotly)
library(digest)
library(signal)
library(glmnet)
library(jpeg)
library(shiny)
webr::install("OpenSpecy", repos = "https://wincowger.com/OpenSpecy-package/")
library(OpenSpecy)
make_rel(1:10)

[wincowgerDEV]

One issue was that since this and the vignettes want github pages you end up having to read the vignettes in in a special way resulting in a new url address but I updated it.