shiny.worker: Speed Up R Shiny Apps by Offloading Heavy Calculations
<em><strong>Updated</strong>: January 26, 2023.</em> Because of the way R Shiny is designed, long-running calculations freeze the UI of Shiny dashboards until the calculations are complete. This can result in a sluggish app and a negative user experience. <a href="https://appsilon.com/" target="_blank" rel="noopener noreferrer">Appsilon</a> has created a package to offload long-running calculations to an external machine so that the UI of Shiny dashboards can remain responsive. <i>This article is part of a series on speeding up Shiny. Learn how to omit the server.r bottleneck and push actions to the browser in </i><a href="https://appsilon.com/r-shiny-faster-updateinput-css-javascript/" target="_blank" rel="noopener noreferrer"><i>this article</i></a><i> by Marcin Dubel. Learn how to diagnose poor Shiny performance, use faster functions, and take advantage of caching operations in </i><a href="https://blog.rstudio.com/2020/07/21/4-tips-to-make-your-shiny-dashboard-faster/" target="_blank" rel="noopener noreferrer"><i>this article</i></a><i> by Krystian Igras. </i> Today you'll learn how to use Appsilon's <code>shiny.worker</code> package and what differences it brings to the table. <ul><li style="list-style-type: none;"><ul><li><a href="#frozen-shiny-dashboards">Long-Running Calculations Result in Frozen Shiny Dashboards</a></li><li><a href="#package">shiny.worker: A Package to Speed Up R Shiny</a></li><li><a href="#how">shiny.worker: How Does It Work?</a></li><li><a href="#get-started">How Can I Start Using shiny.worker?</a></li><li><a href="#speed-up-shiny">More Ways to Speed Up R Shiny</a></li></ul> </li> </ul> <hr /> <h2 id="frozen-shiny-dashboards">Long-Running Calculations Result in Frozen Shiny Dashboards</h2> One of the performance challenges in Shiny is that long-running calculations can paralyze your UI. You will not be able to change input values and parts of your application will be frozen until the heavy task is completed. <a href="https://rstudio.github.io/promises/articles/shiny.html#the-flush-cycle" target="_blank" rel="noopener noreferrer">This is an intentional feature of Shiny</a> to avoid race conditions, but is nonetheless very frustrating for the end-user. Unfortunately, the <a href="https://rstudio.github.io/promises/articles/shiny.html" target="_blank" rel="noopener noreferrer">Shiny promises</a> library doesn’t help with this problem. Promises are a great improvement for Shiny performance, but the problems that Shiny promises solve are different from the problem of heavy calculations. Promises were designed for <a href="https://github.com/rstudio/promises/issues/23#issuecomment-386687705" target="_blank" rel="noopener noreferrer">inter-session reactivity rather than intra-session</a>. They simply don't work like promises in JavaScript, and cannot be used to unfreeze Shiny apps during heavy calculations in a single session. <h2 id="package">shiny.worker: A Package to Speed Up R Shiny</h2> Here is how Appsilon has solved the problem in our projects. We have developed a proprietary R package called <strong>shiny.worker</strong>, which is an abstraction based on futures, for delegating jobs to an external worker. While the job is running and the heavy calculation is being processed by the external worker, you can still interact with the app and the UI remains responsive. Here is an example app where you can see shiny.worker in action: <a href="https://demo.appsilon.ai/apps/shiny-worker/" target="_blank" rel="noopener noreferrer">https://demo.appsilon.ai/apps/shiny-worker/</a> <img class="wp-image-5164 size-full" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021fc6bb6e6a6629db30d_Peek-2020-06-22-15-38.gif" alt="Image 1 - Shiny worker demo example" width="1170" height="495" /> Image 1 - Shiny worker demo example The idea of our solution is very simple. If I have long-running calculations that are likely to freeze the UI of my app, then I delegate them to the worker. The 'worker' is an external machine that executes R code. <img class="wp-image-5180 size-full" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b021fefdb8ad62e9cf42dc_shiny.worker.diagram-scaled.webp" alt="Image 2 - How Shiny Worker offloads heavy calculations" width="2560" height="1347" /> Image 2 - How Shiny Worker offloads heavy calculations <h2 id="how">shiny.worker: How Does It Work?</h2> Here is the code of the shiny.worker demo app: <pre><code class="language-r">library(shiny) library(shiny.worker) <br>worker <- initialize_worker() <br>ui <- fluidPage( <br> # Application title titlePanel("shiny.worker demo"), <br> # Sidebar with a slider input for number of bins sidebarLayout( sidebarPanel( div("Play with the slider. Histogram will be still responsive, even if job is running:"), br(), sliderInput("bins", "Number of bins:", min = 1, max = 50, value = 30 ), div("Then try to run new job again:"), br(), actionButton("triggerButton", "Run job (5 sec.)") ), <br> # Show a plot of the generated distribution mainPanel( fluidRow( column(6, plotOutput("distPlot")), column(6, plotOutput("FuturePlot")) ) ) ) ) <br># Define server logic required to draw a histogram server <- function(input, output) { output$distPlot <- renderPlot({ # generate bins based on input$bins from ui.R x <- faithful[, 2] bins <- seq(min(x), max(x), length.out = input$bins + 1) <br> # draw the histogram with the specified number of bins hist(x, breaks = bins, col = "darkgray", border = "white") }) <br> plotValuesPromise <- worker$run_job("plotValuesPromise", function(args) { Sys.sleep(5) cbind(rnorm(1000), rnorm(1000)) }, args_reactive = reactive({ input$triggerButton print("triggered!") "" }) ) <br> output$FuturePlot <- renderPlot({ x <- plotValuesPromise() title <- if (is.null(x$result)) "Job is running..." else "There you go" points <- if (is.null(x$result)) cbind(c(0), c(0)) else x$result plot(points, main = title) }) } <br>shinyApp(ui = ui, server = server)</code></pre> Here's what the app looks like: <img class="size-full wp-image-17655" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b39d12baf17aed52374493_3.gif" alt="Image 3 - Shiny worker demo" width="1584" height="1174" /> Image 3 - Shiny worker demo As you can see, the "Run job" button is available at all times, even if you've just clicked on it. That wouldn't happen with the default Shiny behavior, as the button would gray out for five seconds. Moreover, you can at all times tweak the first chart, which means the UI and app functionality overall still work, and calculations happen in the backend. The key fragment is the line with the <code>shiny.worker::job()</code> call where you schedule a job. The job is your long-running calculation. When it was a regular reactive, it was blocking the UI, but now it is delegated to the worker. As the calculation is delegated to the worker, the UI is unfrozen while the calculation is being performed. Arguments for the job are provided as reactive (<code>trigger_args</code>). Its value will be passed to the job function as args. This means that every time the value of this reactive changes, shiny.worker will take action, depending on the strategy you choose. It can be triggering a new job and canceling a running job or ignoring the change (no new job is scheduled until it is resolved). It is the developer’s responsibility to implement app logic to avoid potential race conditions. To access the worker’s result, you call it as you do with a reactive (<code>plotValuesPromise()</code>). As a result, you are able to read its state (<code>task$resolved</code>) and returned value (<code>task$result</code>). You decide what should be returned when the job is still running with the argument <code>value_until_not_resolved</code>. <h2 id="get-started">How Can I Start Using shiny.worker?</h2> The shiny.worker package is released to the public, and you can install it by running one of these commands: Install the latest stable version: <pre><code class="language-r">install.packages("shiny.worker")</code></pre> Install the development version: <pre><code class="language-r">remotes::install_github("Appsilon/shiny.worker")</code></pre> <h2 id="speed-up-shiny">More Ways to Make Speed Up R Shiny</h2><ul><li style="font-weight: 400;"><a href="https://blog.rstudio.com/2020/07/21/4-tips-to-make-your-shiny-dashboard-faster/" target="_blank" rel="noopener noreferrer">4 Tips to Make Your Shiny Dashboard Faster</a> by Appsilon Data Scientist <a href="https://www.linkedin.com/in/krystian-igras-a3068b152/" target="_blank" rel="noopener noreferrer">Krystian Igras</a></li><li style="font-weight: 400;">Make R Shiny Dashboards Faster with updateInput, CSS, and JavaScript by Appsilon Software Engineer Marcin Dubel</li><li>Use Appsilon's <a href="https://shiny.tools/" target="_blank" rel="noopener noreferrer">Open Source R Shiny Packages</a></li></ul>
.png)
