Join the R Community at ShinyConf 2023

shiny.worker: Speed Up R Shiny Apps by Offloading Heavy Calculations


Updated: January 26, 2023.

Because of the way R Shiny is designed, long-running calculations freeze the UI of Shiny dashboards until the calculations are complete. This can result in a sluggish app and a negative user experience. Appsilon has created a package to offload long-running calculations to an external machine so that the UI of Shiny dashboards can remain responsive.

This article is part of a series on speeding up Shiny. Learn how to omit the server.r bottleneck and push actions to the browser in this article by Marcin Dubel. Learn how to diagnose poor Shiny performance, use faster functions, and take advantage of caching operations in this article by Krystian Igras. 

Today you’ll learn how to use Appsilon’s shiny.worker package and what differences it brings to the table.


Long-Running Calculations Result in Frozen Shiny Dashboards

One of the performance challenges in Shiny is that long-running calculations can paralyze your UI. You will not be able to change input values and parts of your application will be frozen until the heavy task is completed. This is an intentional feature of Shiny to avoid race conditions, but is nonetheless very frustrating for the end-user. 

Unfortunately, the Shiny promises library doesn’t help with this problem. Promises are a great improvement for Shiny performance, but the problems that Shiny promises solve are different from the problem of heavy calculations. Promises were designed for inter-session reactivity rather than intra-session. They simply don’t work like promises in JavaScript, and cannot be used to unfreeze Shiny apps during heavy calculations in a single session.

shiny.worker: A Package to Speed Up R Shiny

Here is how Appsilon has solved the problem in our projects. We have developed a proprietary R package called shiny.worker, which is an abstraction based on futures, for delegating jobs to an external worker. While the job is running and the heavy calculation is being processed by the external worker, you can still interact with the app and the UI remains responsive.

Here is an example app where you can see shiny.worker in action: https://demo.appsilon.ai/apps/shiny-worker/ 

Image 1 – Shiny worker demo example

The idea of our solution is very simple. If I have long-running calculations that are likely to freeze the UI of my app, then I delegate them to the worker. The ‘worker’ is an external machine that executes R code. 

Image 2 – How Shiny Worker offloads heavy calculations

shiny.worker: How Does It Work?

Here is the code of the shiny.worker demo app:

library(shiny)
library(shiny.worker)

worker <- initialize_worker()

ui <- fluidPage(

  # Application title
  titlePanel("shiny.worker demo"),

  # Sidebar with a slider input for number of bins
  sidebarLayout(
    sidebarPanel(
      div("Play with the slider. Histogram will be still responsive, even if job is running:"),
      br(),
      sliderInput("bins",
        "Number of bins:",
        min = 1,
        max = 50,
        value = 30
      ),
      div("Then try to run new job again:"),
      br(),
      actionButton("triggerButton", "Run job (5 sec.)")
    ),

    # Show a plot of the generated distribution
    mainPanel(
      fluidRow(
        column(6, plotOutput("distPlot")),
        column(6, plotOutput("FuturePlot"))
      )
    )
  )
)

# Define server logic required to draw a histogram
server <- function(input, output) {
  output$distPlot <- renderPlot({
    # generate bins based on input$bins from ui.R
    x <- faithful[, 2]
    bins <- seq(min(x), max(x), length.out = input$bins + 1)

    # draw the histogram with the specified number of bins
    hist(x, breaks = bins, col = "darkgray", border = "white")
  })

  plotValuesPromise <- worker$run_job("plotValuesPromise", function(args) {
    Sys.sleep(5)
    cbind(rnorm(1000), rnorm(1000))
  },
  args_reactive = reactive({
    input$triggerButton
    print("triggered!")
    ""
  })
  )

  output$FuturePlot <- renderPlot({
    x <- plotValuesPromise()
    title <- if (is.null(x$result)) "Job is running..." else "There you go"
    points <- if (is.null(x$result)) cbind(c(0), c(0)) else x$result
    plot(points, main = title)
  })
}

shinyApp(ui = ui, server = server)

Here’s what the app looks like:

Image 3 – Shiny worker demo

As you can see, the “Run job” button is available at all times, even if you’ve just clicked on it. That wouldn’t happen with the default Shiny behavior, as the button would gray out for five seconds. Moreover, you can at all times tweak the first chart, which means the UI and app functionality overall still work, and calculations happen in the backend.

The key fragment is the line with the shiny.worker::job() call where you schedule a job. The job is your long-running calculation. When it was a regular reactive, it was blocking the UI, but now it is delegated to the worker. As the calculation is delegated to the worker, the UI is unfrozen while the calculation is being performed.

Arguments for the job are provided as reactive (trigger_args). Its value will be passed to the job function as args. This means that every time the value of this reactive changes, shiny.worker will take action, depending on the strategy you choose. It can be triggering a new job and canceling a running job or ignoring the change (no new job is scheduled until it is resolved). It is the developer’s responsibility to implement app logic to avoid potential race conditions.

To access the worker’s result, you call it as you do with a reactive (plotValuesPromise()). As a result, you are able to read its state (task$resolved) and returned value (task$result). You decide what should be returned when the job is still running with the argument value_until_not_resolved.

How Can I Start Using shiny.worker?

The shiny.worker package is released to the public, and you can install it by running one of these commands:

Install the latest stable version:

install.packages("shiny.worker")

Install the development version:

remotes::install_github("Appsilon/shiny.worker")

More Ways to Make Speed Up R Shiny