When developing R Shiny apps, making sure they work well and are reliable is important. Rigorous testing isn't just about finding bugs; it's about preventing them, saving time, and ensuring everything runs smoothly.

We did the comparison of shinytest2 and Cypress. Check out this blog post to learn more.

Testing checks that every part of the app meets requirements and performs well. It also makes the code easier to maintain and reuse. Testing is crucial for security, helping to find and fix vulnerabilities and fortifying defenses against potential breaches.

In essence, testing is an essential part of Shiny app development, bringing together quality, reliability, and security to create great user experiences.

There are mainly 3 approach to tests:

1. Unit tests: These are used to test that functions behave as expected. {testthat} is the most popular R package used for this purpose.
2. Server function tests: These tests run the server function of a Shiny application in an environment that simulates a real client session. They can be used to test reactive components and outputs in the server function of a Shiny application, or a module. The testServer() function that comes with the shiny package is used for this purpose.
3. Snapshot-based tests: These tests are performed with the {shinytest2} package, which runs the Shiny application in a headless web browser. The web browser connects to the Shiny application and simulates user actions, such as clicking on buttons and setting inputs to particular values. It takes snapshots of the application state and, in future runs of the tests, compares the application state to those saved snapshots.

For unit tests and server function tests, no web browser is involved, and the tests and test expectations are expressed in R code. This means that the tests run quickly and that changing code in one part of an application will generally not affect tests of another part of the application.

The snapshot-based tests require a headless web browser, and, as the name suggests, use snapshots.

Overview of {shinytest2}

{shinytest2} uses {testthat} 's snapshot-based testing strategy, a technique where the app's state is captured during tests and compared to previously saved snapshots. 

Primary uses of shinytest2 include:

1. Automated Testing: It allows developers to automate the testing process for their Shiny applications, ensuring that all functionalities work as expected across different scenarios and inputs.
2. Regression Testing: shinytest2 enables regression testing, which involves running tests automatically to ensure that new changes or updates to the application do not introduce bugs or issues that were not present before. This is done using record_test().
3. User Interaction Testing: Developers can simulate user interactions with the Shiny app, such as clicking buttons, entering text, and navigating through different pages, to validate the behaviour of the application under various conditions.
4. Integration with Continuous Integration (CI) Pipelines: Snapshot tests can be integrated into continuous integration pipelines, allowing developers to automatically run them as part of the build and deployment process. This ensures that visual regressions are caught early and prevents them from reaching production.

The benefits of using shinytest2 include:

1. Automated testing helps developers identify and fix bugs early in the development process, leading to higher code quality and more reliable applications.
2. Automating testing also saves time compared to manual testing, especially for complex Shiny applications with numerous features and interactions.
3. shinytest2 helps in detecting regressions quickly by running tests automatically whenever changes are made to the codebase, preventing the introduction of unintended bugs.
4. With the comprehensive test coverage, developers gain confidence in the reliability and stability of their Shiny applications, facilitating smoother deployments and updates.

Example of a simple app, placed in demo/app.R:

library(shiny)

ui <- fluidPage(
  textInput("name", "What is your name?"),
  actionButton("greet", "Greet"),
  textOutput("greeting")
)
server <- function(input, output, session) {
  output$greeting <- renderText({
    req(input$greet)
    paste0("Hello ", isolate(input$name), "!")
  })
}

shinyApp(ui, server)

To test the app, load {shinytest2}, and call record_test() with the path to the app:

library(shinytest2)
record_test("demo/")

The above code launches a special Shiny application which displays the Shiny app to be tested, on the left side and some recorder controls on the right sidebar. Then, you interact with the app and “expect values”. This records your clicks and inputs and these values are saved to the snapshot files. Once a recording is completed, it will create or append a new shinytest2 test to the test_file(). The next time you run the test, these values serve as the single source of truth. The snapshot serves as a comparison to what the application displays later. 

Diving into shiny::testServer

While {shinytest2} tests your entire application, the testServer() function offers a streamlined approach to testing server functions and modules within your shiny applications, without the need to execute the entire application. This targeted testing capability enhances efficiency and reliability compared to testing methods used by {shinytest2}, which simulate complete sessions including a headless web browser.

Moreover, integrating testServer() with unit testing frameworks like {testthat} further enhances its utility.

Here is a server function for a simple Shiny application:

server <- function(input, output, session) {
  get_square <- reactive({
    input$x ^ 2
  })
  output$txt <- renderText({
    paste0("I am ", get_square())
  })
}

Now, we build up a collection of tests that we can run against our server to confirm that it always behaves correctly. For this, we use the {testthat} framework for its expect_ functions.

# Bring in testthat for its expectations
library(testthat)

testServer(server, {
  session$setInputs(x = 1)
  expect_equal(myreactive(), 1)
  expect_equal(output$txt, "I am 1")
  session$setInputs(x = 2)
  expect_equal(myreactive(), 4)
  expect_equal(output$txt, "I am 4")
})

Code explanation:

1. The test expression provided here assumes the existence of the variables - input called x, output called txt, and reactive called myreactive(). The objects from inside the server function environment will be made available in the environment of the test expression.
2. By default, the values to any inputs are NULL. We set values by using the session$setInputs() method. 
3. Lastly, we read the output$txt to check its value. When running inside testServer(), you can simply reference an output and it will give the value produced by the render function.

Note that the ‘session’ object used in testServer() differs from the real session object Shiny uses. This helps to tailor it to be more suitable for testing purposes by modifying or creating new methods such as setInputs().

Interested in learning how to conduct user tests for Shiny applications? Learn more in this presentation.

Testing Shiny Modules

testServer() can also be used for testing the server functions for Shiny modules.

Example of a server code for a module:

myModule <- function(id) {
  moduleServer(id, function(input, output, session) {
    get_square <- reactive({
      input$x ^ 2
    })
    output$txt <- renderText({
      paste0("I am ", get_square())
    })
  })
}

The module can be tested in the following way:

testServer(myModule, {
  session$setInputs(x = 1)
  expect_equal(myreactive(), 2)
})

If the module support additional parameter, let’s say flag, simply add additional param in args as list: 

testServer(myModule2, args = list(flag = TRUE), {
  session$setInputs(x = 1)
  expect_equal(myreactive(), 3)
})

If the module returns some value, returned() can be used:

testServer(myModule2, args = list(flag = TRUE), {
  ...
  expect_equal(session$returned(), 3)
  ...
})

The Differences and Similarities between shiny::testServer() and shinytest2

The major differences between testing with shinytest2 and shiny::testServer() lie in their approaches, scope, and functionality:

1. Scope of Testing:

• shinytest2: shinytest2 focuses on end-to-end testing of Shiny applications, simulating user interactions with the app's UI and validating its behavior across different scenarios. It captures snapshots of the entire application state and compares them to detect regressions.
• shiny::testServer(): shiny::testServer() is used for unit testing of server-side code within Shiny applications. It allows developers to test individual server functions and modules in isolation, without the need to run the full Shiny application.

2. Granularity:

• shinytest2: Provides a high-level testing approach by interacting with the application as a whole, including the UI elements and user inputs. It offers a comprehensive view of the application's behavior but may be slower due to simulating the entire session.
• shiny::testServer(): Focuses on testing specific server-side functionalities at a granular level. It enables developers to target and test individual server functions or modules directly, leading to faster and more focused testing.

3. Testing Environment:

• shinytest2: Typically involves running tests in a headless web browser environment, simulating user interactions with the application through a virtual browser session.
•  shiny::testServer(): Runs tests directly within the R environment, without the need for a web browser. It executes server-side code and verifies the output directly within the R session.

4. Dependencies:

• shinytest2: Relies on external packages like {shinytest} and may require additional setup for integration with continuous integration (CI) pipelines or test automation frameworks.
• shiny::testServer(): Built-in function provided by the Shiny package, requiring minimal dependencies and setup. It seamlessly integrates with existing testing frameworks like {testthat}.

5. Use Cases:

• shinytest2: Suitable for end-to-end testing, regression testing, and validation of the entire Shiny application, including its UI/UX aspects.
• shiny::testServer(): Ideal for unit testing and validating the logic and behavior of server-side code within Shiny applications, facilitating faster development iterations and bug detection.

‍

While {shinytest2}and shiny::testServer() differ in their approaches and scopes of testing, they also share some commonalities in their testing objectives and methodologies:

1. Testing within R Environment:

Both {shinytest2}and shiny::testServer() enable testing within the R environment, allowing developers to write and execute tests directly within their R scripts or RStudio sessions.

2. Support for Test Automation:

Both shinytest2 and shiny::testServer() can be integrated with test automation frameworks like {testthat}, enabling developers to automate the execution of tests and incorporate them into their continuous integration (CI) pipelines.

3. Facilitate Regression Testing:

Both testing methods support regression testing, allowing developers to detect and prevent regressions by running tests against previous versions of the application's codebase and comparing the results.

4. Complementary Testing Approaches:

While they serve different purposes, shinytest2 and shiny::testServer() can be used in conjunction to achieve comprehensive testing coverage for Shiny applications. Developers may utilize shinytest2 for end-to-end testing and UI validation while employing shiny::testServer() for unit testing of server-side code.

We recently launched our resources page. Explore our ebooks, Shiny Gatherings and other materials curated by our experts.

Writing shiny::testServer() Tests in Rhino

Let’s explore how we can write unit tests using shiny::testServer for a Shiny application developed using the Rhino framework. If you are new to rhino and require a step by step guide to create a Shiny application using Rhino, check out this blog.

Example code for the main.R file

box::use(shiny[NS, fluidPage, numericInput, 
               textOutput, moduleServer, renderText,
               reactive])

ui <- function(id) {
  ns <- NS(id)
  fluidPage(
    numericInput(inputId = ns("num"), value = 5, label = "Enter a value:"),
    textOutput(outputId = ns("squared"))
  )
}

server <- function(id) {
  moduleServer(id, function(input, output, session) {
    num_square <- reactive({
      input$num ^ 2
    })
    output$squared <- renderText({
      num_square()
    })
  })
}

Now navigate to tests -> testthat -> test-main.R and paste the below code

box::use(
  shiny[testServer],
  testthat[expect_equal, test_that],
)
box::use(
  app/main[server, ui],
)

test_that("main server works", {
  testServer(server, {
    session$setInputs(num = 2)
    expect_equal(num_square(), 4)
    expect_equal(output$squared, "4")
  })
})

Now run the following command in the console

rhino::test_r()

You should get a output as below if the tests run successfully

‍

If you change the expectations in tests, you get an error message as below:

In this example, we have set the expectation to “3”, whereas the output is “4”. So we get a output mismatch error.

Do you know -
rhino comes with CI actions so you don’t need to setup CI file for testthat. We’ve got you covered.

Tips and Best Practices

Optimizing tests, avoiding common pitfalls, and integrating shiny::testServer() into existing testing frameworks are all crucial aspects of ensuring efficient and effective testing in Shiny applications. Here are some insights and strategies for each:

Optimizing Tests:

1. Identify critical user workflows and prioritize testing on these paths to ensure that essential functionalities are thoroughly tested.

2. Break down tests into smaller, modular components to improve maintainability and reusability. This allows for easier debugging and enhances test scalability.

3. Minimize unnecessary setup and teardown steps in tests to reduce execution time. Only include the necessary setup and assertions required to validate the behavior being tested.

Common Pitfalls to Avoid

1. Avoid creating tests that are overly dependent on specific implementation details, as they can become brittle and prone to failure when the underlying code changes.

2. Ensure comprehensive test coverage by testing all critical functionalities and edge cases. Incomplete test coverage may lead to undetected bugs and regressions.

3. Consider the performance implications of tests, especially in the context of end-to-end testing with shinytest2. Optimize tests to minimize execution time and resource usage, particularly for long-running tests.

4. Regularly review and update tests to ensure they remain relevant and effective as the codebase evolves. Neglecting test maintenance can lead to outdated or inaccurate tests that provide false assurance.

Integrating shiny::testServer() into existing testing frameworks:

1. Ensure that the existing testing framework supports integration with shiny::testServer(). Popular testing frameworks like testthat are commonly used for this purpose.

2. Write test cases using the existing testing framework to cover server-side functionalities of the Shiny application. Each test case should focus on a specific server function or module.

3. Integrate the tests into the Continuous Integration (CI) pipeline to automatically run them on each code commit or build. This ensures that server-side functionalities are thoroughly tested as part of the development workflow.

Conclusion

Testing is a cornerstone of robust and reliable Shiny application development, offering numerous benefits including bug detection, quality assurance, maintainability, and security enhancement. Understanding different testing methodologies is crucial for achieving these goals effectively. Here's a recap of the key points emphasizing the value of this understanding:

1. Testing is essential for identifying and rectifying potential bugs and errors before they reach end-users. It acts as a safeguard against bugs, ensures quality assurance, enhances maintainability, and fortifies security.

2. There are primarily three types of tests for Shiny applications: unit tests, server function tests, and snapshot-based tests. Each type serves a specific purpose and contributes to comprehensive testing coverage.

3. Shinytest2 leverages snapshot testing to automate the testing process for Shiny applications. It facilitates automated testing, regression testing, user interaction testing, and integration with Continuous Integration (CI) pipelines.

4. shiny::testServer() offers a streamlined approach to testing server functions and modules within Shiny applications. It is ideal for unit testing server-side code and integrates seamlessly with existing testing frameworks like testthat.

5. shinytest2 and shiny::testServer() have differences in scope, granularity, testing environment, dependencies, and use cases. However, they also share commonalities in testing objectives and methodologies.

Understanding different testing methodologies empowers developers to build robust, high-quality Shiny applications that meet and exceed user expectations. By embracing testing as an integral part of the development lifecycle, developers pave the way for applications that are reliable, maintainable, and secure.

Did you find this blog post useful? Subscribe to get more helpful blog posts and other resources delivered straight to your inbox every week.

References

1. https://shiny.posit.co/r/articles/improve/testing-overview/
2. https://testthat.r-lib.org/
3. https://rstudio.github.io/shinytest2/
4. https://appsilon.github.io/rhino/