Leveraging {targets} for Robust End-to-End Testing in Shiny Applications
With the growing complexity of application development, <b>effective testing strategies</b> like <b>end-to-end testing</b> are crucial for seamless functioning. This post provides guidance for technical developers using R and Shiny on implementing such tests with the <a href="https://github.com/ropensci/targets"><b>targets package</b></a>, offering insights for managers on testing's role in enhancing team efficiency and <b>product reliability</b>. <h2>Understanding End-to-End Testing for App Development</h2> End-to-End (E2E) testing is a method that validates an application's complete workflow. It simulates real user experiences, ensuring the system and its components interact correctly and exchange the right information. For developers, this brings confidence in the product's reliability. For managers, E2E testing provides a time-saving advantage by automating tedious manual testing. It also acts as a safety net, reducing the risk of unexpected errors after updates and catching potential issues before they reach the user. <h2>Introduction to {targets}</h2> The targets package in R is a <b>robust tool designed to automate complex workflows and implement end-to-end tests efficiently</b>. It facilitates the construction of a pipeline as a directed acyclic graph (DAG) to manage dependencies, enhancing the E2E testing process. <img class="size-full wp-image-20554" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac14e8b62b6b7cf104b_Example-of-a-directed-acyclic-graph.webp" alt="" width="440" height="604" /> Example of a directed acyclic graph The package improves testing efficiency by skipping steps that are up-to-date, enabling automatic parallel computations, and ensuring pipeline reproducibility by encapsulating code and data. This approach frees developers to focus on more critical aspects of development. <h2>Practical Implementation of E2E Testing with Targets in Shiny</h2> To provide a practical example of how you can implement end-to-end testing using targets in Shiny applications, let's consider an application with various inputs and a plot output. This scenario is typical of many real-world Shiny applications, making it a practical and useful test case. Our objective is to ensure that this application behaves as expected when different datasets are utilized. Before starting, we will use the following folder structure: <pre><code> ├── _targets.R ├── R/ │ ├── functions.R </code></pre> <h3>Setting Up functions.R</h3> Firstly, we need to establish our testing environment. We begin by defining a server function within the application that we intend to test. This server function takes the input parameters and generates a plot. <pre><code> # R/functions.R library(purrr) library(shiny) library(targets) library(tibble) <br>server <- function(input, output, session) { output$plot <- renderPlot({ plot(input$param1, input$param2) }) } <br></code></pre> Next, we develop a test function using testServer from the Shiny package. <b>This function simulates user inputs</b> and checks <b>whether the plot was created as expected</b>. <pre><code> test_server_fn <- function(dataset) { shiny::testServer(server, { session$setInputs(param1 = dataset$param1, param2 = dataset$param2) plot_created <<- !is.null(isolate(output$plot)) }) return(plot_created) } </code></pre> Next, let's create a function that will return a nested tibble with the demo datasets. This function is not necessary, once it is only facilitating the way the tibble is created in a structure that makes it easier to work with tar_map() in the next steps. <pre><code> # Define datasets as a nested tibble create_demo_datasets <- function(dataset_list) { tibble( dataset_name = names(dataset_list), dataset = dataset_list ) } </code></pre> <h3>Implementing End-to-End Tests on _targets.R</h3> Now, we transition into the stage of actual implementation of our end-to-end tests using the targets package. In the _targets.R file, we start by sourcing our previously created app_functions.R script. After that, we set up our 'targets' using the tar_map() function. Before we delve deeper, let's first familiarize ourselves with the initial section of _targets.R. This part is crucial as it prepares our environment for running the targets package. Among the packages we load here is the {tarchetypes} package. This package is essential for us to access the tar_map() function. We then source our functions, making them accessible for the current session of R. <pre><code> library(targets) library(tarchetypes) <br># Source functions tar_source() </code></pre> Next, we define the targets options. We set the packages needed to execute the code. In this case, we specify `shiny` and `tibble`. We also adjust the `error` parameter to "continue". This setting instructs `targets` not to halt the execution even when it encounters errors. <pre><code> # Define targets tar_option_set(packages = c("shiny", "tibble"), error = "continue") </code></pre> We will start with two datasets as inputs to this application. In the following block of code we initialize our demo datasets, named `a` and `b`. Each dataset is represented as a data frame with two columns `param1` and `param2`. <pre><code> # Create the demo datasets datasets <- create_demo_datasets( dataset_list = list( a = quote(data.frame(param1 = c(1, 2), param2 = c(10, 12))), b = quote(data.frame(param1 = c(3, 4), param2 = c(13, 14))) # ... you can add more datasets; also try to add a string to force an error ) ) </code></pre> The code above should result in a tibble like this: <pre><code> # A tibble: 2 × 2 dataset_name dataset <chr> <named list> 1 a <language> 2 b <language> </code></pre> Note that the `<language>` values are there because of the use of quote(). This function allows us to capture the expressions representing our datasets as they are, without evaluating them. For further information about why we need this approach with static branching in targets, please check <a href="https://github.com/ropensci/tarchetypes/discussions/105">this</a> discussion. The block of code below uses the tar_map() function from the targets package to apply the test_server_fn() to each dataset in our datasets object. <pre><code> # Static branching list( tar_map( values = datasets, names = "dataset_name", tar_target(result, test_server_fn(dataset)) ) ) </code></pre> The function tar_map() effectively creates a target for each dataset, allowing for individual tracking of the success or failure of the tests. <h2>Running and Visualizing targets Results</h2> By running tar_make(), the targets package will execute test_server_fn() on each dataset and record the outcomes. Successful test passes across all datasets signal that your application functions as intended. If a test fails, targets will notify you, prompting further investigation into the problematic dataset. Firstly, we run tar_visnetwork() and we can see a diagram showing that all the datasets (represented by individual targets) are outdated. <img class="size-full wp-image-20562" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac303aa52514f976407_Outdated-targets.webp" alt="" width="1150" height="516" /> Outdated targets By running tar_make() we can see the log of the pipeline: <pre><code> > tar_make() ▶ start target result_a ● built target result_a [1.585 seconds] ▶ start target result_b ● built target result_b [0.036 seconds] ▶ end pipeline [1.68 seconds] </code></pre> If we check tar_visnetwork() again we see everything green, indicating that all the datasets are up to date. <img class="size-full wp-image-20566" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac49ae3b5ffcdb2fcda_Up-to-date-targets.webp" alt="Up-to-date targets" width="1098" height="406" /> Up-to-date targets <h3>Adding and Removing Datasets</h3> We can now proceed with simulating a real situation. Let's add a new dataset to our list. We expect targets to update only the test for the new dataset. <pre><code> datasets <- create_demo_datasets( dataset_list = list( a = quote(data.frame(param1 = c(1, 2), param2 = c(10, 12))), b = quote(data.frame(param1 = c(3, 4), param2 = c(13, 14))), c = quote(data.frame(param1 = c(5, 6), param2 = c(15, 16))) ) ) </code></pre> After this change we simply save targets.R and run tar_visnetwork(). This is the resulting diagram, showing that dataset "c" was not tested yet. <img class="size-full wp-image-20558" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac620852b3d3e13ee7a_Including-new-targets.webp" alt="Including new targets for E2E testing in R targets" width="1018" height="590" /> Including new targets We run tar_make and observe that it skips the datasets that were already tested, as expected. <pre><code> > tar_make() ✔ skip target result_a ✔ skip target result_b ▶ start target result_c ● built target result_c [1.652 seconds] ▶ end pipeline [1.723 seconds] </code></pre> After running tar_visnetwork() we see that everything is up to date again. <img class="size-full wp-image-20560" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac78d837221d297718a_New-target-up-to-date.webp" alt="New target up-to-date in the E2E testing pipeline using R targets" width="1018" height="648" /> New target up-to-date <h3>Forcing an Error in the Pipeline</h3> Here we are going to create a string instead of a data.frame and we expected to cause an error in the pipeline. According to the parameter we set tar_option_set(... , error = "continue") we expect that the pipeline can continue running even after breaking for one target. To test this case we will add one new dataset that should break ("d") and another one that should work ("e"). <pre><code> datasets <- create_demo_datasets( dataset_list = list( a = quote(data.frame(param1 = c(1, 2), param2 = c(10, 12))), b = quote(data.frame(param1 = c(3, 4), param2 = c(13, 14))), c = quote(data.frame(param1 = c(5, 6), param2 = c(15, 16))), d = "Wrong input.", e = quote(data.frame(param1 = c(7:10), param2 = c(17:20))) ) ) </code></pre> Running tar_visnetwork(): <img class="size-full wp-image-20564" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac9a7c6956a1696d2c1_Two-new-targets-added.webp" alt="Two new targets added to R targets testing pipeline" width="1070" height="750" /> Two new targets added After running tar_make() and tar_visnetwork() again we see that everything happened as expected: <pre><code> > tar_make() ▶ start target result_d Error: $ operator is invalid for atomic vectors <br>✖ error target result_d ▶ start target result_e ● built target result_e [1.605 seconds] ✔ skip target result_a ✔ skip target result_b ✔ skip target result_c ▶ end pipeline [1.794 seconds] </code></pre> <img class="size-full wp-image-20556" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65b01ac903aa52514f976695_Expected-error-when-running-targets.webp" alt="Expected error when running R targets" width="1084" height="764" /> Expected error when running R targets <h2>Concluding End-to-End Testing in R targets</h2> This article has demonstrated the utility of the targets package in R for implementing robust end-to-end testing in Shiny applications. Through examples, we've seen how targets enhances efficiency and reliability in software development. Further, it's worth noting that in a production setting, you can establish multiple testServer instances. These can be added in a similar way as demonstrated in the static branching example. This allows you to create individual tests for each server, using either new or the same datasets, providing a more accurate simulation of real-world scenarios. Adopting this methodology can significantly boost confidence in the performance of your Shiny applications in real-world scenarios. With a well-implemented E2E testing strategy, developers and managers alike can ensure their applications function as expected, enhancing the reliability of their software. <h3>Code Example for the targets E2E Testing</h3> Script 1 <pre><code> # R/functions.R library(shiny) library(tibble) <br>server <- function(input, output, session) { output$plot <- renderPlot({ plot(input$param1, input$param2) }) } <br>test_server_fn <- function(dataset) { shiny::testServer(server, { session$setInputs(param1 = dataset$param1, param2 = dataset$param2) plot_created <<- !is.null(isolate(output$plot)) }) return(plot_created) } <br># Define datasets as a nested tibble create_demo_datasets <- function(dataset_list) { tibble( dataset_name = names(dataset_list), dataset = dataset_list ) } </code></pre> Script 2 <pre><code> # _targets.R library(targets) library(tarchetypes) <br># Source functions tar_source() <br># Define targets tar_option_set(packages = c("shiny", "tibble"), error = "continue") <br># Create the demo datasets datasets <- create_demo_datasets( dataset_list = list( a = quote(data.frame(param1 = c(1, 2), param2 = c(10, 12))), b = quote(data.frame(param1 = c(3, 4), param2 = c(13, 14))) # ... you can add more datasets, try to add a string to force an error # see https://github.com/ropensci/tarchetypes/discussions/105 #. to understand the use of quote() ) ) <br># Static branching list( tar_map( values = datasets, names = "dataset_name", tar_target(result, test_server_fn(dataset)) ) ) </code></pre>
