Testing Containers and WebAssembly in Submissions to the FDA

Reading time:

time

min

February 7, 2024

<a href="https://www.r-consortium.org/">The R Consortium Submission Working Group</a> has now successfully made two pilot submissions to the FDA. All the submissions done by the group are focused on improving practices for R-based clinical trial regulatory submissions. Now, the R submission Working Groups, in collaboration with <a href="https://appsilon.com/">Appsilon</a> and Posit, are exploring new technologies such as Containers and WebAssembly. In this article, we dive into the details of this exploration.
<h3>Table of Contents</h3><ul> <li><a href="#how-everything-started"><strong>How Everything Started</strong></a></li><li><strong><a href="#whats-next">What’s next?</a></strong></li><li><strong><a href="#the-journey-with-webassembly-and-containers">The Journey with WebAssembly and Containers</a></strong></li><li><strong><a href="#the-pilot-4-shiny-app-up-and-running-on-webr">The Pilot 4 Shiny App Up and Running on webR!</a></strong></li><li><strong><a href="#webr-shiny-app">webR Shiny App</a></strong></li><li><strong><a href="#next-steps">Next Steps</a></strong></li></ul>

<hr />

<h2 id="how-everything-started">How Everything Started</h2>
<h3>Pilot 1</h3>
This pilot was initially submitted on November 22, 2021. <strong>This submission was the first publicly available R-based submission to the FDA.</strong> This was a test <a href="https://github.com/RConsortium/submissions-pilot1-to-fda">submission</a> that aimed to explore the submission of an R package to the FDA following the eCTD specifications. The submission included an R package, R scripts for analysis, R-based analysis data reviewed guide (ADRG), and other important components. The final <a href="https://github.com/RConsortium/submissions-wg/blob/main/_Documents/Summary_R_Pilot_Submission2.pdf">response letter from the FDA</a> was received on March 14, 2022.
<h3>Pilot 2</h3>
<strong>This was one of the first submission packages containing a Shiny application</strong>. The main goal of this pilot was to test the submission of an R-based Shiny application bundled into a submission package and transfer it successfully to FDA reviewers. The <a href="https://github.com/RConsortium/submissions-pilot2-to-fda">submitted application</a> was built using the datasets and analyses that were used for the R Submission Pilot 1. The deployed version of this application is available on <a href="https://rconsortium.shinyapps.io/submissions-pilot2/">this site</a>. Alternatively, a Rhino-based version of the application can be found <a href="https://connect.appsilon.com/rhino-fda-pilot/">here</a>.

The final response letter from the FDA was reviewed on September 27, 2023.

<section id="pilot-2" class="level2">In this submission, there were many open-source R packages that were used to create and execute the Shiny application. A very well-known shiny-based interactive exploration framework <strong>{teal}</strong> was used mainly for analyzing the clinical trial data; this package is included in the pharmaverse package repository. The full list of open-source and proprietary R analysis packages is available on this <a href="https://rsubmission-draft.website-us-east-1.linodeobjects.com/">Analysis Data Reviewer’s Guide</a> prepared by the R Consortium R Submissions Working Group for the Pilot 2.
<h2 id="whats-next">What’s next?</h2>
<h3>Pilot 3</h3>
This pilot was successfully submitted to the FDA on Aug 28, 2023. <strong>This was the first publicly available R <a href="https://github.com/RConsortium/submissions-pilot3-adam-to-fda">submission</a> that included R scripts to produce ADaM datasets and TLFs</strong>. Both the ADaMs (SDTM .xpt sources from the CDISC Pilot study) and the TLFs (ADaMs .xpt sourced from the ADaMs generated in R by the Pilot 3 team) were created using R. The next step for this pilot is to await FDA’s review and approval, which may take several months to complete.
<h3>Pilot 4</h3>
This pilot aims to <strong>explore using technologies such as containers and WebAssembly</strong> software to package a Shiny application into a self-contained unit, streamlining the transfer and execution process for enhanced efficiency.

This pilot is expected to be divided into two parallel submissions:

(a) <a href="https://github.com/RConsortium/submissions-pilot4-webR">will investigate WebAssembly</a> and

(b) <a href="http://github.com/Appsilon/experimental-fda-submission-4-podman/">will investigate containers</a>.
<h2 id="the-journey-with-webassembly-and-containers">The Journey with WebAssembly and Containers</h2>
Our team at Appsilon teamed up with the dynamic Pilot 4 crew to explore WebAssembly technology and containers. George Stagg and Winston Chang also joined the working group to discuss the web-assembly portion of Pilot 4. This partnership brought together our engineering prowess to contribute to these tools, injecting fresh perspectives into the ongoing pilot project.

Some of the outcomes of the collaboration:
<ol type="1"> <li>We were able to set up a robust container environment for this pilot project.</li><li>We aided the progress made on the use of both experimental technologies: containers and WebAssembly.</li><li>We developed a working prototype submission using <a href="https://podman.io/">Podman</a> container technology.</li><li>We developed a working early-stage prototype for wrapping a small Shiny application using WebAssembly.</li></ol>
<h4>WebAssembly</h4>
<img class=" wp-image-23208" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65e9e64948d812f5370962ec_3326779d_webr.webp" alt="" width="190" height="189" />
webr

WebAssembly allows languages like R to be executed at near-native speed directly within web browsers, providing users with the ability to run R code without having R installed locally. <a href="https://docs.r-wasm.org/webr/v0.2.0/"><strong>WebR</strong></a> is essentially the R programming language adapted to run in a web browser environment using WebAssembly. This project is under active development.
<h3 id="the-pilot-4-shiny-app-up-and-running-on-webr">The Pilot 4 Shiny App Up and Running on webR!</h3>
<img class="aligncenter size-full wp-image-23210" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65e9e64a022a3f955f86d264_6eeca0d2_rconsortiumxappsilon.webp" alt="" width="822" height="400" />

The deployed example of the Shiny app running on webR <a href="https://brilliant-elf-2da930.netlify.app/">is available here</a>. Check out the video of the application running below.

During this pilot, engineers at Appsilon developed a prototype of a Shiny application running on webR. The application reuses most of the code from the previous pilot apps with some tweaks and a couple of hacks/changes to get around non CRAN dependencies, specially for data loading, WebR compatibilities, and shimming some of the functionality from {teal} and other packages that are (for now) not available on CRAN.
<h4 id="webr-shiny-app">webR Shiny App</h4>
During the <strong>second iteration</strong>, which was recently held, Pedro Silva shared the process of developing this Shiny app running on webR.

</section><section id="the-process" class="level4">
<h4 class="anchored" data-anchor-id="the-process">The Process</h4>
<ol type="1"><li><strong>Leverage the last 2 iterations of the application</strong><ul><li>Reuse as much code as possible</li><li>Avoid touching the logic part</li></ul></li><li><strong>Restrict the number of dependencies to packages on CRAN</strong><ul><li>Replace/shim functionality that was lost from removing dependencies</li></ul></li></ol>Here is the list of dependencies to packages on CRAN; those that worked are colored green, and those that were removed are marked in orange. We ended up with just 3 problematic dependencies (bold).

<section id="the-process" class="level4">Issues with library(cowplot):</section></section>
<ul><li>Some issues with low-level dependencies when deployed</li></ul>
Solution:<ul><li>Replace functionality with HTML</li></ul>
Issues with library(teal):<ul><li>Uses {shiny.widgets} (not working for webR)</li></ul>
Solution:<ul><li>Redo the UI</li><li>Load modules directly</li><li>Recreate filter functionality</li></ul>
Issues with library(teal.data):<ul><li>Use rds exports</li></ul>
Solution:<ul><li>Shim functionality, load data directly</li></ul><ol start="3" type="1"><li><strong>Leverage shinylive and httpuv to export and serve the application</strong><ol type="a"><li>Shinylive can help streamline the export process<ol type="i"><li>Problems<ol type="1"><li>shiny.live won’t let us have non-R files in the application directory - this is an outstanding bug that George asked us to raise an issue for.</li><li>We wouldn’t be able to run the application as a traditional shiny app.</li></ol>
</li><li>Solution: <ol type="1"><li>Custom build script</li></ol></li></ol></li><li>{httpuv} can help serve the application<ol type="i"><li>{httpuv} would run natively on a machine to serve the Shiny app</li></ol></li></ol></li></ol>

‍
<h4>Application Structure</h4>
The figure below shows an overview of what we ended with:

<section id="application-structure" class="level4">Some of the issues and solutions found at the very beginning:
<ol type="1"><li>The previous applications were built using golem and another one in Rhino; the support for these frameworks is not great in webR up to now.<ol type="a"><li>Solution<ol type="i"><li>{box} works out of the box (reuse the rhino version modules)</li><li>Simplify the structure and use a simple shiny modular structure</li></ol></li></ol></li><li>Shinylive does not like non-R files when generating the bundle
<ol type="a"><li>Solution<ol type="i"><li>Keep the app folder as clean as possible for now (www folder only)</li></ol>
</li></ol></li><li>{teal} and {teal.data} are not on CRAN
<ol type="a"><li>Solution<ol type="i"><li>Shim and used functionality</li><li>Use a simple tab system for the UI structure</li></ol>
</li></ol></li></ol>
The FDA was previously told that the shiny application being prepared for the Pilot 4 submission would not be a 1 to 1 mapping from the previous one submitted for the Pilot 2 due to certain constraints such as {teal} not being on CRAN; however, this didn’t represent a problem for them since they would mainly like to test the technology.

Pedro Silva, one of the engineers working on the development of this app, mentioned <strong>“While WebR is still in development, it shows tremendous promise! The loading is definitely still a pain point (over 100mb to set up the environment!) but it will only get better moving forward.”</strong>
<h4>Containers</h4>
<img class="aligncenter size-full wp-image-23216" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65e9e64cb6b94b2c157bb066_3e7f561c_podman-docker-singularity.webp" alt="" width="678" height="246" />

</section>Containerization, particularly through technologies like <a href="https://appsilon.com/docker-vs-podman-vs-singularity/amp/">Docker, Podman or Singularity</a>, offers several advantages for deploying Shiny apps.
<h5>Choosing the Right Container</h5>
Choosing the right container was a question that arose in this project. Although <strong>Docker</strong> is the most popular, we decided to move forward with <strong>Podman</strong>.

In our exploration of containerization tools for deploying Shiny applications, we’ve identified key distinctions between Docker and Podman that influenced our choice.

<strong>Podman</strong> stands out for its daemonless architecture, enhancing security by eliminating the need for a central daemon process. Unlike Docker, Podman supports running containers as non-root users, a critical feature for meeting FDA reviewer requirements. Developed by Red Hat and maintained as an open-source project, Podman prioritizes security with its rootless container support, offering a robust solution for security-conscious users.

<section id="goals" class="level5">
<h5 class="anchored" data-anchor-id="goals">Goals</h5>
A Container-based method to deploy Pilot 2 Shiny App.
<h5>What we did</h5>
<ol type="1"><li>Configurable Podman Dockerfile / docker-compose.yml<ol type="a"><li>R version</li><li>Registry / organization name / image name (differences between docker.io and ghcr.io)</li></ol>
</li>
<li>Documentation on creating the container</li>
<li>CI: Automated build on amd64 and arm64 platforms</li>
</ol>
<h5>Podman short-demo</h5>
<iframe title="YouTube video player" src="https://www.youtube.com/embed/H2WnpvVgmyE?si=KasPkHKd2qlF9xnE" width="560" height="315" frameborder="0" allowfullscreen="allowfullscreen"></iframe>

Below is the dockerfile (recipe) for the container:

<img class="aligncenter size-full wp-image-23218" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65e9e64c6be3ca9ec5f109e8_9e8edb41_file-recipe.webp" alt="" width="754" height="773" />
<h2 id="next-steps">Next Steps</h2>
The next steps are waiting for the review of Pilot 3 by the FDA and to submit the two sections that explore the new technologies to regulatory authorities. Thanks to the collaboration between the R Submission Working Group and other institutions, there is already a working prototype of a {teal}-like Shiny application running on webR and further exploration with Podman is underway.
<ol type="1"><li>Submission to FDA</li><li>Rhino Compatibility</li></ol>
<img class="aligncenter wp-image-23220" src="https://webflow-prod-assets.s3.amazonaws.com/6525256482c9e9a06c7a9d3c%2F65e9e64d2e06a4f93defd2fe_74625118_rhino-1-1.webp" alt="" width="187" height="216" />
<div class="quarto-figure quarto-figure-center">

Appsilon is working on the side with Rhino compatibility; eventually, we might be able to just add this framework into the Pilot 4 application.
<ol start="3" type="1"><li>{teal}This and other packages might be on CRAN soon. We could incorporate them after that, replacing the shims created for this version.</li><li>Boot TimeWe need to improve boot time (remove dependencies and keep working on webR).</li></ol>
 

<strong>This article was co-authored by Pedro Silva, André Veríssimo, Vedha Viyash & Tymoteusz Makowski</strong>

<hr />

<em>This post was originally published on <a href="https://pharmaverse.github.io/blog/posts/2024-02-01_containers_webassembly_submission/containers_and_webassembly_submissions.html" target="_blank" rel="noopener">https://pharmaverse.github.io/</a>.</em>

</div>
</section>

‍