shinycoreci

Github Runner Images

shinycoreci uses the following GitHub Runnner Images.

Image Details Status
Ubuntu 20.04 ubuntu-20.04 status20
macOS 26 macos-26 statusumac26
Windows Server 2022 windows-2022 statuswin22

Installation

Install the development version from GitHub with:

pak::pak("rstudio/shinycoreci")

These GitHub packages will be installed to make sure the latest package development is working as expected:

R-Universe

{shinycoreci} testing leverages rOpenSci r-universe, specifically the posit-dev-shinycoreci universe. This universe is used to install the latest development versions of the Shiny related packages (updated hourly) used in the testing apps without the need for a GitHub token. This last detail is important, as it allows GitHub Actions to install packages freely without the worry of being rate limited. This gives us the ability to attempt to install each app’s dependencies independently, leading to higher test coverage as a single dependencies does not block the entire test execution.

Running manual tests

First, install the {shinycoreci} repo via {pak} (from instructions above). Before running any tests, you may need to add your GITHUB_PAT to your R Environ file (See ?usethis::edit_r_environ and ?usethis::create_github_token)

Commands used to test in different situations:

All testing functions may be run from within the IDE (except for R Terminal / R GUI).

IDE Example

# install.packages("pak", repos = sprintf("https://r-lib.github.io/p/pak/stable/%s/%s/%s", .Platform$pkgType, R.Version()$os, R.Version()$arch))

# Install the latest from pak
pak::pkg_install("rstudio/shinycoreci")

# Install shinyverse
# Run all manual tests
shinycoreci::test_in_ide()

View the latest test results

To view the latest test results, please visit https://rstudio.github.io/shinycoreci/results/. This link will update to the latest results when they are pushed.

If you see failures, this indicates that a test has failed. If it is related to a {shinytest2} snapshot failure, we can view and approve these failures with shinycoreci::fix_snaps(). Your working directory must be in a local checkout of the rstudio/shinycoreci repo. Once shinycoreci::fix_snaps() has finished running, use GitHub Desktop to view the changes. Snapshot tests use a default {shinytest2} screenshot threshold of 5 to tolerate minor anti-aliasing differences; set SHINYCORECI_SCREENSHOT_THRESHOLD in the test environment to tune it, or to an empty string to disable this default. Per-call screenshot thresholds continue to take precedence.

If you receive the error No information found for sha: ABC1234 . Do you have a valid sha?, you may have to provide the git sha value directly: shinycoreci::fix_snaps(sha = "XYZ5678").

In the event that all testing failures can not be addressed by updating {shinytest2} baselines, have a look at the GHA actions build log and keep the following troubleshooting tips in mind:

Troubleshooting test failures

  1. Failures on old versions of R

If a testing app passes on recent version(s) of R, but fails in a suprising way on old R version(s), it may be due to an old R package version. In that case, modify the tests to run only if a sufficient version of the relevant package is available (for example).

  1. Other failures that can’t replicated locally

Other surprising failures are often the result of timing issues (which can be difficult, if not impossible, to replicate locally). If your testing app uses dynamic UI and/or doesn’t have proper input/output bindings, shinytest2 probably needs to know how long to wait for value(s) to update (in this case, use app$wait_for_idle(), for example). Somewhat similarly, when checking DOM values with shinyjster, you may need to wait for an update to DOM element(s) before checking value(s), in which case you can write a recursive function that keeps calling itself until the DOM is ready (for example).

  1. All of the windows shinytest plots have failed

When Windows virtual images update on GitHub Actions, the graphics device may behave exactly as the prior graphics device. Check to see if your windows Image Version has updated. (To view this, inspect the top lines in ./inst/apps/sys-info-win-XX.txt for a change.) You should accept the updated shinytest output for the build with the higher Image Version.

Contribute a testing app

When contributing a testing app, try to do the following:

Note that shinycoreci only supports {testthat} testing framework. Call shinytest2::use_shinytest2(APP_DIR) to use {shinytest2} and {testthat}

  1. shinytest2: primarily useful for taking screenshots of shiny output binding(s) (before or after interacting with shiny input bindings). See here for an example (note that shinytest2::record_test() can be used to generate shinytest2 testing scripts).

  2. shinyjster: primarily useful for asserting certain expectations about the DOM (in JavaScript). See here for an example (note that shinyjster::shinyjster_js() needs to be placed in the UI and shinyjster::shinyjster_server(input, output) needs to be placed in the server).

  3. testthat: primarily useful in combination with shiny::testServer() to test server-side reactive logic of the application.

Pruning old git branches

To help us store and manage the test results, git branches are automatically created for each test run. These branches are automatically removed on GitHub after 1 week of no activity, but you may want to periodically remove them on your local machine as well:

git fetch --prune

What workflows are available?

This repo contains several GitHub Actions workflows:

Trigger

There are a handful of methods that can be called to trigger the GHA actions.

A triggered workflow will run without having to push to the repo. Anyone with repo write access can call this command.

Cross-repo package checks are managed by the External package checks workflow rather than the shinycoreci::trigger*() helpers. This workflow uses the Posit Shiny Automations GitHub App (org secrets POSIT_SHINY_AUTOMATION_APP_ID and POSIT_SHINY_AUTOMATION_PEM in rstudio/shinycoreci) to dispatch workflows, poll run status, create remediation issues, and push remediation branches in target repos. Manual dispatch is restricted to members of the repository owner org before any app or Claude secrets are used; scheduled runs are still allowed. If the optional org/repo secret CLAUDE_CODE_OAUTH_TOKEN is configured (generated locally with claude setup-token and stored in GitHub secrets), the workflow runs anthropics/claude-code-action against the Anthropic API using that token and opens a PR assigned to the target maintainer. The model defaults to claude-sonnet-4-6 and can be overridden with the CLAUDE_MODEL repository variable; CLAUDE_FALLBACK_MODEL can optionally be set for Claude Code’s --fallback-model argument, CLAUDE_MAX_TURNS defaults to 40, and CLAUDE_MAX_ATTEMPTS defaults to 2. If a watched run fails, shinycoreci still fails the scheduler job after opening the issue and optional PR so that dashboards can report the failure while the issue tracks remediation. Target repos only need the dispatched package-check workflow to opt in to workflow_dispatch; they do not need a separate reviewer workflow because shinycoreci handles reviewer and assignee handoff centrally. To add another repo, append a new target entry in .github/external-package-checks.json with owner, repo, workflow, ref, and maintainer, and ensure the GitHub App is installed on that owner/repo with actions, contents, issues, and pull request write permissions. The remediation flow intentionally does not request the workflows permission, so Claude must not modify any files under .github/workflows/ in the target repo.

Schedule

Most of the workflows are run on schedule.

Example schedule where the workflow is run at 2am UTC Monday through Friday:

  schedule:
    - cron:  '0 2 * * 1-5'

Schedule of rstudio/shinycoreci workflows:

build-results.yml

Breakdown of what happens in the Build results website workflow:

On completion of apps-test-matrix.yml…

Final results are available at: https://rstudio.github.io/shinycoreci/results/

FAQ:

pkgs <- c('base64enc', 'bslib', 'Cairo', 'clipr', 'curl', 'dbplyr', 'DiagrammeR',
  'dplyr', 'DT', 'evaluate', 'flexdashboard', 'future', 'ggplot2',
  'ggvis', 'hexbin', 'htmltools', 'htmlwidgets',
  'httpuv', 'jsonlite', 'knitr', 'later', 'leaflet', 'magrittr',
  'maps', 'markdown', 'memoise', 'networkD3', 'plotly', 'png',
  'progress', 'promises', 'pryr', 'radiant', 'ragg', 'RColorBrewer',
  'reactable', 'reactlog', 'reactR', 'rlang', 'rmarkdown', 'rprojroot',
  'rsconnect', 'rversions', 'scales', 'sf', 'shiny',
  'shinyAce', 'shinydashboard', 'shinyjs', 'shinymeta',
  'shinytest2', 'shinythemes', 'shinyvalidate', 'showtext', 'sysfonts',
  'systemfonts', 'testthat', 'thematic', 'tidyr', 'tm', 'websocket',
  'withr', 'wordcloud',
  'sessioninfo',
  'debugme', 'highcharter', 'parsedate', 'quantmod', 'rjson', 'rlist', 'showimage', 'TTR', 'XML', 'xts'
);
pak::pkg_system_requirements(pkgs, execute = TRUE);
install.packages(pkgs);
# Now you should be able to go about testing