Chapter 1 About Successful Shiny Apps

I Too Like to Live Dangerously

Austin Powers

1.1 A (very) short introduction to Shiny

If you’re reading this page, chances are you already know what a Shiny App is — a web application that communicates with R, built in R, and working with R. The beauty of {shiny} is that it makes it easy for anybody to create a small app in a matter of hours. With small and simple Shiny apps, no knowledge of HTML, CSS or JavaScript is required, which makes it really easy to use—you can rapidly create a POC for a data-product and showcase an algorithm or present your results with an elegant, simple to build user interface.

The first version of {shiny} has been published in 2012, and since then there it has been one of the top project of the RStudio team. At the time of writing these lines (January 2020), more than 4600 commits has been made to the main GitHub repository, by 46 contributors. It’s now downloaded around 400K times a month, according to cranlogs, and has 703 reverse dependencies (package that depends on it) according to devtools::revdep("shiny").

If you’re very new to Shiny, this book might feel a little bit complex, as it relies on the assumption that you already know how to build (at least basic) Shiny applications, and that you want to push your Shiny skills to the next level. If you are very new to Shiny, we suggest you start with the Mastering Shiny book before coming back to this book.

Ready to get started with complex Shiny App?

1.2 What’s a complex Shiny App?

One of the unfortunate things about reality is that it often poses complex problems that demand complex solutions

1.2.1 Reaching the cliff of complexity

Things are quite simple when it comes to small prototypes or proof of concepts. But things change when your application reaches “the cliff of complexity”1.

But what do we mean by complexity? Getting a clear definition is not an easy task2 as it very depends on who is concerned and who you’re talking to. A good definition is the one from The Art of Unix Programming: “Complex software is harder to think about, harder to test, harder to debug, and harder to maintain — and above all, harder to learn and use.” But let’s try to come with a definition that will serve us in the context of engineering Shiny applications. When building software, we can think of complexity from two points of view: the complexity as it is seen by the coder, and the complexity as it is seen by the customer / end user.3

With great complexity comes great responsibility

When your program reaches this state, there is a lot of potential for failure, be it from a developer or user perspective. For the code, bugs are harder to anticipate: it’s hard to think about all the different paths the software can follow and difficult to detect bugs because they are deeply nested in the numerous routines the app is doing. It’s also hard to think about what the state of your app is at a given moment in time because of the numerous inputs and outputs your app contains. From the user perspective, the more complex an app, the more steep the learning curve is, which means that the user will have to invest more time learning how the app works, and will be even more disappointed if ever they realize this time has been a waste.

1.2.1.1 Two views of complexity

  • From the developer point of view

An app is to be considered complex when it is big enough in terms of size and functionality that it makes it impossible to reason about it at once, and you must rely on tools to understand and handle this complexity. This type of complexity is called implementation complexity. One of the goal of this book is to present you a methodology and toolkit that will help you reduce this form of complexity.

For example, we’ll talk about a design / prototype / build / secure / deploy framework, which helps you reduce the complexity of implementing and maintaining new app features.

We’ll also be talking at length about the {golem} package, which provides a toolkit for reducing the cognitive load of handling complexity in large Shiny applications. For example, creating a new Shiny module requires following a very strict structure. One way to do that is to remember how to do that and to code it from scratch (a method which has an important cognitive load and is very error prone). Another way is to copy and paste an existing module and to adapt it, which is, as any copy and paste, likely to lead to errors. Or there is the {golem} way, in which you rely on a robust tool to build the foundations for a new module.

Another benefit of adopting automation for common application components is that it ensures that you’re following a convention. And conventions are crucial when it comes to building and maintaining complex systems: by imposing a formalized structure for a piece of code, it enhances readability, lessens the learning curve, and lightens the chance for typos and/or copy and paste errors.

  • Customers and users

On the other hand, customers and end user see complexity as interface complexity. Interface complexity can be driven by a lot of elements, for example the probability of making an error while using the app, the difficulty to understand the logical progression in the app, the presence of unfamiliar behaviour or terms, visual distractions… This book will also bring you strategy to help you cope with the need for simplification when it comes to designing interface.

1.2.1.2 Balancing complexities

There is an inherent tension between these two source of complexity, as designing an app means finding a good balance between implementation and interface complexity.

Reducing implementation complexity means one has to make some decisions that will lower one while rising the other.

For example, we can think of something very common in Shiny: the “too much reactivity” pattern. In some cases, coders try to make everything reactive: e.g., three sliders and a selectInput, all updating a single plot. This behavior lowers the interface complexity: users don’t have to think a lot about what they are doing, they just move things around and it updates. But this kind of pattern can make the app compute too many things: users rarely go to the slider value they need on their first try. They usually miss what they actually want to select in an input. So, way more computation for R. One solution can be to delay reactivity or to cache things so that R computes fewer things. But that comes with a cost: handling delayed reactivity and caching elements increases implementation complexity. One other solution is simply to add a button after the input, and only update the plot when the user clicks on it. This pattern makes it easier to control reactivity from an implementation side. But it can make the interface a little bit more complex for the user who have to perform another action in addition to changing their inputs.

We’ll argue somewhere else in the book that not enough reactivity is better than too much reactivity, as the latter increases computation time, and relies on the assumption that the user makes the right action on the first try. Another good example is {shiny}’s dateRangeInput() function, which takes a start which is posterior to the end (which is the behavior of the JavaScript plugin used in {shiny} to create this input). Handling this special case is doable: with a little bit of craft, you can watch what the user inputs and throw an error if the start is after the end.4 That solution augments the implementation complexity, while leaving it as is requires the user to think about whether or not the starting date is before the ending date, thus increasing the interface complexity.

So what should we do? It’s up to you: deciding where to draw the line between interface & implementation complexity very much depends of the kind of project you’re working on, and on the kind of users that will use your app. Things that you can assess here is the ease of working on implementation of feature (for your team), how much your app will be used, how frequently, by what kind of users… Drawing the line between the two is not the simplest thing on earth, and requires to restrain yourself from implementing too much, and still create an application that is easy to use.

1.2.2 Assessing complexity

How can we get a good overview of this complexity?

1.2.2.1 Codebase size

Another measure that sometimes comes in the discussion is the codebase size. It’s relatively hard to use this number of lines metric, as R is very permissive when it comes to indentation and line breaks, and (unlike JavaScript or CSS), it is generally not minified5.
It also depends on your coding style and the packages you’re using. For example, {tidyverse} packages encourage the use of the pipe (%>%)6 with one function by line, producing more lines in the end code.

Consider this piece of valid R code:

iris[
  1
  :
    5, 
  c
  (
    "Species"
  )
  ]
[1] setosa setosa setosa setosa setosa
Levels: setosa versicolor virginica

9 lines of code for something that could also be written in one line.

iris[1:5, c("Species")]
[1] setosa setosa setosa setosa setosa
Levels: setosa versicolor virginica

In other words, using this kind of writing style can make the code base larger in term of lines, without really adding complexity to the general program.

Another drawback of this metric is that it focuses on numbers instead of readability, and in the long run, yes, readability matters.7

Still, this metric can be useful to reinforce what you’ve learned from other metrics: it’s rather unlikely that you’ll find this “extreme” coding style we’ve just showed, and even if it might not make sense to compare two code base that just differ by 1 or 2 % of lines of code, it’s very likely that a code base which is ten, twenty, one hundred times larger is a more complex software. Also, another good sign related to this metric is the number of files in the project: R developers tend to split their functions into several files, so the more files you’ll find in a project, the more large the code base is. Numerous files can also be a sign of maintenance complexity: it’s harder to reason about an app logic split into several files than about something that fits into one linear code inside one file.

If you want to use this metric, you can do it straight from R with the {cloc} package, available at https://github.com/hrbrmstr/cloc.

if (!requireNamespace("cloc")){
  remotes::install_github("hrbrmstr/cloc")
}

For example, let’s compare a rather big package ({shiny}) with a small one ({attempt}):

library(cloc)
library(dplyr)
shiny_cloc <- cloc_cran("shiny", .progress = FALSE, repos = "http://cran.irsn.fr/" )
attempt_cloc <- cloc_cran("attempt", .progress = FALSE, repos = "http://cran.irsn.fr/" )
clocs <- bind_rows(
  shiny_cloc, 
  attempt_cloc
)

# Counting lines of code
clocs %>%
  group_by(pkg) %>%
  summarise(
    loc = sum(loc)
  )
# A tibble: 2 x 2
  pkg       loc
  <chr>   <int>
1 attempt  4011
2 shiny   71478
# Counting files
clocs %>%
  group_by(pkg) %>%
  summarise(
    files = sum(file_count)
  )
# A tibble: 2 x 2
  pkg     files
  <chr>   <int>
1 attempt    35
2 shiny     269

Here, just from these two metrics, we can safely assume that {shiny} is a more complex package than {attempt}.

1.2.2.2 Cyclomatic complexity

Cyclomatic complexity is a software engineering measure which allows to define the number of different linear paths a piece of code can take. It’s computed based on a control-flow graph8 representation of an algorithm. The complexity number is then computed by taking the number of nodes, and subtracting the number of edges plus two times the number of connected components of this graph: M = E − N + 2P, where M is the measure, E the number of edges, N the number of nodes and 2P 2 time the number of connected components.

We won’t go deep into this topic, as there are a lot things going on in this computation and you can find many documentation about this online. Please refer to the bibliography for further readings about the theory behind this measurement.

In R, the cyclomatic complexity is computed with the {cyclocomp} package, which can be installed from CRAN with:

install.packages("cyclocomp")

The cyclocomp package comes with three main functions: cyclocomp(), cyclocomp_package(), and cyclocomp_package_dir (). The one we’ll be interested in is cyclocomp_package_dir(): building successful shiny apps with the {golem} framework (we’ll get back on that later) means you’ll be building your app as a package. You can then use cyclocomp_package_dir() to compute the complexity of your app. Here is for example the cyclomatic complexity of the default golem template:

library(cyclocomp)
cyclocomp_package("golex") %>% head()
                          name cyclocomp
1                   app_server         1
2                       app_ui         1
3 golem_add_external_resources         1
4                      run_app         1

And the one from another small application:

cyclocomp_package("tidytuesday201942") %>% head()
                  name cyclocomp
24      mod_dataviz_ui         8
23  mod_dataviz_server         7
35                  rv         6
14             display         4
39           undisplay         4
37 tagRemoveAttributes         3

And, finally, the one for {shiny}:

cyclocomp_package("shiny") %>% head()
                       name cyclocomp
494                  untar2        75
115            diagnoseCode        54
389                  runApp        50
150 find_panel_info_non_api        37
371             renderTable        37
102          dataTablesJSON        34

And, bonus, this cyclocomp_package() function can also be used to retrieve the number of functions inside the package.

Why does it matter? Successful Shiny apps implies two things: you’ll be writing unit tests for your code, and all the inputs from the users will be checked during the application runtime. Both means that the more a function is complex, the more it will be hard to reason about: it’s almost impossible to make a mental model of a very complex function. Checking users inputs is also harder, as a higher complexity might implies that there are several inputs, and / or that an input can have a large number of states along the function execution. It’s also harder to maintain complex function: bugs are harder to spot as they might happen only in one of the multiple paths the code can take. And of course, integrating changes is harder as you have to think about how new elements will impact all the possible paths the code might take.

So, as The Clash said, “what are we gonna do now?” You might have heard this saying that “if you copy and paste a piece of code twice, you should write a function”. Then, this might be a solution here: splitting things in smaller pieces lower the local cyclomatic complexity. One thing that can also be done is leveraging the strength of S3 methods to split elements: if you have a function that behave differently based on the type of the input, generics and methods are easier to reason about than if statements inside a larger function. But that’s not a magic solution: (A) because the global complexity of the app is not lowered by splitting things into pieces, (B) because the deeper the call stack the harder it can be to reason about the big picture.

1.2.2.3 Other measures

Complexity can come from other sources: unsufficient code coverage leading to bugs that are hard to spot, dependencies that breaks the implementation, old package, or a lot of other things.

How do we assess that? We can have a look at the {packageMetrics2} package to have some of these metrics: for example, the number of dependencies, the code coverage, the number of releases and the date of the last one, etc.

library(packageMetrics2)
frame_metric <- function(pkg){
  metrics <- package_metrics(pkg)
  tibble::tibble(
    n = names(metrics), 
    val = metrics, 
    expl = list_package_metrics()[names(metrics)]
  )
}
frame_metric("golem") %>% knitr::kable()
Metrics: ARR ATC DWL DEP DPD CCP FLE FRE LIB LLE LNC LNR LRE NAT NTF NUP OGH SAP SEM SEQ SWD VIG 
n val expl
ARR 0 Number of times = is used for assignment
ATC NA Author Test Coverage
DWL 5572 Number of Downloads
DEP 27 Num of Dependencies
DPD 0 Number of Reverse-Dependencies
CCP 2.33898305084746 Cyclomatic Complexity
FLE 16.8833333333333 Average number of code lines per function
FRE 2019-08-05T14:50:02+00:00 Date of First Release
LIB 0 Number of library and require calls
LLE 26 Number of code lines longer than 80 characters
LNC 0 Number of lines of compiled code
LNR 1464 Number of lines of R code
LRE 2019-08-05T14:50:02+00:00 Date of Last Release
NAT 0 Number of attach and detach calls
NTF 0 Number of times T/F is used instead of TRUE/FALSE
NUP 1 Updates During the Last 6 Months
OGH 1 Whether the package is on GitHub
SAP 1 Number of sapply calls
SEM 0 Number of trailing semicolons in the code
SEQ 0 Number of 1:length(vec) expressions
SWD 18 Number of setwd calls
VIG 4 Number of vignettes
frame_metric("shiny") %>% knitr::kable()
Metrics: ARR ATC DWL DEP DPD CCP FLE FRE LIB LLE LNC LNR LRE NAT NTF NUP OGH SAP SEM SEQ SWD VIG 
n val expl
ARR 14 Number of times = is used for assignment
ATC 29.2461947330273 Author Test Coverage
DWL 11492161 Number of Downloads
DEP 22 Num of Dependencies
DPD 714 Number of Reverse-Dependencies
CCP 3.70817843866171 Cyclomatic Complexity
FLE 24.2420168067227 Average number of code lines per function
FRE 2012-12-01T07:16:17+00:00 Date of First Release
LIB 1 Number of library and require calls
LLE 424 Number of code lines longer than 80 characters
LNC 0 Number of lines of compiled code
LNR 24737 Number of lines of R code
LRE 2019-10-10T11:50:02+00:00 Date of Last Release
NAT 0 Number of attach and detach calls
NTF 0 Number of times T/F is used instead of TRUE/FALSE
NUP 1 Updates During the Last 6 Months
OGH 1 Whether the package is on GitHub
SAP 10 Number of sapply calls
SEM 1 Number of trailing semicolons in the code
SEQ 0 Number of 1:length(vec) expressions
SWD 6 Number of setwd calls
VIG 0 Number of vignettes

If you’re building an app with {golem}, a framework for building shiny apps, you can use the DESCRIPTION file, the one that contains the dependencies, as a starting point for assessing the state of your dependencies:

desc::desc_get_deps("golex/DESCRIPTION")
     type package version
1 Imports   shiny       *
2 Imports   golem       *

Then, it can be used as a series of inputs for our previous function.

1.2.3 Production Grade Software Engineering

Complexity is still frowned upon by a lot of developers, notably because it has been seen as something to avoid according to the Unix philosophy. But there are dozens of reasons why an app can become complex: for example, the question your app is answering is quite complicated and involves a lot of computation and routines. The resulting app is rather ambitious and implements a lot of features, etc. So yes, there is a chance that if you’re reading this page, you’re working or are planning to work on a complex Shiny app. And this is not necessarily a bad thing! Shiny apps can definitely be used to implement production-grade9 software, but production-grade software implies production-grade software engineering. To make your project a success, you need to use tools that reduce the complexity of your app and ensure that your app is resilient to aging.

In other words, production-grade Shiny apps require working with a software engineering mindset, which is not always an easy task in the R world. R comes from the land of the academics and is still used as an “experimentation tool”, where production quality is one of the least concerns. Many developers in the R world have learned R as a tool for making statistics, not as a tool for building software. These contexts are very different and require different mindsets, skills, and tools.

With {shiny}, as we said before, it’s quite easy to prototype a simple app, without any “hardcore” software engineering skills. And when we’re happy with our little proof of concept, we’re tempted to add something new. And another. And another. And without any structured methodology, we’re almost certain to reach the cliff of complexity very soon and end up with a code base that is hardly (if ever) ready to be refactored to be sent to production.

The good news is that building a complex app with R (or with any other language) is not an impossible task. But it requires planning, rigor, and correct engineering.
This is what this book is about: how to organise your Shiny App in a way that is time and code efficient, and how to use correct engineering to make your app a success.

1.3 What’s a successful Shiny App?

The good news is that your application does not have to be complex to be successful. Even more, in a world where “less is more”, the more you can reduce your application complexity, the more you’ll be prepared for success.

So what’s a successful Shiny app? Defining such a metric is not an easy a task, but we can extract some common patterns when it comes to applications that would be considered successful.

1.3.1 It exists

First of all, an app is successful if it was delivered. In other words, the developer team was able to move from specification to implementation to testing to delivering. This is a very engineering-oriented definition of success, but it’s a pragmatic one: an app that never reaches the state of usability is not a successful app, as something along the way has blocked the process of finishing the code.

This implies a lot of things: but mostly it implies that the team was able to organise itself in an efficient way, so that they were able to work together in making the project a success. And anybody that has already worked on a code base as a team knows it’s not an easy task.

1.3.2 It’s accurate

The app was delivered, and it answers the question it is supposed to answer, or serves the purpose it is supposed to serve. Delivering is not the only thing to keep in mind: you can deliver a working app but it might not work in the way it is supposed to work.

Just as before, accuracy means that between the moment the idea appears in someone’s mind and the moment the app is actually ready to be used, everybody was able to work together toward a common goal.

1.3.3 It’s usable

The app was delivered, it answers the question it is supposed to answer, and it is user-friendly. Unless you’re coding for the sake of the art, there will always be one or more end users. And if these people can’t use the app because it’s too hard to use, too hard to understand, because it’s too slow or there is no inherent logic in how the user experience is designed, then it’s inappropriate to call the app is a success.

1.3.4 It’s immortal

Of course that’s a little bit far fetched, but when designing the app, you should set the ground for robustness in time and aim at a (theoretical) immortality of the app.

Planning for the future is a very important component of a successful Shiny App project. Once the app is out, it’s successful if it can exist in the long run, with all the hazards that implies: new package versions that potentially break the code base, implementation of new features in the global interface, changing key features of the UI or the back-end, and not to mention passing the code base along to someone who has not worked on the first version, and who is now in charge of developing the next version. And this, again, is hard to do without effective planning and efficient engineering. In fact, this new person might simply be you, a month from now. And "You’ll be there in the future too, maintaining code you may have half forgotten under the press of more recent projects. When you design for the future, the sanity you save may be your own.10


  1. We borrow this term from Charity Major, as heard in Test in Production with Charity Majors, CoRecursive

  2. Ironic right?

  3. from The Art of Unix Programming, “Chapter 13: Speaking of Complexity”

  4. see shiny/issues/2043#issuecomment-525640738 for an example

  5. The minification process is the process of removing all blank characters and put everything on one line so that the file in the output is much smaller.

  6. %>% should always have a space before it, and should usually be followed by a new line.”, tidyverse style guide

  7. “Pressure to keep the codebase size down by using extremely dense and complicated implementation techniques can cause a cascade of implementation complexity in the system, leading to an un-debuggable mess.”, from The Art of Unix Programming, “Chapter 13: Speaking of Complexity”

  8. A control flow graph is a graph representing all the possible paths a piece of code can take while it’s executed.

  9. By production-grade, we mean a software that can be used in a context where people use it for doing their job, and where failures or bugs have real-life consequences

  10. The Art of Unix Programming, Eric Steven Raymond


ThinkR Website