Overview
Shiny Umbrellas contains four interfaces to umbrella plots that demonstrate the association between means and standard deviations of rating scale data. The 'Discrete Umbrella' describes this association in its dependency on the sample size, and the other three tabs are use cases of a formalization that generalizes across sample sizes.
Discrete Umbrella
Heathers et al. (2018) visualize the possible combinations of means and standard deviations for a five point scale with ten participants. This tab provides a (for computational reasons) very limited interface to simulate this illustration. The algorithm identifies individual participant data sets using the gtools package (Warnes et al., 2023).
Error Checking
In 'DEBIT: A simple consistency test for binary data', Heathers & Brown (2019) use a Bernoulli-formalization to detect inconsistencies in reports of means an standard deviations for binary data. In 'Under my Umbrella: Rating Scales Obscure Statistical Power and Effect Size Heterogeneity' (Jens H. Fünderich, Lukas J. Beinhauer, Frank Renkewitz, 2025), we use a similar Bernoulli-formalization to identify the umbrella from Heathers et al. (2018) that is independent of the sample size. This provides a computationally quick, but - especially in smaller samples - less precise method than SPRITE for checking the congruence between information on a scale, mean and standard deviation.
Click here, if you want to find out more about forensic meta-science techniques.Power
A priori power analyses require us to make assumptions around both, the unstandardized effect and the expected variability, the standard deviations. The restrictions of the scale allow us to identify the largest possible standard deviations for an assumed mean difference. This restriction applies to a single rating scale and aggregations of any number of identical rating scales (a measure with five 7-point rating scales, for example). Considering such restrictions in a priori calculations of nominal (!) power can contribute to an appropriate allocation of resources.
Meta-Analysis
The pooled standard deviation is a weighted average of the standard deviations of both groups. Thus, in the case of data from a rating scale, we standardize with a pooled_SD that is affected by the restrictions described by the umbrella. This can lead to deviations between the distributions of d and MD, making the identified heterogeneity dependent on the choice to standardize the effects or not. This tab visualizes meta-analytic data sets of means and standard deviations as well as relative heterogeneity measures for unstandardized and standardized mean differences (MD and d). Meta-analyses are implemented using metafor (Viechtbauer, 2010)
Ressources
Under my Umbrella: Rating Scales Obscure Statistical Power and Effect Size Heterogeneity (Fünderich et al., 2025)Heathers & Brown (2019)
Heathers et al. (2018)
Viechtbauer (2010)
About
This Shiny App was developed by Jens H. Fünderich.
ORCIDVersion: 0.3.0
GitHub Repository with the code for this Shiny App.