Calculate the proportion of true effect sizes above a meaningful threshold
Source:R/proportionMID.R
proportionMID.Rd
Based on results of the runMetaAnalysis()
, this function allows to
estimate the proportion of true effect sizes that exceed a user-defined
meaningful (e.g. clinically relevant) threshold.
Arguments
- model
A class
runMetaAnalysis
object, created by therunMetaAnalysis()
function.- mid
A
numeric
value, indicating a clinically relevant effect threshold (e.g. a minimally important difference; \(MID\); Cuijpers et al., 2014) that should be used to estimate the proportion of true effect sizes that exceed this cut-off. If the outcome measure used inmodel
is Hedges' \(g\), the provided value should also be a standardized mean difference. If the outcome measure ofmodel
is a risk ratio, the treshold should also be provided as an (untransformed) risk ratio.- which
The model in
model
that should be used to estimate the proportions. Defaults to"all"
, which means that proportions are calculated for all models. Alternatively, possible values are"overall"
,"combined"
,"lowest"
,"highest"
,"outliers"
,"influence"
and"rob"
, if these models are available in themodel
object. IfcorrectPublicationBias()
has been run,"trimfill"
,limitmeta
andselection
are also possible options. It is also possible to concatenate model names, meaning that proportions are calculated for all the supplied models.- test
By default, the function estimates the proportion of true effects below the provided threshold in
mid
(test="smaller"
). Alternatively, one can specifytest="bigger"
. This will calculate the proportion of true effects above the treshold.- plot
Should a density plot illustrating the proportions be returned? Defaults to
FALSE
. Please note that an S3plot
method is available for outputs of this function even whenplot=FALSE
(see "Details").
Value
Returns an object of class "proportionMID"
. An S3 plot
method
is defined for this object class, which allows to create a density plot illustrating
the estimated proportions, using the model-based estimate of the pooled effect size
and between-study heterogeneity \(\tau^2\).
Details
The proportionMID
function implements an approach to estimate
the proportion of true effect sizes exceeding a (scientifically or clinically)
relevant threshold, as proposed by Mathur & VanderWeele (2019). These estimated
proportions have been suggested as a useful metric to determine
the impact that between-study heterogeneity in a meta-analysis has on the
"real-life" interpretation of results.
If, for example, a pooled effect is significant, high between-study heterogeneity can still mean that a substantial proportion of true effects in the studies population are practicially irrelevant, or even negative. Conversely overall non-significant effects, in the face of large heterogeneity, can still mean that a substantial proportion of studies have non-negligible true effects.
As recommended by Mathur & VanderWeele (2019),
the proportionMID
function
also automatically calculates the proportion of true effects exceeding the
"inverse" of the user-defined effect (e.g., if mid=-0.24
, by default,
the function also estimates the proportion of true effects that is larger
than \(g\)=0.24; note the changed sign). This can be used to check for e.g. clinically
relevant negative effects.
When the plot
method is used, or when plot
is set to TRUE
in the function,
a plot showing the assumed distribution of true effects based on the estimated
meta-analytic model is created. Notably, is is assumed that the random-effects
distribution of true effect sizes is approximately normal. This simplifying assumption
is required for this (and many other meta-analytic methods;
Jackson & White, 2018)
to hold.
Confidence intervals provided by the functions are calculated using the
asymptotic closed-form solution derived using the Delta method in Mathur &
VanderWeele (2019).
Following their recommendations, a warning is printed
when \(p\)<0.15 or \(p\)>0.85, since in this case the asymptotic
CIs should be interpreted cautiously; CIs based on boostrapping would be preferable
in this scenario and can be calculated using the MetaUtility::prop_stronger()
function.
References
Cuijpers, P., Turner, E. H., Koole, S. L., Van Dijke, A., & Smit, F. (2014). What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depression and Anxiety, 31(5), 374-378.
Jackson, D., & White, I. R. (2018). When should meta-analysis avoid making hidden normality assumptions?. Biometrical Journal, 60(6), 1040-1058.
Mathur, M. B., & VanderWeele, T. J. (2019). New metrics for meta-analyses of heterogeneous effects. Statistics in Medicine, 38(8), 1336-1342.
Author
Mathias Harrer mathias.h.harrer@gmail.com, Paula Kuper paula.r.kuper@gmail.com, Pim Cuijpers p.cuijpers@vu.nl
Examples
if (FALSE) {
# Run meta-analysis; then estimate the proportion
depressionPsyCtr %>%
filterPoolingData(condition_arm1 == "cbt") %>%
runMetaAnalysis() -> x
proportionMID(x, mid = -0.24)
proportionMID(x, mid = -0.24, "outliers") %>% plot()
# If bootstrap CIs are requested in runMetaAnalysis,
# calculation of CIs around p is possible for all available
# models.
depressionPsyCtr %>%
filterPoolingData(condition_arm1 == "cbt") %>%
runMetaAnalysis(i2.ci.boot = TRUE, nsim.boot = 1000) %>%
correctPublicationBias() -> x
proportionMID(x, mid = -0.33)
# Run meta-analysis based on RRs; then estimate proportion
# of true effects bigger than the defined threshold
depressionPsyCtr %>%
filterPoolingData(
condition_arm1 %in% c("cbt", "pst",
"dyn", "3rd wave")) %>%
runMetaAnalysis(es.measure = "RR") %>%
proportionMID(mid = 1.13, test = "bigger")
}