# Calculate the proportion of true effect sizes above a meaningful threshold

Source:`R/proportionMID.R`

`proportionMID.Rd`

Based on results of the `runMetaAnalysis()`

, this function allows to
estimate the proportion of true effect sizes that exceed a user-defined
meaningful (e.g. clinically relevant) threshold.

## Arguments

- model
A class

`runMetaAnalysis`

object, created by the`runMetaAnalysis()`

function.- mid
A

`numeric`

value, indicating a clinically relevant effect threshold (e.g. a minimally important difference; \(MID\); Cuijpers et al., 2014) that should be used to estimate the proportion of true effect sizes that exceed this cut-off. If the outcome measure used in`model`

is Hedges' \(g\), the provided value should also be a standardized mean difference. If the outcome measure of`model`

is a risk ratio, the treshold should also be provided as an (untransformed) risk ratio.- which
The model in

`model`

that should be used to estimate the proportions. Defaults to`"all"`

, which means that proportions are calculated for all models. Alternatively, possible values are`"overall"`

,`"combined"`

,`"lowest"`

,`"highest"`

,`"outliers"`

,`"influence"`

and`"rob"`

, if these models are available in the`model`

object. If`correctPublicationBias()`

has been run,`"trimfill"`

,`limitmeta`

and`selection`

are also possible options. It is also possible to concatenate model names, meaning that proportions are calculated for all the supplied models.- test
By default, the function estimates the proportion of true effects

*below*the provided threshold in`mid`

(`test="smaller"`

). Alternatively, one can specify`test="bigger"`

. This will calculate the proportion of true effects*above*the treshold.- plot
Should a density plot illustrating the proportions be returned? Defaults to

`FALSE`

. Please note that an S3`plot`

method is available for outputs of this function even when`plot=FALSE`

(see "Details").

## Value

Returns an object of class `"proportionMID"`

. An S3 `plot`

method
is defined for this object class, which allows to create a density plot illustrating
the estimated proportions, using the model-based estimate of the pooled effect size
and between-study heterogeneity \(\tau^2\).

## Details

The `proportionMID`

function implements an approach to estimate
the proportion of true effect sizes exceeding a (scientifically or clinically)
relevant threshold, as proposed by Mathur & VanderWeele (2019). These estimated
proportions have been suggested as a useful metric to determine
the impact that between-study heterogeneity in a meta-analysis has on the
"real-life" interpretation of results.

If, for example, a pooled effect is significant, high between-study heterogeneity
can still mean that a substantial proportion of true effects in the studies
population are practicially irrelevant, or even negative. Conversely
overall non-significant effects, in the face of large heterogeneity, can still
mean that a substantial proportion of studies have non-negligible *true* effects.

As recommended by Mathur & VanderWeele (2019),
the `proportionMID`

function
also automatically calculates the proportion of true effects exceeding the
*"inverse"* of the user-defined effect (e.g., if `mid=-0.24`

, by default,
the function also estimates the proportion of true effects that is larger
than \(g\)=0.24; note the changed sign). This can be used to check for e.g. clinically
relevant negative effects.

When the `plot`

method is used, or when `plot`

is set to `TRUE`

in the function,
a plot showing the assumed distribution of true effects based on the estimated
meta-analytic model is created. Notably, is is assumed that the random-effects
distribution of true effect sizes is approximately normal. This simplifying assumption
is required for this (and many other meta-analytic methods;
Jackson & White, 2018)
to hold.

Confidence intervals provided by the functions are calculated using the
asymptotic closed-form solution derived using the Delta method in Mathur &
VanderWeele (2019).
Following their recommendations, a warning is printed
when \(p\)<0.15 or \(p\)>0.85, since in this case the asymptotic
CIs should be interpreted cautiously; CIs based on boostrapping would be preferable
in this scenario and can be calculated using the `MetaUtility::prop_stronger()`

function.

## References

Cuijpers, P., Turner, E. H., Koole, S. L., Van Dijke, A., & Smit, F. (2014).
What is the threshold for a clinically relevant effect? The case of
major depressive disorders. *Depression and Anxiety, 31*(5), 374-378.

Jackson, D., & White, I. R. (2018). When should meta-analysis avoid making
hidden normality assumptions?. *Biometrical Journal, 60*(6), 1040-1058.

Mathur, M. B., & VanderWeele, T. J. (2019). New metrics for meta-analyses of
heterogeneous effects. *Statistics in Medicine, 38*(8), 1336-1342.

## Author

Mathias Harrer mathias.h.harrer@gmail.com, Paula Kuper paula.r.kuper@gmail.com, Pim Cuijpers p.cuijpers@vu.nl

## Examples

```
if (FALSE) {
# Run meta-analysis; then estimate the proportion
depressionPsyCtr %>%
filterPoolingData(condition_arm1 == "cbt") %>%
runMetaAnalysis() -> x
proportionMID(x, mid = -0.24)
proportionMID(x, mid = -0.24, "outliers") %>% plot()
# If bootstrap CIs are requested in runMetaAnalysis,
# calculation of CIs around p is possible for all available
# models.
depressionPsyCtr %>%
filterPoolingData(condition_arm1 == "cbt") %>%
runMetaAnalysis(i2.ci.boot = TRUE, nsim.boot = 1000) %>%
correctPublicationBias() -> x
proportionMID(x, mid = -0.33)
# Run meta-analysis based on RRs; then estimate proportion
# of true effects bigger than the defined threshold
depressionPsyCtr %>%
filterPoolingData(
condition_arm1 %in% c("cbt", "pst",
"dyn", "3rd wave")) %>%
runMetaAnalysis(es.measure = "RR") %>%
proportionMID(mid = 1.13, test = "bigger")
}
```