| Title: | Longitudinal Meta-Analysis with Robust Variance Estimation and Sensitivity Analysis |
|---|---|
| Description: | Tools for longitudinal meta-analysis where studies contribute effect sizes at multiple follow-up time points. Implements robust variance estimation (RVE) with Tipton small-sample corrections following Hedges, Tipton, and Johnson (2010) <doi:10.1002/jrsm.5> and Tipton (2015) <doi:10.1037/met0000011>, time-varying sensitivity analysis via the Impact Threshold for a Confounding Variable (ITCV) following Frank (2000) <doi:10.1177/0049124100029002003>, benchmark calibration of the ITCV threshold against observed study-level covariates, spline-based nonlinear time-trend modeling with a nonlinearity test, and leave-k-out fragility analysis across the follow-up trajectory. Designed for researchers synthesising evidence from studies with repeated outcome measurement in education, psychology, health, and the social sciences. |
| Authors: | Subir Hait [aut, cre] (ORCID: <https://orcid.org/0009-0004-9871-9677>) |
| Maintainer: | Subir Hait <[email protected]> |
| License: | MIT + file LICENSE |
| Version: | 0.1.0 |
| Built: | 2026-06-01 07:07:42 UTC |
| Source: | https://github.com/causalfragility-lab/metalong |
Extract stored fitted model objects
fits(x)fits(x)
x |
An |
Named list of fitted model objects, one per estimable time point.
For each follow-up time point, regresses each observed study-level covariate
on the effect sizes using RVE meta-regression, extracts the covariate's
partial correlation with the outcome, and compares it to the
significance-adjusted ITCV threshold from ml_sens(). A covariate that
beats the threshold demonstrates that real-world confounding of at least
that magnitude exists, which is direct evidence of effect fragility.
ml_benchmark( data, meta_obj, sens_obj, yi, vi, study, time, covariates, alpha = NULL, rho = 0.8, small_sample = TRUE, min_k = 3L )ml_benchmark( data, meta_obj, sens_obj, yi, vi, study, time, covariates, alpha = NULL, rho = 0.8, small_sample = TRUE, min_k = 3L )
data |
Long-format |
meta_obj |
Output from |
sens_obj |
Output from |
yi, vi, study, time
|
Column names. |
covariates |
Character vector of observed moderator column names to benchmark. |
alpha |
Significance level (inherits from |
rho |
Working within-study correlation for V matrix. |
small_sample |
Logical; use CR2 + Satterthwaite? |
min_k |
Minimum studies required at a time point. Default |
Object of class ml_benchmark (a data.frame) with columns:
timeFollow-up time.
covariateCovariate name.
kNumber of studies.
r_partialPartial correlation of covariate with effect size.
t_stat, df, p_val
RVE inference for the covariate slope.
itcv_alphaITCV_alpha threshold at this time point.
beats_thresholdLogical: does |r_partial| >= itcv_alpha?
skip_reasonCharacter; reason a cell was skipped, else NA.
The "fragile_summary" attribute contains one row per time with counts.
If an observed covariate (e.g., publication year, sample quality, attrition
rate) has |r_partial| >= ITCV_alpha(t), then an unobserved confounder with
the same relationship to exposure and outcome would be sufficient to nullify
the pooled effect at time . This does not prove confounding–it
calibrates the plausibility threshold.
dat <- sim_longitudinal_meta(k = 15, times = c(0, 6, 12), seed = 2) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") sens <- ml_sens(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time") bench <- ml_benchmark(dat, meta, sens, yi = "yi", vi = "vi", study = "study", time = "time", covariates = c("pub_year", "quality")) print(bench) plot(bench)dat <- sim_longitudinal_meta(k = 15, times = c(0, 6, 12), seed = 2) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") sens <- ml_sens(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time") bench <- ml_benchmark(dat, meta, sens, yi = "yi", vi = "vi", study = "study", time = "time", covariates = c("pub_year", "quality")) print(bench) plot(bench)
Computes fragility indices for each time point by systematically removing
studies and re-estimating the pooled effect. The fragility index at time
is the minimum number of studies whose removal changes the
statistical conclusion (significant -> non-significant or vice versa).
ml_fragility( data, meta_obj, yi, vi, study, time, max_k = 5L, max_combinations = 500L, alpha = NULL, rho = 0.8, small_sample = TRUE, seed = NULL )ml_fragility( data, meta_obj, yi, vi, study, time, max_k = 5L, max_combinations = 500L, alpha = NULL, rho = 0.8, small_sample = TRUE, seed = NULL )
data |
Long-format |
meta_obj |
Output from |
yi, vi, study, time
|
Column names. |
max_k |
Maximum number of studies to remove. Default |
max_combinations |
Maximum number of combinations to test per |
alpha |
Significance level. |
rho |
Working correlation. |
small_sample |
Use CR2 + Satterthwaite? |
seed |
Random seed for sampling combinations. Default |
At each time point, studies are removed one at a time (or in combinations
for the leave-k-out version) and the model is re-fit. The fragility index
is the smallest such that removing any set of studies
flips the significance of the pooled estimate. A fragility index of 1
means a single study's removal changes the conclusion.
For the leave-k-out version, a random sample of combinations is used when
the number of combinations is large (controlled by max_combinations).
Object of class ml_fragility (a data.frame) with columns:
timeFollow-up time.
k_studiesNumber of studies at this time point.
p_originalOriginal p-value.
sig_originalWas the original result significant?
fragility_indexMin number of removals to flip significance.
NA if not found within max_k.
fragility_quotientfragility_index / k_studies (proportion).
study_removedStudy ID whose removal achieved the flip (leave-one-out only).
dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 5) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") frag <- ml_fragility(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time", max_k = 1L, seed = 1) print(frag)dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 5) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") frag <- ml_fragility(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time", max_k = 1L, seed = 1) print(frag)
Fits a random-effects meta-analytic model at each unique time point in a
long-format dataset of multi-wave effect sizes. Inference uses robust
variance estimation (RVE) with optional Tipton (2015) small-sample
corrections via the clubSandwich package.
ml_meta( data, yi, vi, study, time, alpha = 0.05, rho = 0.8, small_sample = TRUE, min_k = 2L, method = "REML", engine = c("rma.uni", "rma.mv") )ml_meta( data, yi, vi, study, time, alpha = 0.05, rho = 0.8, small_sample = TRUE, min_k = 2L, method = "REML", engine = c("rma.uni", "rma.mv") )
data |
A |
yi |
Character. Name of the effect-size column. |
vi |
Character. Name of the sampling-variance column. |
study |
Character. Name of the study-ID column (cluster variable). |
time |
Character. Name of the follow-up time column (numeric). |
alpha |
Significance level for confidence intervals and p-values.
Default |
rho |
Assumed within-study correlation between effect sizes (used only
when |
small_sample |
Logical. If |
min_k |
Integer. Minimum number of studies required to fit a model at
a given time point. Default |
method |
Character. Variance estimator passed to metafor. Default
|
engine |
Character. Fitting engine: |
An object of class ml_meta (a data.frame) with one row per time
point and columns: time, k, theta, se, df, t_stat, p_val,
ci_lb, ci_ub, tau2, note.
Attributes:
"fits"Named list of fitted model objects (one per time point).
"weights_by_time"Named list of weight vectors for downstream
use by ml_sens() and ml_benchmark().
"engine", "alpha", "rho", "small_sample"
Call metadata.
Two fitting engines are supported:
"rma.uni" (default)metafor::rma.uni() – appropriate when each
study contributes exactly one effect size per time point. Simpler,
faster, and stores tau2 directly from the REML estimate.
"rma.mv"metafor::rma.mv() with a prebuilt working covariance
matrix – appropriate when studies contribute multiple effect sizes at
the same time point (dependent effects within cluster). Requires the
rho argument.
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010). Robust variance estimation in meta-regression with dependent effect size estimates. Research Synthesis Methods, 1(1), 39-65.
Tipton, E. (2015). Small sample adjustments for robust variance estimation with meta-regression. Psychological Methods, 20(3), 375-393.
ml_sens(), ml_benchmark(), ml_spline()
dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 1) result <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") print(result) plot(result) # rma.mv engine for dependent effects result_mv <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time", engine = "rma.mv", rho = 0.8)dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 1) result <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") print(result) plot(result) # rma.mv engine for dependent effects result_mv <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time", engine = "rma.mv", rho = 0.8)
Produces a multi-panel figure combining the pooled trajectory, confidence band, spline fit (if supplied), and ITCV sensitivity profile. Designed for direct inclusion in manuscripts.
ml_plot( meta_obj, sens_obj = NULL, bench_obj = NULL, spline_obj = NULL, frag_obj = NULL, ncol = NULL, main = NULL, col_effect = "#2166ac", col_sens = "#d73027", col_spline = "#1a9641", delta = NULL )ml_plot( meta_obj, sens_obj = NULL, bench_obj = NULL, spline_obj = NULL, frag_obj = NULL, ncol = NULL, main = NULL, col_effect = "#2166ac", col_sens = "#d73027", col_spline = "#1a9641", delta = NULL )
meta_obj |
Output from |
sens_obj |
Output from |
bench_obj |
Output from |
spline_obj |
Output from |
frag_obj |
Output from |
ncol |
Number of columns in the panel layout. Default auto. |
main |
Overall figure title. |
col_effect |
Colour for the pooled effect trajectory. |
col_sens |
Colour for the ITCV line. |
col_spline |
Colour for the spline curve. |
delta |
Fragility benchmark line on the ITCV panel. Inherits from
|
Invisibly returns a list of the objects passed in.
dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 1) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") sens <- ml_sens(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time") ml_plot(meta, sens_obj = sens) spl <- ml_spline(meta, df = 2) ml_plot(meta, sens_obj = sens, spline_obj = spl, main = "Longitudinal Meta-Analysis Profile")dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 1) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") sens <- ml_sens(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time") ml_plot(meta, sens_obj = sens) spl <- ml_spline(meta, df = 2) ml_plot(meta, sens_obj = sens, spline_obj = spl, main = "Longitudinal Meta-Analysis Profile")
Computes the Impact Threshold for a Confounding Variable (ITCV) at each
follow-up time point using the pooled estimates and robust inference from
ml_meta(). Two versions are returned: the raw ITCV (threshold to nullify
the pooled effect) and the significance-adjusted ITCV_alpha (threshold to render
the result non-significant under small-sample-corrected inference).
ml_sens(data, meta_obj, yi, vi, study, time, alpha = NULL, delta = 0.15)ml_sens(data, meta_obj, yi, vi, study, time, alpha = NULL, delta = 0.15)
data |
A |
meta_obj |
Output from |
yi, vi, study, time
|
Column names (same meaning as in |
alpha |
Significance level. Defaults to the value stored in
|
delta |
Numeric. User-defined practical fragility benchmark: time
points with |
An object of class ml_sens (a data.frame) with columns:
timeFollow-up time.
theta, se, df
Copied from meta_obj.
syWeighted SD of observed effect sizes.
r_effectPooled effect on correlation scale.
itcvRaw ITCV: confounding needed to nullify the estimate.
itcv_alphaSignificance-adjusted ITCV: confounding needed to make the result non-significant.
fragileLogical; TRUE when itcv_alpha < delta.
Attributes include trajectory summaries (itcv_min, itcv_mean,
fragile_prop) and a "fragile_times" character vector.
At each time , let be the pooled effect,
the weighted variance of observed effect sizes, and
the
minimum effect still deemed significant. The correlation-scale pooled effect
is
and the raw ITCV is . The significance-adjusted version
replaces with .
Frank, K. A. (2000). Impact of a confounding variable on a regression coefficient. Sociological Methods & Research, 29(2), 147-194.
ml_meta(), ml_benchmark(), ml_plot()
dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 1) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") sens <- ml_sens(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time") print(sens) plot(sens)dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 1) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") sens <- ml_sens(dat, meta, yi = "yi", vi = "vi", study = "study", time = "time") print(sens) plot(sens)
Fits a natural cubic spline meta-regression over follow-up time using the
pooled time-point estimates from ml_meta(). Produces a smooth pooled
trajectory with simultaneous pointwise confidence bands and tests for
nonlinearity.
ml_spline(meta_obj, df = 3L, n_pred = 200L, alpha = NULL, test_linear = TRUE)ml_spline(meta_obj, df = 3L, n_pred = 200L, alpha = NULL, test_linear = TRUE)
meta_obj |
Output from |
df |
Degrees of freedom for the natural cubic spline. Default
|
n_pred |
Number of prediction points for the smooth curve. Default
|
alpha |
Confidence level (inherits from |
test_linear |
Logical. If |
The spline is fit by weighted least squares on the ml_meta() estimates,
using 1 / se^2 as weights (i.e., inverse squared SE weighting to reflect
the precision of each time-point estimate). This is a second-stage model.
For a fully joint spline model at the individual-effect level, users should
call metafor::rma.mv() directly with mods = ~ ns(time, df). This
function is primarily intended for visualisation and trajectory testing.
Object of class ml_spline with elements:
preddata.frame with time, fit, ci_lb, ci_ub for
the smooth prediction grid.
coefSpline coefficient estimates.
vcovCoefficient covariance matrix.
r_squaredWeighted R-squared of the spline fit.
p_nonlinearp-value for nonlinearity test (if requested).
dfSpline degrees of freedom used.
meta_objThe original ml_meta object (for plotting).
dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12, 24), seed = 3) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") spl <- ml_spline(meta, df = 2) print(spl) plot(spl)dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12, 24), seed = 3) meta <- ml_meta(dat, yi = "yi", vi = "vi", study = "study", time = "time") spl <- ml_spline(meta, df = 2) print(spl) plot(spl)
Generates a synthetic long-format dataset suitable for testing and
illustrating all metaLong functions. Studies contribute effect sizes at
multiple follow-up time points with within-study correlation.
sim_longitudinal_meta( k = 20L, times = c(0, 6, 12, 24), mu = 0.4, tau = 0.2, v_range = c(0.02, 0.12), missing_prop = 0, add_covariates = TRUE, seed = NULL )sim_longitudinal_meta( k = 20L, times = c(0, 6, 12, 24), mu = 0.4, tau = 0.2, v_range = c(0.02, 0.12), missing_prop = 0, add_covariates = TRUE, seed = NULL )
k |
Number of studies. Default |
times |
Numeric vector of follow-up time points.
Default |
mu |
Named numeric vector of true effects at each time point,
or a single value (recycled). Default |
tau |
Between-study SD. Default |
v_range |
Two-element vector for the uniform sampling variance range.
Default |
missing_prop |
Proportion of study x time combinations to set missing
(simulates unbalanced follow-up). Default |
add_covariates |
Logical. If |
seed |
Random seed. Default |
The true effect at time for study is
where is a time-varying mean effect (optionally nonlinear),
is a study-level random effect, and
is sampling error. Within-study
correlation between time points is introduced through .
A data.frame in long format with columns:
studyStudy identifier (character).
timeFollow-up time.
yiObserved effect size.
viSampling variance.
pub_year, quality, n
Study-level covariates
(if add_covariates = TRUE).
dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 42) head(dat) # Nonlinear true trajectory mu_t <- c("0" = 0.2, "6" = 0.5, "12" = 0.4, "24" = 0.1) dat2 <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12, 24), mu = mu_t, missing_prop = 0.1, seed = 99)dat <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12), seed = 42) head(dat) # Nonlinear true trajectory mu_t <- c("0" = 0.2, "6" = 0.5, "12" = 0.4, "24" = 0.1) dat2 <- sim_longitudinal_meta(k = 10, times = c(0, 6, 12, 24), mu = mu_t, missing_prop = 0.1, seed = 99)
Tidy an metaLong object into a clean data frame
tidy(x, ...)tidy(x, ...)
x |
A |
... |
Additional arguments (unused). |
A tidy data.frame.