Side by side with the published figures
Source:vignettes/articles/figure-comparison.Rmd
figure-comparison.RmdA replication that only reports numbers asks you to trust that the figures line up too. This page puts them next to each other.
The published panels are reproduced from Diegert, Masten and
Poirier (2026), Assessing Omitted Variable
Bias when the Controls are Endogenous (arXiv:2206.02303v6) and
Masten and Poirier (2026), The Effect of Omitted
Variables on the Sign of Regression Coefficients
(arXiv:2208.00552v5), for the purpose of comparison. Copyright rests
with the authors. The panels beneath each one are produced by this
package from the bundled bfg2020 data, by the code shown
with them.
This page is part of the website only. It is excluded from the package tarball, so the reproduced figures are not redistributed on CRAN.
Figure 1, left panel
Bounds on against at . Solid lines leave unrestricted; dashed lines impose . The marks on the zero line are the Table 4 calibration values.
regsensitivity
rx <- seq(0, 1.4, length.out = 300)
solid <- regsen_bounds(form, bfg2020, compare = w1, cbar = 1, rxbar = rx)
dashed <- regsen_bounds(form, bfg2020, compare = w1, cbar = 1, rxbar = rx,
rybar_expr = function(r) r)
a <- as.data.frame(solid); a$spec <- "rybar = Inf"
b <- as.data.frame(dashed); b$spec <- "rybar = rxbar"
df <- rbind(a[, c("rxbar", "bmin", "bmax", "spec")],
b[, c("rxbar", "bmin", "bmax", "spec")])
df$bmin[!is.finite(df$bmin)] <- NA
df$bmax[!is.finite(df$bmax)] <- NA
ticks <- calibrate_rho(form, bfg2020, compare = w1)$rho / 100
ggplot(df, aes(x = rxbar, linetype = spec)) +
geom_hline(yintercept = 0, colour = "grey30", linewidth = 0.3,
linetype = "dotted") +
geom_line(aes(y = bmin), linewidth = 0.5, na.rm = TRUE) +
geom_line(aes(y = bmax), linewidth = 0.5, na.rm = TRUE) +
annotate("segment", x = ticks, xend = ticks, y = -0.25, yend = 0.25,
linewidth = 0.35, colour = "black") +
scale_linetype_manual(values = c("rybar = Inf" = "solid",
"rybar = rxbar" = "dashed")) +
scale_x_continuous(breaks = seq(0, 1.4, 0.2)) +
scale_y_continuous(breaks = seq(-2, 6, 2)) +
coord_cartesian(xlim = c(0, 1.5), ylim = c(-3.2, 7.2)) +
labs(x = expression(bar(r)[X]), y = expression(beta[long]),
linetype = NULL) +
theme_regsen(base_size = 10) +
theme(legend.position = "none")
The two curves cross zero at the paper’s two breakdown points:
c(`rybar = Inf` = abs(solid$breakdown), # paper: 0.804
`rybar = rxbar` = abs(dashed$breakdown)) # paper: 0.959 (96% in prose)
#> rybar = Inf rybar = rxbar
#> 0.8035643 0.9583522Both bounds go infinite at under the dashed specification, and just below 1 under the solid one, which is where both curves leave the panel in the published figure too.
Figure 1, right panel
The breakdown frontier in space, one curve per .
regsensitivity
# One breakdown solve per grid point, parallelised over the rybar grid
# (forking is unavailable on Windows, hence the core fallback).
frontier <- function(cbar, rys, w = w1) {
cores <- if (.Platform$OS.type == "windows") 1L else
min(4L, max(1L, parallel::detectCores(), na.rm = TRUE))
res <- parallel::mclapply(rys, function(ry) {
r <- try(regsen_breakdown(form, bfg2020, compare = w,
cbar = cbar, rybar = ry), silent = TRUE)
if (inherits(r, "try-error")) NA_real_ else abs(r$results$breakdown[1])
}, mc.cores = cores)
rx <- vapply(res, function(z) if (is.numeric(z)) z else NA_real_,
numeric(1))
data.frame(rxbar = rx, rybar = rys, cbar = factor(cbar))
}
# Dense where the curve bends; the flat arm above needs only anchors.
rys <- c(seq(0.6, 1.7, by = 0.05), 2, 3, 4)
fr <- do.call(rbind, lapply(c(1, 0.9, 0.75, 0.5), frontier, rys = rys))
# As in the paper: thick black is cbar = 1; the greys are 0.9, 0.75,
# and 0.5, moving outward. Dotted guides mark rxbar = rybar = 1.
frontier_plot <- function(fr) {
ggplot(fr, aes(x = rxbar, y = rybar, group = cbar)) +
geom_hline(yintercept = 1, colour = "grey60", linewidth = 0.3,
linetype = "dotted") +
geom_vline(xintercept = 1, colour = "grey60", linewidth = 0.3,
linetype = "dotted") +
geom_line(aes(linewidth = cbar == "1", colour = cbar == "1"),
na.rm = TRUE) +
scale_linewidth_manual(values = c(`TRUE` = 0.9, `FALSE` = 0.4),
guide = "none") +
scale_colour_manual(values = c(`TRUE` = "black",
`FALSE` = "grey55"),
guide = "none") +
scale_x_continuous(breaks = seq(0, 4, 0.5)) +
scale_y_continuous(breaks = seq(0, 4, 0.5)) +
coord_cartesian(xlim = c(0, 4), ylim = c(0, 4)) +
labs(x = expression(bar(r)[X]), y = expression(bar(r)[Y])) +
theme_regsen(base_size = 10)
}
frontier_plot(fr)
Where this one stops short, and why. The published panel runs to , each curve having flattened onto a horizontal arm. This reproduction covers the vertical arm only. That arm carries the quantity the paper reports – each curve approaches as grows, which is the breakdown point of Table 1, Panel C. The horizontal arm lives in a region the package declines to compute:
tryCatch(
regsen_bounds(form, bfg2020, compare = w1,
cbar = 1, rxbar = 2, rybar = 0.6),
error = conditionMessage
)
#> [1] "Bounds calculation not implemented in the region where rxbar > rmax(c) > rybar (see DMP 2026)"Rather than return a number it cannot stand behind, the package
raises an error carrying that message (tryCatch() above
captures it for display). So the missing arm is a stated limitation of
the implementation, not a disagreement with the paper.
Figure 3: calibrating with state fixed effects
Figure 3 repeats Figure 1 with the state fixed effects moved into the calibration set. The paper’s point is that is only meaningful relative to the covariates it calibrates against: adding controls with a lot of explanatory power raises selection on observables, so the same data yields a much smaller breakdown point – the paper says it drops from about 80% to about 30%.
regsensitivity
w1_fe <- c(w1, "statea") # state fixed effects now calibrate too
s3 <- regsen_bounds(form, bfg2020, compare = w1_fe, cbar = 1, rxbar = rx)
d3 <- regsen_bounds(form, bfg2020, compare = w1_fe, cbar = 1, rxbar = rx,
rybar_expr = function(r) r)
a3 <- as.data.frame(s3); a3$spec <- "rybar = Inf"
b3 <- as.data.frame(d3); b3$spec <- "rybar = rxbar"
df3 <- rbind(a3[, c("rxbar", "bmin", "bmax", "spec")],
b3[, c("rxbar", "bmin", "bmax", "spec")])
df3$bmin[!is.finite(df3$bmin)] <- NA
df3$bmax[!is.finite(df3$bmax)] <- NA
ggplot(df3, aes(x = rxbar, linetype = spec)) +
geom_hline(yintercept = 0, colour = "grey30", linewidth = 0.3,
linetype = "dotted") +
geom_line(aes(y = bmin), linewidth = 0.5, na.rm = TRUE) +
geom_line(aes(y = bmax), linewidth = 0.5, na.rm = TRUE) +
scale_linetype_manual(values = c("rybar = Inf" = "solid",
"rybar = rxbar" = "dashed")) +
# The published Figure 3 panel uses a wider y window than Figure 1,
# so the reproduction adopts the same one.
scale_y_continuous(breaks = seq(-5, 10, 5)) +
coord_cartesian(xlim = c(0, 1.5), ylim = c(-8.5, 14)) +
labs(x = expression(bar(r)[X]), y = expression(beta[long]),
linetype = NULL) +
theme_regsen(base_size = 10) +
theme(legend.position = "none")
The published Figure 3 also has a right panel: the breakdown frontiers recomputed with the state fixed effects in the calibration set. All of them have shifted inward, the point the paper makes in the surrounding text.
rys3 <- c(seq(0.3, 1.3, by = 0.05), 1.7, 2.5, 4)
fr3 <- do.call(rbind, lapply(c(1, 0.75), frontier, rys = rys3, w = w1_fe))
frontier_plot(fr3)
As in Figure 1, the horizontal arm past the finiteness threshold is not computed; the vertical arm now sits near 0.29 instead of 0.80.
Masten and Poirier (2026), Figures 1 and 2
These two are built on the authors’ application to Satyanath et al. (2017), which is not bundled with this package, and the paper does not report the moments needed to rebuild the curves – it prints coefficients and breakdown points, but no R-squared values and no variances. A full numerical reproduction would require re-running the regressions on the Satyanath et al. microdata (Harvard Dataverse, doi:10.7910/DVN/EE6I7N). The figures are shown here for structure, next to the stylized illustration in the MP examples vignette.
This is not a numerical reproduction, and it should not be read as one. In the published figure the sign-change breakdown is 0.586 while the explain-away breakdown is – a separation of a factor of 55, which is precisely the asymmetry the figure exists to demonstrate. In constructed data the two quantities come out close together; reproducing the gap needs the actual Satyanath moments, not a plausible-looking substitute.
Coverage: every figure in the three papers
| Figure | Status | Reason |
|---|---|---|
| DMP Fig 1 (left) | reproduced | matches on both specifications; crossings 0.8036 / 0.9584 |
| DMP Fig 1 (right) | partly reproduced | vertical arm matches; horizontal arm needs a region the package does not implement |
| DMP Fig 2 | not reproducible | outcome ‘Cut Spending on Poor’ is not in the bundled bfg2020 data |
| DMP Fig 3 (left) | reproduced | matches; breakdown falls to 0.2909 against the paper’s ‘about 30%’ |
| DMP Fig 3 (right) | partly reproduced | vertical arm matches, shifted inward as in the paper; horizontal arm as in Fig 1 (right) |
| MP Fig 1 | structure only | built on Satyanath et al. (2017); moments not reported in full |
| MP Fig 2 | structure only | as Figure 1 |
| MP Fig S1 | not reproducible | Satyanath et al. (2017) data, not bundled |
| Oster Figs 1, 4-6 | out of scope | meta-analysis over Oster’s hand-collected samples of published papers (Figs 1, 4, 5: the 27-paper coefficient-stability sample; Fig 6: her randomized-trials sample) |
| Oster Fig 2 | out of scope | a Monte Carlo of the bias-adjusted estimator’s sampling distribution, a simulation study rather than an analysis the package performs |
| Oster Fig 3 | out of scope | NLSY-79 validation exercise with a bespoke data construction, not a package analysis |
Every panel this package’s methods can produce from the bundled data is reproduced above. The two bounds panels match in full; the two frontier panels match on their vertical arms, while the horizontal arms lie in a region the package deliberately does not implement, as the table records. The rest need data that is not distributed with the package – the full BFG outcome set, or the Satyanath et al. microdata – or, in Oster’s case, are simulation and literature meta-analysis exercises rather than sensitivity analyses of a dataset.