Measurement error

This exercise is a side quest I wanted to crystalize for myself: what exactly happens to R² and the slope when the thing you measure has noise in it? Here is the setup.

True model: y = α + β·x + ε, ε ~ N(0, σ_ε²). We observe ỹ = y + u instead of y, where u ~ N(0, σ_u²) is independent of both x and ε. In the simulation below, α = 0, β = 1, σ_x = 10, σ_ε = 1.

Regressing ỹ on x is the same as regressing (α + β·x + ε + u) on x. Let η = ε + u be the composite error. Since ε and u are independent:

Var(η) = Var(ε + u) = Var(ε) + Var(u) = σ_ε² + σ_u²

η is still uncorrelated with x (u is independent of x by assumption, and ε is uncorrelated with x by OLS assumptions), so the slope estimate stays unbiased. But the extra variance does two things: it inflates standard errors, and it mechanically drags down R².

To see why, recall R² = 1 − SSR/SST. In population terms, R² = Var(ŷ)/Var(ỹ) = β²σ_x² / Var(ỹ). Since Var(ỹ) = β²σ_x² + Var(η) = β²σ_x² + σ_ε² + σ_u²:

R² = β²σ_x² / (β²σ_x² + σ_ε² + σ_u²)

With β = 1, σ_x = 10, σ_ε = 1: at σ_u = 0, R² = 100/101 ≈ 0.99. At σ_u = 10, R² = 100/201 ≈ 0.50. As σ_u → ∞, R² → 0 — the true relationship is unchanged but completely buried in noise.

True model: y = β·x + ε. We observe x̃ = x + u instead of x, where u ~ N(0, σ_u²) is independent of both x and ε.

Running OLS of y on x̃ gives a biased estimator. The probability limit is:

plim(β̂) = β · σ_x² / (σ_x² + σ_u²)

This is attenuation bias: the slope shrinks toward zero by the factor σ_u²/(σ_x² + σ_u²). More measurement noise relative to true variation → harder for OLS to distinguish signal from noise → slope collapses. Unlike Exercise 1, this is a probability-limit result: no amount of additional data removes it.

The left panel below plots attenuation bias against σ_u (the SD of measurement error); the right panel plots against σ_u² (the variance). The variance panel makes the theoretical formula linear — the bias is a straight line through the origin in variance space.