Estimating Beta Using Regression in Python

Quant Lab

The Investor Problem

Last week’s Analyst Desk argued that beta (an asset’s sensitivity to market movements) is the CAPM’s central quantity: the measure of the risk that diversification cannot remove. This week we stop taking beta on faith and estimate it ourselves.

The exercise matters for two reasons. Practically, beta estimates feed real decisions; costs of capital, hedge ratios, performance attribution, and anyone consuming those numbers should know how they are made. Educationally, beta estimation is the perfect first regression: simple enough to fit in twenty lines of Python, rich enough to teach the entire discipline of interpreting statistical estimates; standard errors, instability, and the gap between a number and the truth it estimates. The analyst who has computed her own betas never again mistakes a published beta for a fact.

The Core Idea

Beta is, mechanically, the slope of a regression line. Take an asset’s returns and the market’s returns over the same periods, plot one against the other, and fit the best straight line:

Rᵢ,ₜ = α + β × Rm,ₜ + εₜ

The slope β says: when the market moves 1%, how much does this asset move on average? The intercept α is the return unexplained by market exposure (the CAPM says it should be zero; managers advertise it when positive). The residuals ε are the asset’s idiosyncratic movements, the diversifiable part. Ordinary least squares (OLS) chooses the slope minimizing squared residuals, which yields the textbook identity: beta equals the covariance of asset and market returns divided by the market’s variance.

The Model: Python Implementation

We simulate a market and three stocks with known true betas; a defensive utility (β = 0.5), a cyclical bank (β = 1.3), and a speculative small-cap (β = 1.8, mostly noise). We can compare estimates against truth, a luxury real data never offers.

import numpy as np
rng = np.random.default_rng(3)
# ------------------------------------------------------------------
# 1. Simulate 5 years of monthly returns with known betas
# ------------------------------------------------------------------
n = 60                                     # 60 months
rm = rng.normal(0.008, 0.045, n)           # market: ~10%/yr, ~16% vol
true = {"Utility": (0.5, 0.02), "Bank": (1.3, 0.04),
        "SmallCap": (1.8, 0.09)}           # (beta, idio vol)
stocks = {name: b * rm + rng.normal(0, s, n)
          for name, (b, s) in true.items()}
# ------------------------------------------------------------------
# 2. OLS beta by hand: cov/var identity, plus standard error
# ------------------------------------------------------------------
def ols_beta(r, m):
    mx, my = m.mean(), r.mean()
    beta = ((m - mx) * (r - my)).sum() / ((m - mx) ** 2).sum()
    alpha = my - beta * mx
    resid = r - alpha - beta * m
    se = np.sqrt((resid @ resid) / (len(r) - 2) / ((m - mx) ** 2).sum())
    r2 = 1 - (resid @ resid) / ((r - my) @ (r - my))
    return beta, alpha, se, r2
print(f"{'Stock':<9} {'true β':>6} {'est β':>6} {'SE':>5} "
      f"{'95% CI':>14} {'R²':>5}")
for name, r in stocks.items():
    b, a, se, r2 = ols_beta(r, rm)
    lo, hi = b - 1.96 * se, b + 1.96 * se
    print(f"{name:<9} {true[name][0]:>6.2f} {b:>6.2f} {se:>5.2f} "
      f"[{lo:>5.2f},{hi:>5.2f}] {r2:>5.2f}")
# ------------------------------------------------------------------
# 3. Instability: rolling 24-month beta for the bank
# ------------------------------------------------------------------
r = stocks["Bank"]
roll = [ols_beta(r[t-24:t], rm[t-24:t])[0] for t in range(24, n)]
print(f"\nBank rolling 24m beta: min {min(roll):.2f}, "
      f"max {max(roll):.2f} (true: 1.30)")

Representative output:

Stock     true β  est β    SE         95% CI    R²
Utility     0.50   0.43  0.06 [ 0.32, 0.55]  0.48
Bank        1.30   1.22  0.10 [ 1.02, 1.42]  0.71
SmallCap    1.80   1.91  0.22 [ 1.47, 2.35]  0.56
Bank rolling 24m beta: min 1.11, max 1.40 (true: 1.30)

Three lessons stand in this small table, and they generalize to every beta you will ever meet.

The estimate is a range wearing a point’s clothing. Even with clean simulated data and a constant true beta, five years of monthly observations delivers the bank’s beta only as “somewhere between 1.0 and 1.5, probably.” The small-cap which is mostly idiosyncratic noise, like most small-caps, spans a full point. Any decision that changes materially when beta moves from 1.2 to 1.4 is a decision resting on statistical fog. This is why practitioners quote betas rounded to a decimal at most, and why services publish adjusted betas shrunk toward 1 (Blume’s adjustment, acknowledging that extreme estimates are usually part error).

R² tells you how much beta describes. The bank’s 0.71 means the market explains most of its movement; the utility and small-cap sit near 0.5, so roughly half their variance is company-specific, invisible to beta, removable by diversification, and irrelevant to CAPM expected return. Two stocks can share a beta and differ wildly in total volatility; beta was never meant to measure the latter.

The rolling window shows the deeper trouble. Even with a constant true beta, 24-month estimates wander between roughly 1.1 and 1.4 on sampling noise alone — and other seeds wander further. Real betas also drift as companies change leverage, business mix, and size, so a published beta is a noisy estimate of a moving target, computed on a window someone chose. Change the window (2 vs 5 years), the frequency (daily vs monthly), or the index, and the number changes, sometimes by a lot. When two data providers quote different betas for the same stock, neither is wrong; they made different choices.

Running it on real data

Substituting real returns requires only replacing the simulation block: download a stock’s and an index’s price history (any data library works), compute percentage returns, and align dates. Two practical warnings for thin markets: use monthly rather than daily returns where trading is sporadic. Stale daily prices bias beta downward severely (a stock that doesn’t trade the day the market falls shows spurious independence); and check that the local index is itself diversified enough to stand in for “the market”, a five-stock index makes every beta partly a beta against its largest member.

Limitations and Real-World Complexity

Beyond estimation noise, remember what even a perfect beta would and wouldn’t give you. It is a historical, linear, average relationship: it says nothing about behaviour in crashes specifically (assets often co-move more in stress. The betas that are conditional on bad months run higher than full-sample betas), nothing about company-specific catastrophes, and per last week’s discussion its link to expected returns is empirically weak: low-beta stocks have historically out-earned their CAPM due. Estimating beta well and trusting it moderately are both parts of the craft. And the regression’s tidy output conceals the joint-hypothesis problem inherited from the model: our α and β are estimates relative to an index proxy, not to the true market portfolio, which no one has ever measured.

The Long-Term Lesson

The first regression an investor runs personally is a small rite of passage, because the experience deposits a permanent instinct: every statistical number in finance is an estimate with a confidence interval, produced by choices, from a sample that may not resemble the future. Betas, volatilities, correlations, Sharpe ratios, all of them wear the same fog our little table made visible.

That instinct has a practical edge. When an analyst’s report says “beta: 1.42,” you will now silently append “±0.25, on their chosen window, assuming stability”, and weight the conclusion accordingly. When a model’s output depends delicately on such inputs, you will demand to see the sensitivity, not the point estimate. Statistical literacy in investing is not the ability to compute the numbers. It is the refusal to believe them more than they deserve, and the only reliable way to acquire it is to compute a few yourself, against a truth you know, and watch how far they miss.