Correlation-adjusted Bayesian regularization

method=‘correlation adjusted’: joint Gaussian conditioning

Scope and names

This method is called correlation adjusted. Select method="correlation adjusted" in BayesianPredictionSettings.

This is a different statistical model from the independent update, not a correction applied to its three precision weights. The consensus bias option remains available and is applied before fitting either model.

Coordinates and aligned sample

Levels use the same previous-actual growth coordinates, in the symbols of the notation page: realised growth \(g_t\), consensus growth \(x_{C,t}\) from the bias-adjusted consensus \(c_t^*\), and model growth \(x_{M,t}\):

\[ g_t=\frac{y_t}{y_{t-1}}-1,\qquad x_{C,t}=\frac{c_t^*}{y_{t-1}}-1,\qquad x_{M,t}=\frac{m_t}{y_{t-1}}-1. \]

The joint sample contains only periods where all three growth observations are present. Missing values are removed jointly, so covariance entries use the same sample.

Stack the three growths of past period \(i\) into \(u_i=(g_i,x_{C,i},x_{M,i})^\top\). Over the \(n\) complete periods, the joint mean is \(\mu\) and the joint covariance \(\Sigma\):

\[ \mu=\frac1n\sum_i u_i, \qquad \Sigma=\frac1{n-1}\sum_i(u_i-\mu)(u_i-\mu)^\top. \]

The sample requires at least \(\max(2,\texttt{minimum\_observations})\) complete observations.

Conditional expectation

Partition the joint mean and covariance into realised growth \(g\) and the two observed signals \(S=(C,M)\):

\[ \mu=\begin{pmatrix}\mu_g\\\mu_S\end{pmatrix},\qquad \Sigma=\begin{pmatrix} \Sigma_{gg}&\Sigma_{gS}\\ \Sigma_{Sg}&\Sigma_{SS} \end{pmatrix}. \]

The published growth estimate \(\hat\theta\) is the expected realised growth given today’s consensus growth \(x_C\) and model growth \(x_M\):

\[ \hat\theta =\mu_g+\Sigma_{gS}(\Sigma_{SS}+\varepsilon I)^{-1} \left(\begin{pmatrix}x_C\\x_M\end{pmatrix}-\mu_S\right), \qquad \varepsilon=10^{-12}. \]

The implementation solves the linear system; it does not form an explicit matrix inverse. The prediction level is \(\hat y_t=y_{t-1}(1+\hat\theta)\).

  • Numerical ridge: \(\varepsilon I\) keeps the signal matrix invertible in ordinary singular cases. It is not estimated covariance shrinkage.
  • Weights: Conditional coefficients can be negative and need not sum to one. There is an intercept from the joint means.
  • Refusal: An insufficient sample or failed linear solve produces no prediction. There is no switch to the independent estimator.
  • Diagnostics: Only prediction is available for this method. Independent precision weights and intervals are not meaningful substitutes.

Worked example

Take all three historical means equal to 8 and covariance

\[ \Sigma=16\begin{pmatrix} 1&0.9&0.7\\ 0.9&1&0.8\\ 0.7&0.8&1 \end{pmatrix}. \]

Ignoring the tiny numerical ridge, the conditional coefficients are

\[ \Sigma_{gS}\Sigma_{SS}^{-1} =\begin{pmatrix}17/18&-1/18\end{pmatrix}. \]

For consensus growth \(x_C=9\) and model growth \(x_M=13\):

\[ \hat\theta =8+\frac{17}{18}(9-8)-\frac1{18}(13-8) =8\frac23. \]

The model’s coefficient is negative because its extra movement is redundant with consensus in this covariance structure. Independent precision weighting cannot express that effect.

Historical and forward fitting

  • Backtests: At each timestamp in the complete joint sample, fit only sample rows strictly earlier than that timestamp. The current actual never enters its own moments.
  • Predictions: Fit joint means and covariance once on the complete realized training history. Apply that fit to aligned consensus and live model growth.
  • Combined: Keep historical predictions strictly before the first available forward prediction, then append the forward series.
  • Training start: start_date cuts off levels before growth conversion and joint-sample construction.
  • Bias correction: Shifted Kalman consensus bias correction precedes growth and covariance estimation. Its equations and defaults are in the independent reference.

Configuration restrictions

  • Model: Required; include_model=False is rejected.
  • Values: Only ("prediction",) is supported.
  • Dispersion/count: Not supported for this method.
  • Publication-relative horizons: Not supported for this method.
  • Explicit as-of: The panel is still anchored before the requested UTC reading moment, and historical estimates are read before first publication.

Assumptions and limits

  • Joint Gaussian model: The formula is the conditional mean of a jointly Gaussian vector (Winkler 1981). Outside that model it is a linear projection.
  • Small samples: Joint covariance estimates can be unstable even when the matrix solve succeeds (Smith and Wallis 2009).
  • Collinearity: The numerical ridge does not eliminate unstable coefficients caused by nearly redundant signals.
  • Exchangeability: Historical model backtests must represent the process that produces live model predictions.
  • Intervals: No correlation-adjusted interval is defined, and the package does not invent one.

Executable example

Show the code 
import pandas as pd
import tutorial_support as examples
import postforecast as pf

settings = pf.BayesianPredictionSettings(
    version="correlation-example",
    method="correlation adjusted",
    series="predictions",
)
pf.bayesian_kpi_predictions(
    examples.quarterly_history(),
    pd.Timestamp("2026-05-10", tz="UTC"),
    forecast_source="model",
    settings=settings,
)[["entity", "period", "prediction", "eligible", "eligibility_reason"]]
entity period prediction eligible eligibility_reason
0 ZS 2026-06 812.084193 True eligible

References

Smith, Jeremy, and Kenneth F. Wallis. 2009. “A Simple Explanation of the Forecast Combination Puzzle.” Oxford Bulletin of Economics and Statistics 71 (3): 331–55. https://doi.org/10.1111/j.1468-0084.2008.00541.x.
Winkler, Robert L. 1981. “Combining Probability Distributions from Dependent Information Sources.” Management Science 27 (4): 479–88. https://doi.org/10.1287/mnsc.27.4.479.