Independent Bayesian regularization

method=‘independent’: prior, model and consensus precision

Scope and names

This method is called independent. postforecast.bayesian_kpi_predictions implements it over a tidy input panel. It reads the panel at an explicit UTC as_of moment.

• Independent: Combines a prior and independent noisy readings using inverse variances.
• Correlation adjusted: Conditions a joint Gaussian distribution on both readings; see the separate mathematical reference.
• Bias correction: An option applied to consensus levels before either method.
• Dispersion/count weighting: Options for independent consensus precision.
• Backtests/predictions/combined: Output selections, not different estimators.

The main documentation uses BayesianPosterior, whose input is a prepared subject frame. It has different missing-model-precision semantics and uses underscore column names. The complete workflow uses hyphenated output names and the stdev_weighted_consensus argument.

Growth coordinates

Symbols follow the notation page. Let \(y_t\) be the reported actual and \(y_{t-1}\) the latest nonmissing actual with a period end strictly before \(t\). Every level \(v\) becomes growth \(v/y_{t-1}-1\): realised growth \(g_t\), model growth \(x_{M,t}\) from the model level \(m_t\), and consensus growth \(x_{C,t}\) from the consensus level \(c_t\). The posterior growth \(\hat\theta_t\) converts back to the published level \(\hat y_t\):

\[ g_t=\frac{y_t}{y_{t-1}}-1,\qquad x_{M,t}=\frac{m_t}{y_{t-1}}-1,\qquad x_{C,t}=\frac{c_t}{y_{t-1}}-1,\qquad \hat y_t=y_{t-1}(1+\hat\theta_t). \]

• Zero base: Growth is unavailable when \(y_{t-1}=0\).
• Realised growth: The prior’s historical sample is \(g_i\) over past periods \(i\).
• Missing observations: Only aligned nonmissing observations enter an error sample.
• Training start: start_date truncates levels before growth is calculated.
• History selection: Historical model and consensus observations are anchored before their actual’s first publication; later revisions cannot change them.

Prior and error precision

With \(n\) realised growth observations \(g_i\), the prior mean \(\mu_0\), prior variance \(\sigma_0^2\) and prior precision \(\tau_0\) are

\[ \mu_0=\frac1n\sum_i g_i,\qquad \sigma_0^2=\frac{1}{n-1}\sum_i(g_i-\mu_0)^2,\qquad \tau_0=\frac1{\max(\sigma_0^2,\varepsilon)},\quad \varepsilon=10^{-12}. \]

At least \(\max(2,\texttt{minimum\_observations})\) observations are needed for this sample variance. For the model and consensus readings, \(j\in\{M,C\}\), the variance \(\sigma_j^2\) is the mean squared error \(e_{j,i}\) against realised growth, not a centred sample variance, and the precision \(\tau_j\) is its inverse:

\[ e_{j,i}=|x_{j,i}-g_i|,\qquad \sigma_j^2=\frac1{n_j}\sum_i e_{j,i}^2,\qquad \tau_j=\frac1{\max(\sigma_j^2,\varepsilon)}. \]

use_absolute_errors=False instead uses

\[ e_{j,i}=\left|\frac{x_{j,i}-g_i}{|g_i|}\right|. \]

Zero realised growth is excluded from that relative-error sample. Each source needs minimum_observations errors; its default is 3 and its permitted minimum is 1. The prior still needs at least two realised growth observations.

• Backtests: Expanding prior and error moments are shifted by one observation. A target’s actual and error cannot affect its own posterior. Moments are aligned to the original observation indexes; missing historical precisions are not filled forward.
• Predictions: The complete realised training sample is fitted once and its moments are held fixed across the forward horizon.
• Combined: Each output column takes historical values strictly before that column’s first nonmissing forward value, then appends its forward values.

Independent posterior

The unknown growth \(\theta\) has a normal prior, and the model and consensus growth readings are independent normal observations of it (Gelman et al. 2013):

\[ \theta\sim N(\mu_0,\tau_0^{-1}),\qquad x_M\mid\theta\sim N(\theta,\tau_M^{-1}),\qquad x_C\mid\theta\sim N(\theta,\tau_C^{-1}). \]

The posterior growth \(\hat\theta\) is the precision-weighted mean. The total precision \(T\) sets the posterior standard deviation \(\sigma_\theta\), and each reading’s weight \(w_j\) is its share of \(T\):

\[ T=\tau_0+\tau_M+\tau_C,\qquad \hat\theta=\frac{\mu_0\tau_0+x_M\tau_M+x_C\tau_C}{T},\qquad \sigma_\theta=T^{-1/2},\qquad w_j=\frac{\tau_j}{T}. \]

Adding an independent precision reduces the computed variance under these assumptions. It does not guarantee better predictive accuracy with correlated errors or misspecified variances.

• Model excluded: include_model=False uses \(T=\tau_0+\tau_C\) and reports \(w_M=0\). Model growth and model precision cannot be requested.
• Missing historical model precision: An enabled model with unavailable precision prevents a historical posterior.
• Missing fixed model precision: The forecast update becomes prior plus consensus. Its diagnostics still report missing model precision, total precision, weights and interval. The prediction can therefore exist without an interval. This asymmetry is preserved and tested.
• Missing prior or consensus precision: No posterior is available.

Consensus dispersion and analyst count

The analyst dispersion \(d_t\) is a level. As growth it is \(\delta_t\), which the variance multiplier \(\kappa\) turns into the consensus variance \(\sigma_{C,t}^2\). With count weighting, \(k_t\) contributing analysts divide it by \(k_t^\alpha\) (Barron et al. 1998). Both \(d_t\) and \(k_t\) must be positive:

\[ \delta_t=\frac{d_t}{|y_{t-1}|},\qquad \sigma_{C,t}^2=\kappa\,\delta_t^2,\qquad \sigma_{C,t}^{2,\mathrm{count}}=\frac{\kappa\,\delta_t^2}{k_t^\alpha},\qquad \tau_{C,t}=\frac1{\max(\sigma_{C,t}^2,\varepsilon)}. \]

• Activation: stdev_weighted_consensus=True tries this precision first.
• Calibration: consensus_stdev_variance_multiplier is \(\kappa>0\), default 1.
• Count weighting: consensus_count_exponent is \(\alpha\), default unset.
• Fallback: Missing or nonpositive dispersion/count uses the historical consensus error precision, if available.
• Configuration: Count weighting and a nondefault multiplier require dispersion weighting. These options are unavailable for correlation adjusted.

Consensus bias correction

Bias correction happens in level space before growth. Each past period \(i\) gives a relative consensus error \(\rho_i\) against its actual:

\[ \rho_i=\frac{c_i-y_i}{y_i}. \]

A local-level Kalman filter (Kalman 1960; Durbin and Koopman 2012) tracks the persistent part of these errors, the consensus bias \(\beta\), in period order. The process variance \(Q\) lets the bias drift, the measurement variance \(R\) is the noise in each error, \(P_i\) is the variance of the bias estimate and \(K_i\) the gain:

\[ P_i^-=P_{i-1}+Q,\qquad K_i=\frac{P_i^-}{P_i^-+R},\qquad \beta_i=\beta_{i-1}+K_i(\rho_i-\beta_{i-1}),\qquad P_i=(1-K_i)P_i^-. \]

Target \(t\) reads the state after the last reported period strictly before it: \(\beta_t=\beta_{i^*}\) with \(i^*=\max\{i<t\}\).

• Initialization: The first finite observation sets \(\beta=\rho\) and \(P=R\).
• Missing error: State is held; variance is not advanced on missing observations.
• Defaults: consensus_bias_process_noise=1, consensus_bias_measurement_noise=0.5, consensus_bias_lambda=0.
• Adjustment: The strength \(\lambda\in[0,1]\) is the share of the bias removed: \(c_t^*=c_t/(1+\lambda\beta_t)\).
• Invalid scale: An unavailable or nonpositive scale retains the original consensus.
• Diagnostic: consensus-bias-pct is \(100\beta_t\), in percentage points.
• Zero strength: Requesting the bias diagnostic still runs the filter when \(\lambda=0\); consensus remains unchanged.
• Forecast rows: Every forecast row reads the state after the last reported period, so it includes the latest reported error. Historical rows read the state through the period before them.
• Earlier behaviour: Shifting errors before filtering, as the original signal did, drops the latest reported error from forecast rows and leaves them one quarter stale. Historical rows are the same under both rules.

Both methods use \(c^*\) for the consensus growth \(x_C\) and its error history. Neither reinterprets dispersion around the adjusted level.

Student-t intervals

The interval is a band of coverage \(p\) around the posterior growth \(\hat\theta\), scaled by the normal update’s standard deviation \(\sigma_\theta\). The quantiles \(q_L\) and \(q_U\) come from \(F\), a Student-t distribution with degrees of freedom \(\nu\), location \(\ell\) and scale \(\omega\):

\[ q_L=F^{-1}\!\left(\tfrac{1-p}{2}\right),\qquad q_U=F^{-1}\!\left(1-\tfrac{1-p}{2}\right),\qquad F=t_\nu(\ell,\omega), \]

\[ \theta_L=\hat\theta+\sigma_\theta q_L,\qquad \theta_U=\hat\theta+\sigma_\theta q_U. \]

• Coverage default: \(p=0.90\), set by interval_width.
• Shape: \(\nu\), \(\ell\) and \(\omega\) are interval_df, interval_loc and interval_scale, fitted per profile. Omitted, they come from the calibration for dispersion-weighted, bias-adjusted consensus at \(\lambda=0.5\): \(\nu=1.56\), \(\ell=0.084+0.340/\sqrt\kappa\), \(\omega=0.254+1.512/\sqrt\kappa\). Any other profile publishing an interval must give all three. See interval calibration.
• Growth bounds: posterior-growth-lower and posterior-growth-upper.
• Level bounds: Convert both with \(y_{t-1}(1+\theta)\), then sort them so a negative base cannot reverse prediction-lower and prediction-upper.
• Interpretation: These configured heavy-tailed intervals are not the conjugate Gaussian posterior’s ordinary credible interval (Mandelbrot 1963).

Empirical intervals

With interval_method="empirical", \(F\) is the empirical distribution of past standardised residuals \(z_i\) instead of a Student-t. Each residual is a past period’s realised growth \(g_i\) against its own posterior, in units of its own \(\sigma_{\theta,i}\):

\[ z_i=\frac{g_i-\hat\theta_i}{\sigma_{\theta,i}},\qquad F=\text{empirical distribution of }\{z_i\}. \]

The bounds \(\theta_L\) and \(\theta_U\) then follow from \(q_L\) and \(q_U\) as for the Student-t.

• Pool: Residuals from every entity and target, since one series rarely holds enough of them.
• Point in time: A row reads only residuals whose actual was known strictly before its own reading moment.
• Minimum: Fewer than empirical_interval_minimum knowable residuals publish no interval.

Directional confidence

directional-confidence is the stated probability that the outcome lands on the prediction’s side of raw consensus. The standardised consensus gap \(\gamma\) measures raw consensus growth \(x_C^{\mathrm{raw}}\), before bias correction, against the posterior:

\[ \gamma=\frac{x_C^{\mathrm{raw}}-\hat\theta}{\sigma_\theta}. \]

The confidence is \(F(\gamma)\) below consensus and \(1-F(\gamma)\) above, with the same \(F\) as the interval.

• Cap: directional_confidence_cap limits it where the research synthesis found it overstated: below consensus, raw consensus growth \(x_C^{\mathrm{raw}}\le0\), and at most directional_cap_horizon_days before publication. Without a publication schedule a row counts as zero days out.

Winsorised history

winsorize_fraction clips each tail of realised growth \(g_i\), and the upper tail of error magnitudes \(e_{j,i}\), before the prior moments \(\mu_0\), \(\sigma_0^2\) and the precisions \(\tau_j\). Each expanding window is clipped at its own quantiles, so a later observation never moves an earlier window’s bounds.

Reference output names

Output	Quantity
prediction	\(\hat y_t=y_{t-1}(1+\hat\theta)\)
prediction-lower	Lower sorted level bound, from \(\theta_L\) or \(\theta_U\)
prediction-upper	Upper sorted level bound, from \(\theta_L\) or \(\theta_U\)
model	\(m_t\)
consensus	\(c_t\)
posterior-growth	\(\hat\theta\)
posterior-growth-lower	\(\theta_L\)
posterior-growth-upper	\(\theta_U\)
model-growth	\(x_M\)
consensus-growth	\(x_C\), growth of \(c_t^*\)
actual-growth	\(g_t\)
prior-mean	\(\mu_0\)
prior-precision	\(\tau_0\)
model-precision	\(\tau_M\)
consensus-precision	\(\tau_C\)
total-precision	\(T\)
posterior-growth-stdev	\(\sigma_\theta=T^{-1/2}\)
prior-weight	\(w_0\)
model-weight	\(w_M\)
consensus-weight	\(w_C\)
consensus-stdev	\(d_t\)
consensus-stdev-growth	\(\delta_t=d_t/\lvert y_{t-1}\rvert\)
consensus-count	\(k_t\)
adjusted-consensus	\(c_t^*\)
consensus-bias-pct	\(100\beta_t\)
standardised-residual	\(z_t\)
standardised-consensus-gap	\(\gamma_t\)
directional-confidence	\(F(\gamma)\) or \(1-F(\gamma)\)

Growth outputs are fractions, not percentages. Only consensus-bias-pct uses percentage points. Raw levels and consensus-derived levels remain distinct from relative outputs; requesting a level may have licensing implications for the caller’s publication policy.

Publication-relative reads

• Fixed horizon: relative_to="publish", relative_days=h, with \(h\le0\). The pd alias is accepted. Omitting the horizon selects zero days.
• Latest horizon: relative_days="latest" derives the day offset between the latest live model revision and that period’s supplied publication date.
• Missing latest horizon: A company without a usable live prediction or publication date receives latest_horizon_unavailable; other companies continue.
• Availability gate: Target \(t\) requires \(\mathrm{publication}_t+h\ge \mathrm{publication}_{t-1}\); equality passes this gate.
• Reading boundary: Model and consensus values must be known strictly before the smaller of the explicit as_of and the horizon cutoff, using the package’s exclusive timestamp contract.
• Delayed history: Earlier actuals must also have been published by the reading cutoff; unavailable history refuses the row and its diagnostics.
• Supplied schedule: Pass subject keys and UTC publication_date in publications. Schedule provenance remains the caller’s responsibility.

Executable example

import pandas as pd
import tutorial_support as examples
import postforecast as pf

settings = pf.BayesianPredictionSettings(
    version="independent-example",
    method="independent",
    series="predictions",
    values=("prediction", "posterior-growth", "prior-weight", "model-weight", "consensus-weight"),
)
result = pf.bayesian_kpi_predictions(
    examples.quarterly_history(),
    pd.Timestamp("2026-05-10", tz="UTC"),
    forecast_source="model",
    settings=settings,
)
result[["entity", "period", *settings.values, "eligible"]]

	entity	period	prediction	posterior-growth	prior-weight	model-weight	consensus-weight	eligible
0	ZS	2026-06	800.648824	0.049481	0.830506	0.094336	0.075158	True

See integration boundaries for what stays with the caller.

References

Barron, Orie E., Oliver Kim, Steve C. Lim, and Douglas E. Stevens. 1998. “Using Analysts’ Forecasts to Measure Properties of Analysts’ Information Environment.” The Accounting Review 73 (4): 421–33.

Durbin, James, and Siem Jan Koopman. 2012. Time Series Analysis by State Space Methods. 2nd ed. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780199641178.001.0001.

Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. Bayesian Data Analysis. 3rd ed. Chapman; Hall/CRC. https://doi.org/10.1201/b16018.

Kalman, R. E. 1960. “A New Approach to Linear Filtering and Prediction Problems.” Journal of Basic Engineering 82 (1): 35–45. https://doi.org/10.1115/1.3662552.

Mandelbrot, Benoit. 1963. “The Variation of Certain Speculative Prices.” The Journal of Business 36 (4): 394–419. https://doi.org/10.1086/294632.