Learnings that shaped this package
Everything here comes from building and then rearchitecting the buy-side consensus study. Each item cost something to learn; each one is a constraint this package carries.
About point-in-time data
- The as-of join is the whole discipline, and it is easy to get subtly wrong. In the study, a join de-duplicated on the event key alone and broadcast one formation day’s signal across the whole holding window. 580 of 586 daily returns changed, and every unit test still passed. Only a comparison against the original scripts caught it. One shared, heavily tested anchoring primitive is worth more than careful joins in ten places.
- Strictly before, not at or before. The event’s own timestamp is the common reading moment, and a value knowable at that instant is usually the value under study. The default must exclude it, with the inclusive case opt-in and named.
- Maximum age is part of the join, not a later filter. A forecast nobody has revised in a year is not a current view. The study caps a model at 21 days and a consensus at 60; those are research decisions, so they are arguments, never constants in the library.
- First reported, never restated. An actual that has been revised since the event is not what the market saw. Where a source revises history, take the first published value.
- Exchange-local timing, with the weekend case handled once. A report filed after the close reacts the next session; a report filed on a weekend must not be shifted twice.
About fitted parameters
- Look-ahead arrives through convenience. The legacy pre-report analysis winsorised the full sample before splitting it, so a test-period outlier moved a training-period bound. Nobody intended it; it was one line in the wrong order. Hence
fitandapplyas separate types in this library: the cheap path has to be the correct one. - A shipped parameter set is data with a version. The anchoring signal ships two weight sets, each with a version string and the date it was fitted through. Same inputs, different version, different published number, and the difference is explainable. Parameters fitted at call time cannot be defended a year later.
- Train-only bounds change results, and that is worth saying out loud. When the study fixed its winsorisation, the numbers moved. The fix shipped as a documented behavioural change with its own test, not folded into a refactor.
About degenerate input
Every one of these crashed the study before it was fixed at the root, and each is now a test:
- clustered estimates where the cluster label is missing;
- an empty training or test period;
- an empty portfolio or an empty panel, which must come back typed and empty, not untyped;
- a variance of zero in a fit;
- a zero denominator in a ratio, which must yield missing, never infinity.
Publish nothing rather than a fallback. A stale input and a real number must not become indistinguishable downstream. The anchoring signal returns nothing when a subject is not publishable, and the reason travels with it.
About declaring a contract instead of checking one
- A schema is read; a sequence of checks is skimmed. The panel format was a list of hand-written guards before it was a Pandera schema. Same rules, but the schema is the documentation, and it validates lazily: one call reports every way a panel is malformed rather than stopping at the first, so a caller fixing one gets the whole list.
- Keep the domain error at the boundary. Pandera’s report is good data and a poor error message. It is caught and re-raised as
PanelError, with each custom check carrying the sentence it wants printed. A caller never learns which validation library is underneath. - A shipped parameter set validates itself. Making it a frozen Pydantic model buys three things at once: mutation is refused, a non-finite coefficient never reaches a published number, and
model_dumpis the audit record, so there is no second hand-written serialiser to drift from it. - Staleness has to survive the read. Capping the age inside the anchoring join is right when you are choosing a value and wrong when a method has to say why it published nothing: the row vanishes, and “stale prediction” collapses into “missing input”. The cap moved into the method, and
age_atcarries the age through as a value.
About structure
- “Core” is not a layer name. The study’s calculation layer was called
core, so everything landed there: regressions, portfolios, strategies, robustness. Name packages for what they own. - Generic and specific separate cleanly, if you look. Three modules each held a reusable half and a study-specific half. Splitting them along that line is exactly what makes this package possible.
- Enforce the direction with a tool.
import-lintercatches what review does not. Purity is a contract, not a habit: a calculation package that cannot importpathlibcannot grow a file read. - One command, one calculation, no arithmetic at the edges. Guards and de-duplications that drift into a CLI are untested arithmetic.
About proving an extraction
- Compare against the original, on identical inputs. The study’s parity check extracts the pre-migration scripts from a fixed commit, runs both paths on synthetic data, and compares keys, missingness and values. Six comparisons,
rtol=1e-10. It found the as-of bug. Anything ported here is proven the same way. - A tutorial with a worked example is a test suite. Both reference implementations for this package end in a self-check with real numbers. Those become golden tests here, so the library and the published research cannot drift apart silently.
- Synthetic data proves the pipeline, not the findings. Never claim numerical parity with published results from a synthetic run.
About the name
postforecast, because the package acts after a forecast exists and before a decision. It borrows “statistical post-processing of forecasts”, which is standard language in weather and energy forecasting, and it sits beside forecast combination and forecast reconciliation.- Rejected
postcast: in the-castfamily the prefix says when the target is (fore, now, back, hind), so it reads as a forecast about the past, which is the exact hindsight misreading to avoid. It is also one character frompodcast. - Rejected
consensuskit,benchcast,asofstudy: each names one half of the job.asofstudystays available if the point-in-time panel machinery ever deserves a lower-level package of its own.
Prior art worth reading
Checked out under .repos/, read but never vendored:
- forecastlens — a diagnostic layer over
dartsandneuralforecastoutputs: regime-aware calibration, leak-safe backtesting, decision-value evaluation. Closest neighbour in position: it also acts on forecasts produced elsewhere. - metaforecast — meta-learning and data-centric forecasting; useful for how it structures families of methods behind one interface.
- statsforecast — the model-list and cross-validation shape worth copying for the evaluation harness.