Local (Hyvärinen) scoring rules

The log score and CRPS need a normalized density: you must know the partition function $Z$. A local proper scoring rule does not. It depends on the forecast only through the derivatives of $\log p$ at the realised outcome, and since $\log p = \log\tilde p - \log Z$, every derivative drops the constant. An energy-based or un-normalised model can therefore be scored directly, no partition function in sight.

What "m-local" means

A scoring rule is $m$-local if it depends on the quoted density only through its value and its first $m$ derivatives at the outcome $y$. Ordinary proper rules are $0$-local (they use $p(y)$ alone). Parry, Dawid & Lauritzen (2012) proved that genuinely local proper rules on the real line exist iff $m$ is even: there is no proper $1$-local rule, but there are proper rules at every even order, trading more derivative information for more modelling freedom while staying free of the normalizing constant.

The Hyvärinen score ($m = 2$)

The canonical local rule is the Hyvärinen score, in loss form (lower is better):

$S(y, p) = \Delta \log p(y) + \tfrac12\,\lVert \nabla \log p(y)\rVert^2.$

Its population minimiser is score matching (Hyvärinen, 2005), and the divergence it induces is the Fisher divergence

$D_F(p_{\text{data}}, p) = \tfrac12\!\int p_{\text{data}}\,\lVert \nabla\log p - \nabla\log p_{\text{data}}\rVert^2,$

non-negative and zero iff $p = p_{\text{data}}$, so the score is strictly proper relative to $D_F$. For a Gaussian $N(\mu,\sigma^2)$ it has the closed form $S = (y-\mu)^2/(2\sigma^4) - 1/\sigma^2$.

Why dropping $Z$ matters

In high-frequency forecasting, forcing every agent to compute a normalizing constant for its predictive density is a real tax, often the dominant cost for flexible models. A local rule lets a lightweight agent submit a complex, unnormalised density (a mixture or energy model written without its $Z$) and be scored faithfully. The trade-off: local rules reward getting the shape of $\log p$ right near the outcome and are correspondingly less sensitive to far-tail mass than the log score.

In this repo

mechanisms/local_scoring.py provides hyvarinen_score (from analytic $\nabla\log p$ and $\Delta\log p$), hyvarinen_score_fd (finite-difference, for any un-normalised log_pdf callable), gaussian_hyvarinen_score (closed form), and fisher_divergence_gaussian. The demo examples/sim_local_scoring.py ranks several un-normalised models against a true stream, recovering the right order, and confirms the scores are invariant to an arbitrary additive log-constant.

Try it

The Hyvärinen (local) score of a Gaussian quote $N(\mu,\sigma^2)$ at an outcome $y$: $S=(y-\mu)^2/2\sigma^4 - 1/\sigma^2$. It uses only derivatives of $\log p$, so it never needs the normalising constant (loss form, lower is better).

outcome y

quote μ

quote σ

Code: mechanisms/local_scoring.py · Demo: examples/sim_local_scoring.py · Related: scoring rules · Research: gaps-and-roadmap.md