sklearn.covariancepartial_fit contract.
Where sklearn.covariance estimators are batch-only, every estimator here
updates one observation at a time and exposes covariance_,
correlation_, precision_ and location_. On top of
the estimators it adds a registry, river-style adapters for variables that enter and leave
a universe, a panel of assessors for judging an estimate, a recommender that
picks an estimator from observable data features, and the Schur pseudo-likelihood
— a one-parameter bridge between the full and block-diagonal Gaussian likelihoods.
pip install precise
numpy is the only required dependency (tested on numpy 1.26 and
2.x, Python 3.9–3.13). Optional extras: [pandas] (DataFrame output for
the keyed adapters), [research], [dev].
import numpy as np
from precise import EwaCovariance
est = EwaCovariance(r=0.05) # exponentially weighted, online
for y in stream_of_vectors(): # y is a 1-D array, one observation
est.partial_fit(y)
est.covariance_ # (d, d) ndarray, always symmetric PSD
est.correlation_ # unit-diagonal correlation
est.precision_ # inverse (when well-conditioned)
est.location_ # running mean
Every estimator follows the same contract, so a test harness (or your code) only needs
to look for .fit / .partial_fit. Discover them programmatically:
from precise import all_estimators, estimator_from_name
all_estimators() # list of estimator classes
estimator_from_name("LedoitWolfCovariance")
Twenty truly-online estimators across families (no buffered recompute). Those marked experimental are new methods, not yet peer-reviewed or validated outside this package; their behaviour may change between minor releases.
| Class | Family | Notes |
|---|---|---|
EmpiricalCovariance | sample | Welford running covariance |
DiagonalCovariance | sample | variances only |
BlockCovariance | sample | block-diagonal, O(p·b) state for very large p |
EwaCovariance | weighted | exponentially weighted |
AdaptiveEwaCovariance | weighted | surprise-smoothed forgetting |
LedoitWolfCovariance | shrinkage | online linear shrinkage |
OASCovariance | shrinkage | oracle approximating shrinkage |
ShrunkCovariance | shrinkage | fixed-intensity target |
NonlinearShrinkageCovariance | shrinkage | analytical nonlinear spectrum shrinkage |
WindowedNonlinearShrinkageCovariance | shrinkage | nonlinear spectrum shrinkage over a rolling window |
EwaNonlinearShrinkageCovariance | shrinkage | nonlinear spectrum shrinkage, exponentially weighted |
SchurCovariance | shrinkage | experimental — cross-block coupling damping (linear/geodesic) |
SchurLedoitWolfCovariance | shrinkage | experimental — cross-block damping set by a Ledoit–Wolf reliability estimate |
SchurConditionalCovariance | shrinkage | experimental — damped, ridge-regressed block conditionals |
PartialMomentsCovariance | semi | downside / semi-covariance |
HuberCovariance | robust | online Huber M-estimator |
TylerCovariance | robust | recursive Tyler M-estimator |
GeodesicEwaCovariance | geometric | affine-invariant SPD geodesic step |
DCCCovariance | dynamic | decoupled vol & correlation |
FactorCovariance | factor | low-rank + diagonal, O(d·k)/step |
Finance rarely offers a fixed set of variables. The keyed adapters wrap any positional estimator into a river-style one over named variables that enter and leave:
from precise import keyed, EwaCovariance
est = keyed(EwaCovariance(r=0.05))
est.partial_fit({"AAPL": 0.01, "MSFT": -0.02}) # dict in, dict-of-dicts out
est.covariance_["AAPL"]["MSFT"]
FixedUniverse imputes missing keys;
DynamicUniverse handles staleness/longevity eviction and pairwise assembly
across changing key sets.
Because no estimator dominates every regime, precise treats
choosing and judging an estimate as first-class:
from precise import all_assessors, suggest
all_assessors() # LogLikelihood, BlockPseudoLikelihood,
# SchurLikelihood, SteinLoss, ... (higher = better)
suggest(X, top=3) # recommend estimator classes from data features
suggest maps observable, truth-free features (p/n, effective rank,
sphericity, condition number, off-diagonal mass, excess kurtosis) to an estimator via a
frozen, numpy-only decision tree. It is experimental: the tree is
trained on simulated data in this repository and has not been validated outside it, and
its own benchmarks find that choosing per data set is close to, and sometimes worse than,
always using one good estimator, except when the number of variables approaches the
number of observations. The SchurLikelihood assessor is experimental too
(see below); the other assessors are standard scores.
Experimental. This is a new method, introduced with this package and described in an unrefereed working paper. It has not been peer-reviewed or validated elsewhere.
The full Gaussian likelihood factorizes exactly, through Schur complements, into
block-conditional terms. A single coupling-strength parameter
γ ∈ [0,1] damps that conditioning, giving the Schur
pseudo-likelihood ℓγ: γ=1 is the full
likelihood (fragile in high dimensions), γ=0 is the block-diagonal /
composite likelihood (robust), and the interior is the bridge they lack. The same
γ damping of the same Schur complement is Schur complementary portfolio
allocation — interpolating hierarchical risk parity and minimum variance.
For the estimator/predictive use, the optimal trust has a closed form — the reliability of the coupling (a Wiener / James–Stein shrinkage):
γ* = (n−2) ρ² / [ (n−2) ρ² + (1 − ρ²) ] (ρ² = block coupling, the conditional R²)
rising to 1 as the sample size or coupling grows, and to 0 as the coupling becomes noise. See the working paper, Schur Covariance Evaluation: A Principled Pseudo-Likelihood in High Dimensions (read online · PDF).
pip install preciseOnline estimation does a constant amount of work per observation instead of recomputing
over a growing window — orders of magnitude less compute, and energy, over a long stream.
As a bonus, the partial_fit contract makes lookahead bias structurally hard: an
estimator only ever sees data up to the current observation, never the future. Drop the
precise skill into your own agent:
Read https://raw.githubusercontent.com/microprediction/precise/main/SKILL.md
and create a project skill from it.