Diagnostics¶
Three tools say what an expression does with your data before you trust its numbers.
Why a value is missing: explain_missing¶
explain_missing(signal, data, universe=None) traces each missing cell of an expression back to the node where it
began, and gives the reason (the reasons are listed in Missing Values):
from qex import qex, explain_missing
report = explain_missing(qex("ts_mean(rank(returns(1)), 10)"), data, universe=data['member'].notna())
report.summary(within=data['member'].notna()) # each reason's share of the missing member-days
report.on('2003-06-02') # that day's missing names, each with its reason and node
report.reason('2003-06-02', 'AAI') # one cell: 'short history: outside the universe on earlier days'
Evaluate it in the universe the signal is used in: pass universe (a mask, a field name or a universe expression),
or call it inside cross_section_universe. Explaining changes nothing: it reads the values the expression's own
evaluation produced.
For a stock with no group, report.reason(day, "H") gives 'no group, at group_demean(field('value'), 'sector')'.
explain_missing ¶
explain_missing(signal: Signal, data: Dict[str, DataFrame], universe: Optional[DataFrame] = None) -> MissingReport
The reason for every missing cell of signal's value on data (see the module's docstring). Evaluate it in the cross-sectional universe the signal is used in: pass universe, or call it inside cross_section_universe.
MissingReport ¶
Why each missing cell of a signal is missing. codes holds a reason (REASONS) per cell, 0 where there is a value; causes, for a short history, what made the earlier days missing (a reason, or 0 for the start of the data); origins, the node where it began (nodes[i] is its qex).
text
staticmethod
¶
A reason in words: a short history says what made the earlier days missing.
summary ¶
Each reason's share of the cells with no value (only the cells where within is true, if given), most common first. A short history is split by what made the earlier days missing.
on ¶
Every missing name on a date (or only names), with its reason and the node where it began.
reason ¶
Why one cell is missing, and where it began; "" if it has a value.
Adjusted data and look-ahead: qex.adjust.check¶
check(signal, fields, mode="refuse") follows each field's adjustment exponent through an expression (the rule is
in Adjusted Data and Look-Ahead):
from qex import qex
from qex.adjust import check
check(qex("rank(close())"), fields) # raises LookAhead, saying what to write instead
check(qex("rank(close())"), fields, mode="warn") # returns the problems and lets it run
fields is the data's declarations, {"close": {"adjustment": 1, "unit": "usd_per_share"}, ...}; a panel's
manifest carries them. A field without a declaration is not checked, and neither is anything computed from it.
mode="off" skips the check. alpha=False checks an expression that is not itself an alpha (a regressor, a
universe), whose value need not be comparable across stocks.
check ¶
check(signal: Signal, fields: Dict[str, dict], mode: str = 'refuse', alpha: bool = True) -> List[dict]
The look-ahead and unit problems in an expression: [{kind, part, message}]. Raises LookAhead on the first one when mode is 'refuse'.
alpha: the value itself is compared across stocks (an alpha's book ranks it), so it must not be an adjusted level.
Numbers and labels: qex.kinds¶
qex() already refuses an expression that uses labels where numbers are needed. qex.kinds.numbers_only(signal)
says why an expression can't be an alpha (it gives labels), or returns None when it can.