Backtesting¶
Evaluating signal performance.
Basic Backtest¶
from qex import qex, Backtest
signal = qex("demean(rank(returns(20)))")
bt = Backtest(signal)
result = bt.run(data)
print(result.summary())
With Transaction Costs¶
transaction_cost is charged on every dollar traded: each day's P&L is reduced by
turnover * transaction_cost, where turnover is sum(|w_t - w_{t-1}|).
# 10 basis points per dollar traded
bt = Backtest(signal, transaction_cost=0.001)
result = bt.run(data)
Book Settings (Finding Alphas)¶
Backtest turns a signal into weights with signal.to_weights() by default: the signal
scaled so absolute weights sum to one, traded at the same close it was computed on. Pass
settings to build the book the way Finding Alphas (and WorldQuant BRAIN) do instead:
from qex import qex, Backtest, BookSettings
settings = BookSettings(
delay=1, # day t's weights use data up to day t-1's close
universe="member", # a data field, 1 for names in the universe that day, NaN otherwise
neutralize="sector", # "market", a group field, or None
truncation=0.05, # no name above 5% of the book
)
result = Backtest(qex("-returns(5)"), settings=settings).run(data)
print(result.fitness, result.margin_bps)
Each day the weights are neutralized, scaled to a book of exactly one (gross), and clipped at the truncation, repeated until all three hold. The size of the book never changes, so a signal's magnitude only moves weight between names; it cannot time the book.
The universe inside the signal¶
With a universe set, the signal's cross-sectional operators (rank, demean, zscore,
regression_neut, the group operators, ...) compare only the names in the universe that
day, as BRAIN does, and give NaN outside it. Time-series operators still see every name's
full history, so a name that joined today has its past returns and beta. See
the cross-sectional universe.
BookSettings(mask_cross_section=False) applies the universe only to the finished signal
instead, so rank(x) ranks a member among every name with data. That reproduces books from
qex before 0.5.
Market beta¶
Dollar-neutral is not beta-neutral. A signal correlated with beta (low volatility, size, many value measures) gives a book with a market exposure, and over a rising market its Sharpe can come from the market alone. Regress beta out before the book is built:
data["market"] = spy_close # a date-indexed Series
signal = qex("""
let b = ts_beta_shrunk(returns(1), returns(1, 'market'), 252)
in regression_neut(rank(-returns(5)), b)
""")
w = build_book(signal, data, BookSettings(delay=1, universe="member", truncation=0.05))
The book is then close to beta-neutral, not exactly: betas are estimated, and truncation comes after the regression. Check the realized correlation of the book's P&L with the market. Market Beta covers estimating, shrinking and removing beta.
Weights and statistics on their own¶
from qex import build_book, hold_untradable, forward_returns, turnover, fitness, margin_bps
w = build_book(signal, data, settings) # dates x names: the target book
w = hold_untradable(w, data["close"]) # what Backtest holds (see below)
pnl = (w * forward_returns(data["close"]).reindex_like(w).fillna(0)).sum(axis=1)
to = turnover(w) # fraction of the book traded each day
fitness(pnl, to) # Sharpe * sqrt(|annual return| / max(turnover, 0.125))
margin_bps(pnl, to) # P&L per dollar traded, in basis points
Missing prices¶
A stock with no price is held, not sold, and the move across a halt is booked when it trades again: see Halts, Delistings and Gaps.
Understanding Results¶
result = bt.run(data)
# Summary statistics
print(f"Total Return: {result.total_return:.1%}")
print(f"Sharpe: {result.sharpe:.2f}")
print(f"Max Drawdown: {result.max_drawdown:.1%}")
print(f"Avg Turnover: {result.avg_turnover:.1%}")
# Time series
result.returns # Daily returns (P&L / gross)
result.pnl # Daily P&L
result.cumulative_pnl # Cumulative P&L
result.weights # Daily position weights
result.drawdowns # The five deepest drawdowns
Date Filtering¶
Walk-Forward Analysis¶
Score consecutive out-of-sample windows. Nothing is fitted: the first train_period days
are skipped (as warm-up), then the signal is scored on each block of test_period days.
results = bt.run_walk_forward(
data,
train_period=252, # days skipped before the first window
test_period=63, # days per window
)
print(results) # DataFrame: start, end, sharpe, return, volatility, max_drawdown
Quantile Analysis¶
Understand signal predictive power:
from qex import QuantileAnalysis
qa = QuantileAnalysis(signal, n_quantiles=5)
result = qa.run(data)
print(result.summary())
The summary lists, for each quantile, its annualized return, Sharpe and hit rate; then the long-short spread (top minus bottom quantile), the rank IC, and whether returns are monotonic in the quantile. On 50 random-walk prices, where there is nothing to find:
Returns by Quantile (annualized):
Q1: 5.28% Sharpe: 0.52 Hit: 50.2%
Q2: 8.51% Sharpe: 0.82 Hit: 53.2%
Q3: 2.20% Sharpe: 0.21 Hit: 49.5%
Q4: 4.80% Sharpe: 0.49 Hit: 52.4%
Q5: -0.99% Sharpe: -0.10 Hit: 47.8%
Long-Short Spread (Q5 - Q1):
Annual Return: -6.27%
Sharpe Ratio: -0.43
Rank IC: -0.005
Monotonic: No
QuantileAnalysis(signal, n_quantiles=5, holding_period=1) measures forward returns over
holding_period days.
IC Analysis¶
Information Coefficient (correlation of the signal with forward returns, across names, each day):
ic_summary returns a Series with the mean, standard deviation and IR (mean / std) of the
rank and Pearson IC, the fraction of days with a positive IC, and the average number of names.
Key metrics:
- Mean IC: Average predictive power (0.03+ is decent)
- IC IR: IC / Std(IC), consistency of prediction
- % Positive: How often the signal's ordering agrees with the next return's
Comparing Signals¶
signals = {
'momentum': qex("rank(returns(60)) - 0.5"),
'reversal': qex("rank(-returns(5)) - 0.5"),
'combined': qex("0.5*rank(returns(60)) + 0.5*rank(-returns(5)) - 0.5"),
}
for name, signal in signals.items():
result = Backtest(signal, transaction_cost=0.001).run(data)
print(f"{name}: Sharpe = {result.sharpe:.2f}, MaxDD = {result.max_drawdown:.1%}")
Signal Decay Analysis¶
How quickly does the signal's edge fade? Measure the spread over longer forward returns:
from qex import qex, QuantileAnalysis
signal = qex("rank(returns(20))")
for hold in [1, 5, 10, 20]:
result = QuantileAnalysis(signal, n_quantiles=5, holding_period=hold).run(data)
print(f"Hold {hold}d: Spread Sharpe = {result.spread_sharpe:.2f}, Rank IC = {result.rank_ic:.3f}")
Individual Metrics¶
from qex import (
sharpe_ratio,
max_drawdown,
top_drawdowns,
calmar_ratio,
sortino_ratio,
)
returns = result.returns
cumulative = (1 + returns).cumprod() # drawdowns are measured on cumulative returns
print(f"Sharpe: {sharpe_ratio(returns):.2f}")
print(f"Sortino: {sortino_ratio(returns):.2f}")
print(f"Calmar: {calmar_ratio(returns):.2f}")
print(f"Max DD: {max_drawdown(cumulative):.1%}")
# Top drawdowns with dates
for dd in top_drawdowns(cumulative, n=3):
print(f"{dd.depth:.1%} from {dd.start:%Y-%m-%d} to {dd.end:%Y-%m-%d}")
Tips¶
- Always use transaction costs - Real performance is lower
- Check turnover - High turnover + high TC = bad
- Use quantile analysis - Verify monotonic relationship
- Check market exposure - A dollar-neutral book can still be a market bet
- Score out of sample - Use walk-forward windows and a held-out period
- Check IC stability - Consistent > high average