Cross-Sectional¶
Across the stocks on each day, one day at a time. Each compares the stocks in the universe that day and gives every other stock no value (Universes). A stock with no value takes no part: on Jan 04 below, the day has three stocks, not four.
Ranking and scaling¶
rank¶
rank(x)
Each stock's percentile among the day's stocks with a value, from 1/n (the lowest) to 1 (the highest); ties share
their average rank. An infinity ranks first or last. With \(n\) stocks:
returns(1)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.100 | −0.050 | 0.017 | 0.050 |
| Jan 04 | 0.091 | −0.053 | · | −0.024 |
| Jan 05 | −0.083 | 0.028 | 0.016 | 0.049 |
| Jan 08 | 0.182 | −0.081 | 0.032 | 0.023 |
| Jan 09 | 0.077 | −0.059 | −0.016 | 0.045 |
rank(returns(1))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 1.00 | 0.25 | 0.50 | 0.75 |
| Jan 04 | 1.00 | 0.33 | · | 0.67 |
| Jan 05 | 0.25 | 0.75 | 0.50 | 1.00 |
| Jan 08 | 1.00 | 0.25 | 0.75 | 0.50 |
| Jan 09 | 1.00 | 0.25 | 0.50 | 0.75 |
rank(returns(20)) # higher return, higher rank
rank(-returns(5)) # higher return, lower rank: a reversal
zscore¶
zscore(x)
The distance from the day's mean in standard deviations. No value on a day the stocks don't vary.
returns(1)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.100 | −0.050 | 0.017 | 0.050 |
| Jan 04 | 0.091 | −0.053 | · | −0.024 |
| Jan 05 | −0.083 | 0.028 | 0.016 | 0.049 |
| Jan 08 | 0.182 | −0.081 | 0.032 | 0.023 |
| Jan 09 | 0.077 | −0.059 | −0.016 | 0.045 |
zscore(returns(1))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 1.13 | −1.26 | −0.20 | 0.33 |
| Jan 04 | 1.13 | −0.76 | · | −0.38 |
| Jan 05 | −1.46 | 0.43 | 0.24 | 0.79 |
| Jan 08 | 1.32 | −1.11 | −0.06 | −0.15 |
| Jan 09 | 1.07 | −1.16 | −0.45 | 0.55 |
gaussrank¶
gaussrank(x)
The day's rank mapped through the inverse of the normal distribution: a z-score's shape without its outliers. With \(r_i\) the stock's rank from 1 to \(n\) among the day's stocks with a value,
returns(1)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.100 | −0.050 | 0.017 | 0.050 |
| Jan 04 | 0.091 | −0.053 | · | −0.024 |
| Jan 05 | −0.083 | 0.028 | 0.016 | 0.049 |
| Jan 08 | 0.182 | −0.081 | 0.032 | 0.023 |
| Jan 09 | 0.077 | −0.059 | −0.016 | 0.045 |
gaussrank(returns(1))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 1.15 | −1.15 | −0.32 | 0.32 |
| Jan 04 | 0.97 | −0.97 | · | 0.00 |
| Jan 05 | −1.15 | 0.32 | −0.32 | 1.15 |
| Jan 08 | 1.15 | −1.15 | 0.32 | −0.32 |
| Jan 09 | 1.15 | −1.15 | −0.32 | 0.32 |
With four stocks the values are always \(\Phi^{-1}\) of 1/8, 3/8, 5/8 and 7/8: ±1.150 and ±0.319. On Jan 04, with three, they are ±0.967 and 0.
How it compares with its neighbors:
rank(x) |
zscore(x) |
gaussrank(x) |
|
|---|---|---|---|
| Shape | uniform, from 1/n to 1 | whatever x's shape is | normal, whatever x's shape is |
| One extreme stock | takes the top rank, no more | can dominate the day | takes the top score, no more |
| Uses x's values | only their order | their sizes | only their order |
| Sums to zero | no (centered on about ½) | yes | yes |
So gaussrank keeps rank's robustness (only the order matters, so any increasing transform of x gives the same
result) and gives values on zscore's scale, with the weight in the tails a normal sample has: the extremes count
more than in rank, and no single outlier counts more than a normal sample's extreme would. That makes it the natural
way to put several signals on one scale before combining them:
Tied stocks share the average of the scores they occupy, so every day still sums to zero. An infinity ranks first or
last, as in rank.
demean¶
demean(x)
x minus the day's mean: the result sums to zero, the weights of a dollar-neutral book.
rank(returns(1))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 1.00 | 0.25 | 0.50 | 0.75 |
| Jan 04 | 1.00 | 0.33 | · | 0.67 |
| Jan 05 | 0.25 | 0.75 | 0.50 | 1.00 |
| Jan 08 | 1.00 | 0.25 | 0.75 | 0.50 |
| Jan 09 | 1.00 | 0.25 | 0.50 | 0.75 |
demean(rank(returns(1)))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.375 | −0.375 | −0.125 | 0.125 |
| Jan 04 | 0.333 | −0.333 | · | 0.000 |
| Jan 05 | −0.375 | 0.125 | −0.125 | 0.375 |
| Jan 08 | 0.375 | −0.375 | 0.125 | −0.125 |
| Jan 09 | 0.375 | −0.375 | −0.125 | 0.125 |
scale¶
scale(x)
x divided by the day's sum of absolute values, so the absolute values sum to 1: weights with a gross of one,
Below, a one-day reversal book: long yesterday's losers, short its winners, half the gross on each side.
demean(-returns(1))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | −0.071 | 0.079 | 0.013 | −0.021 |
| Jan 04 | −0.086 | 0.057 | · | 0.029 |
| Jan 05 | 0.086 | −0.025 | −0.014 | −0.046 |
| Jan 08 | −0.143 | 0.120 | 0.007 | 0.016 |
| Jan 09 | −0.065 | 0.071 | 0.028 | −0.033 |
scale(demean(-returns(1)))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | −0.386 | 0.432 | 0.068 | −0.114 |
| Jan 04 | −0.500 | 0.334 | · | 0.166 |
| Jan 05 | 0.500 | −0.148 | −0.082 | −0.270 |
| Jan 08 | −0.500 | 0.421 | 0.024 | 0.055 |
| Jan 09 | −0.330 | 0.360 | 0.140 | −0.170 |
quantile¶
quantile(x, n)
The bucket each stock falls in, 1 (lowest) to n, by percentile rank. Buckets are numbers, and also labels: a
quantile can be the groups of a group operator.
returns(1)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.100 | −0.050 | 0.017 | 0.050 |
| Jan 04 | 0.091 | −0.053 | · | −0.024 |
| Jan 05 | −0.083 | 0.028 | 0.016 | 0.049 |
| Jan 08 | 0.182 | −0.081 | 0.032 | 0.023 |
| Jan 09 | 0.077 | −0.059 | −0.016 | 0.045 |
quantile(returns(1), 2)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 2 | 1 | 1 | 2 |
| Jan 04 | 2 | 1 | · | 2 |
| Jan 05 | 1 | 2 | 1 | 2 |
| Jan 08 | 2 | 1 | 2 | 1 |
| Jan 09 | 2 | 1 | 1 | 2 |
Clipping¶
winsorize¶
winsorize(x, limit)
Clips each day's values at that day's limit and 1 − limit quantiles: winsorize(x, 0.05) caps at the 5th and 95th
percentiles. Below, with four stocks and limit 0.25, each day's highest and lowest returns are pulled in to the
25th and 75th percentiles: A's 18% on Jan 08 no longer dominates the day.
returns(1)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.100 | −0.050 | 0.017 | 0.050 |
| Jan 04 | 0.091 | −0.053 | · | −0.024 |
| Jan 05 | −0.083 | 0.028 | 0.016 | 0.049 |
| Jan 08 | 0.182 | −0.081 | 0.032 | 0.023 |
| Jan 09 | 0.077 | −0.059 | −0.016 | 0.045 |
winsorize(returns(1), 0.25)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.063 | 0.000 | 0.017 | 0.050 |
| Jan 04 | 0.034 | −0.038 | · | −0.024 |
| Jan 05 | −0.009 | 0.028 | 0.016 | 0.033 |
| Jan 08 | 0.070 | −0.003 | 0.032 | 0.023 |
| Jan 09 | 0.053 | −0.026 | −0.016 | 0.045 |
truncate¶
truncate(x, max)
Clips each value to [−max, max]. Element by element, not across stocks: the limit is a number, not a percentile.
Its use is a position cap: below, the reversal book from scale with no position above 30% of the gross.
After the cap the gross is under one, and a book built from the signal scales it back.
scale(demean(-returns(1)))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | −0.386 | 0.432 | 0.068 | −0.114 |
| Jan 04 | −0.500 | 0.334 | · | 0.166 |
| Jan 05 | 0.500 | −0.148 | −0.082 | −0.270 |
| Jan 08 | −0.500 | 0.421 | 0.024 | 0.055 |
| Jan 09 | −0.330 | 0.360 | 0.140 | −0.170 |
truncate(scale(demean(-returns(1))), 0.3)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | −0.300 | 0.300 | 0.068 | −0.114 |
| Jan 04 | −0.300 | 0.300 | · | 0.166 |
| Jan 05 | 0.300 | −0.148 | −0.082 | −0.270 |
| Jan 08 | −0.300 | 0.300 | 0.024 | 0.055 |
| Jan 09 | −0.300 | 0.300 | 0.140 | −0.170 |
Neutralizing¶
regression_neut¶
regression_neut(y, x1, x2, ...)
A regression across stocks, not through time
regression_neut fits one regression per day, across the stocks: its data points are that day's stocks, and
each x is a value per stock, such as each stock's beta or size. It is not a regression through time; for one
stock's regression on the market over its history, use ts_beta,
ts_alpha and ts_resid.
Two kinds of regression sets them side by side.
Takes out of y, each day, the part the regressors explain. Across the day's stocks with a value for all of them, it
fits
by least squares, and gives the residual \(\hat\varepsilon_i = y_i - \hat a - \sum_j \hat b_j x_{ij}\). Least squares makes the residual orthogonal to the intercept and to every regressor:
so it has zero mean and zero covariance with each regressor, which is what "neutral to x" means. With one regressor it is
Below, each stock's market beta is taken out of the day's returns: what is left is the part of each return that beta doesn't explain. On Jan 03, A was up 10%, but with a beta of 1.4 almost all of that lines up with beta, and 0.9% is left; D, up 5% with a beta of 0.8, has the largest return beta doesn't explain.
returns(1)| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.100 | −0.050 | 0.017 | 0.050 |
| Jan 04 | 0.091 | −0.053 | · | −0.024 |
| Jan 05 | −0.083 | 0.028 | 0.016 | 0.049 |
| Jan 08 | 0.182 | −0.081 | 0.032 | 0.023 |
| Jan 09 | 0.077 | −0.059 | −0.016 | 0.045 |
field('beta')| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | 1.4 | 0.7 | 1.1 | 0.8 |
| Jan 03 | 1.4 | 0.7 | 1.1 | 0.8 |
| Jan 04 | 1.4 | 0.7 | 1.1 | 0.8 |
| Jan 05 | 1.4 | 0.7 | 1.1 | 0.8 |
| Jan 08 | 1.4 | 0.7 | 1.1 | 0.8 |
| Jan 09 | 1.4 | 0.7 | 1.1 | 0.8 |
regression_neut(returns(1), field('beta'))| A | B | C | D | |
|---|---|---|---|---|
| Jan 02 | · | · | · | · |
| Jan 03 | 0.009 | −0.033 | −0.028 | 0.052 |
| Jan 04 | −0.001 | −0.004 | · | 0.005 |
| Jan 05 | −0.019 | −0.024 | 0.031 | 0.013 |
| Jan 08 | 0.015 | −0.025 | −0.039 | 0.048 |
| Jan 09 | 0.015 | −0.033 | −0.040 | 0.059 |
With several regressors the fit is joint, and the result is neutral to each. That is not the same as nesting,
regression_neut(regression_neut(y, beta), size): when the regressors are correlated, the second step brings back
some of what the first took out. Regressors that duplicate each other are allowed. A day needs at least k + 2 stocks
for k regressors, or it has no value; a day a regressor doesn't vary gives y demeaned.
let beta = ts_beta_shrunk(returns(1), returns(1, 'market'), 252),
size = log(field('mcap'))
in regression_neut(zscore(-returns(5)), beta, size)
The signal is exactly neutral; a book built from it is close to neutral, since a book's caps and scaling come after.
regression_alpha¶
regression_alpha(y, x1, ..., xk)
The intercept \(\hat a\) of regression_neut's fit, the same regression across the day's stocks:
One number per day, the same for every stock. The intercept belongs to the day's cross-section, not to a stock, so every stock gets it, in the universe or not, and its history is the same for all. On a day with too few stocks for a fit (fewer than k + 2), every stock has no value.
With today's returns as y and characteristics known at yesterday's close as the regressors, it is the return of a
stock with none of the characteristics: with betas, the zero-beta portfolio's return.
let b = delay(ts_beta_shrunk(returns(1), returns(1, 'market'), 252), 1)
in ts_mean(regression_alpha(returns(1), b), 252) # the zero-beta portfolio's average daily return
regression_beta¶
regression_beta(y, x1, ..., xk, j)
The j-th slope \(\hat b_j\) of the same fit (j from 1 to k; with one regressor it may be left out). Like the
intercept, one number per day for every stock. With today's returns on characteristics known at yesterday's close,
it is the day's return to the j-th characteristic, net of the others; its average over time is the
characteristic's premium (Fama and MacBeth, 1973).
A day's slope is the same for every stock, so on its own it is not a signal: a book demeans it away. Signals multiply a characteristic by something learned from its premium's history:
let z = zscore(delay(field('book_to_price'), 1)),
prem = regression_beta(returns(1), z, 1)
in sign(ts_mean(prem, 126)) * z # factor momentum: lean into value while it pays
Lag the characteristics (delay(..., 1)): a characteristic's return is what a portfolio formed before the day
earned during it. Regressed on the same day's characteristics, the slope mostly measures how they move with today's
price (today's book-to-price falls when today's price rises), not what they earned. The lag is about what is
measured; the engine already rules out look-ahead. Factor Premia and Timing
builds factor momentum, volatility-managed blends, Fama-MacBeth weights and beta timing from these.
regression_alpha and every regression_beta on the same y and regressors read one internal fit node, computed
once. regression_neut on the same inputs fits the same regression on its own, and its residual is exactly
\(y - \hat a - \sum_j \hat b_j x_j\).
pca_factor¶
pca_factor(x, window, k[, refit])
Each stock's beta to the k-th statistical factor of x (usually returns(1)): the common factors are found from the
data alone, by a principal component analysis across stocks.
- Point in time. Every
refitdays (default 21; the first oncewindowdays exist), the analysis uses the lastwindowdays ofx, ending that day, and holds until the next refit. Before the first, no value. - Which stocks. Those with values on at least 90% of the window, and in the universe that day.
- Correlation, not covariance. Each stock is scaled by its own volatility first, so a few volatile stocks don't make up the first factors.
- A beta. The value is the stock's beta to the factor's return, in
x's units: the exposure a position carries, and whatregression_neutshould take out (below). - Factors keep their identity. Each refit's factors are matched to the last refit's and signed to agree, so factor
2 stays factor 2. The first factor is the market, signed long.
kis at most 20. - Shared.
pca_factorwith the samex,windowandrefitand differentkshare one decomposition.
The math. At a refit, take the window's \(T\) days of \(x\) for the \(N\) stocks in the fit, and standardize each stock by its own mean and volatility over the window (a missing day counts as the stock's mean):
The correlation matrix and its eigenvectors, largest eigenvalue first, are
The k-th factor's return is the eigenportfolio \(f_{kt} = \sum_i v_{ik} \, z_{ti}\), with variance \(\lambda_k\). Since \(\mathrm{cov}(z_i, f_k) = (C v_k)_i = \lambda_k v_{ik}\), each stock's beta to it, in \(x\)'s units, is simply
let r = returns(1)
in regression_neut(zscore(-returns(5)), pca_factor(r, 252, 1), pca_factor(r, 252, 2), pca_factor(r, 252, 3))
Infinities¶
zscore, demean, winsorize, scale and regression_neut leave an infinite value out of the day's statistics and
give that stock no value, so one division by zero (a zero-volume day in volume(20) / volume(1)) can't empty the
day. rank and quantile rank an infinity first or last.