Skip to content

Time Series

Each stock's own history, one stock at a time. period counts rows of the data (trading days), not calendar days.

Window rules

Every rolling operator follows one of three rules for how much history it needs and what a missing day does. The toy data shows all three on C, which has no price on Jan 04.

Rule Operators A value once the window holds A missing day in the window
Part of the window ts_mean, ts_std, ts_sum, ts_max, ts_min, ts_var, ts_quantile, the pair operators, volatility, volume, adv min(20, period) values is skipped, as long as enough values remain
The whole window ts_rank, ts_argmax, ts_argmin, ts_skew, ts_kurt, decay_linear period values hides the result until it has left the window
Exponential ewma, ewma_var, ewma_cov, ts_beta_ewma one value is skipped; the average carries on

With a window of 20 days or less, part of the window needs every day too, as in the 3-day panels below. The difference shows on longer windows: a 60-day mean needs 20 values and skips the gaps between them. How qex Evaluates explains why each rule is what it is.

Rolling statistics

ts_mean

ts_mean(x, period)

The mean of each stock's last period values. Part of the window.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
ts_mean(close(), 3)
ABCD
Jan 02····
Jan 03····
Jan 0411.0019.00·41.00
Jan 0511.3318.50·42.00
Jan 0812.0017.83·42.67
Jan 0912.6717.1731.5044.33
ts_mean(returns(1), 20)                    # average daily return over a month
close() / ts_mean(close(), 200) - 1        # distance from the 200-day average

ts_std

ts_std(x, period)

The standard deviation of the window (sample, n − 1). Part of the window.

ts_std(returns(1), 20)                     # daily volatility, not annualized

ts_var

ts_var(x, period)

The variance of the window, ts_std squared. Part of the window.

ts_sum

ts_sum(x, period)

The sum of the window. Part of the window.

ts_sum(log(1 + returns(1)), 20)            # the 20-day log return, from daily returns

ts_max

ts_max(x, period)

The largest value in the window. Part of the window.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
ts_max(close(), 3)
ABCD
Jan 02····
Jan 03····
Jan 0412.020.0·42.0
Jan 0512.019.0·43.0
Jan 0813.018.5·44.0
Jan 0914.018.532.046.0
close() / ts_max(close(), 252)             # how far below the 52-week high

ts_min

ts_min(x, period)

The smallest value in the window. Part of the window.

ts_quantile

ts_quantile(x, period, q[, min_periods])

The q-quantile of the window, q from 0 to 1, interpolating between the two nearest values: 0.5 is the median, 0 and 1 are ts_min and ts_max. Part of the window; min_periods changes the minimum.

-ts_quantile(returns(1), 252, 0.05)                                     # 5% value-at-risk, as a loss
ts_quantile(returns(1), 60, 0.75) - ts_quantile(returns(1), 60, 0.25)   # interquartile range

Lag and difference

delay

delay(x, period)

The value period rows ago; period ≥ 0. There is no negative delay: a value can only come from the past.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
delay(close(), 1)
ABCD
Jan 02····
Jan 0310.020.030.040.0
Jan 0411.019.030.542.0
Jan 0512.018.0·41.0
Jan 0811.018.531.043.0
Jan 0913.017.032.044.0
delay(returns(231), 21)                    # 12-month momentum, skipping the last month

delta

delta(x, period)

Today's value minus the value period rows ago: x - delay(x, period). Missing when either is.

delta(returns(20), 5)                      # how momentum changed over a week

Ranking and extremes

ts_rank

ts_rank(x, period)

Where today's value ranks among the stock's last period values, from 1/period (the lowest) to 1 (the highest); ties share their average rank. The whole window: after C's missing day there is no rank until Jan 09.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
ts_rank(close(), 3)
ABCD
Jan 02····
Jan 03····
Jan 041.000.33·0.67
Jan 050.500.67·1.00
Jan 081.000.33·1.00
Jan 091.000.330.671.00
ts_rank(volume(1), 20)                     # is today's volume unusual for this stock?

ts_argmax

ts_argmax(x, period)

How many days ago the window's largest value was, 0 if it is today; on a tie, the oldest. The whole window. For "days since the latest event", use days_since.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
ts_argmax(close(), 3)
ABCD
Jan 02····
Jan 03····
Jan 0402·1
Jan 0512·0
Jan 0801·0
Jan 090210
ts_argmax(close(), 252)                    # days since the 52-week high

ts_argmin

ts_argmin(x, period)

How many days ago the window's smallest value was. The whole window.

Pairs of series

Regressions through time, one stock at a time

The pair operators work down each stock's own history: ts_beta(y, x, 252) fits one regression per stock, on that stock's last 252 days, so every stock gets its own beta, alpha and residual. x is usually one series for every stock, such as the market's return. For a regression across the stocks on one day, use regression_neut; Two kinds of regression sets them side by side.

The pair operators use only the days on which both series have a value. min_periods (default min(20, period)) is the fewest such days a window needs: with the default, a stock 20 days after listing gets a "252-day" beta from 20 days. Ask for more when it matters.

ts_corr

ts_corr(x, y, period[, min_periods])

The correlation of x and y over the window. Part of the window. Over the window's \(n\) paired days,

\[ \rho_{xy} = \frac{\sum_t (x_t - \bar x)(y_t - \bar y)}{\sqrt{\sum_t (x_t - \bar x)^2 \, \sum_t (y_t - \bar y)^2}} \]
ts_corr(abs(returns(1)), volume(1), 60)    # do big moves come with volume?

ts_cov

ts_cov(x, y, period[, min_periods])

The covariance of x and y over the window. Part of the window.

\[ \mathrm{cov}(x, y) = \frac{1}{n - 1} \sum_t (x_t - \bar x)(y_t - \bar y) \]

ts_beta

ts_beta(y, x, period[, min_periods])

The least-squares slope of y on x over the window: the \(\beta\) of \(y_t = \alpha + \beta x_t + \varepsilon_t\), fitted on the paired days. Part of the window.

\[ \beta = \frac{\mathrm{cov}(y, x)}{\mathrm{var}(x)} = \frac{\sum_t (x_t - \bar x)(y_t - \bar y)}{\sum_t (x_t - \bar x)^2} \]
ts_beta(returns(1), returns(1, 'market'), 252)          # market beta
ts_beta(returns(1), returns(1, 'market'), 252, 126)     # no value until 126 days are paired

ts_beta_shrunk

ts_beta_shrunk(y, x, period[, min_periods][, groups])

ts_beta, pulled toward the day's average beta in proportion to its own noise (Vasicek 1973). For stock \(i\), with \(\beta_i\) its ts_beta from \(n_i\) paired days:

\[ \beta_i^{\text{shrunk}} = w_i \, \beta_i + (1 - w_i) \, \bar\beta, \qquad w_i = \frac{\tau^2}{\tau^2 + s_i^2} \]
  • \(\bar\beta\) and \(\tau^2\) are the mean and variance of the day's betas across the stocks in the universe: the prior.
  • \(s_i^2\) is the sampling variance of stock \(i\)'s own estimate, from its residual variance \(\hat\sigma_i^2\):
\[ s_i^2 = \frac{\hat\sigma_i^2}{(n_i - 1)\mathrm{var}(x)}, \qquad \hat\sigma_i^2 = \frac{n_i - 1}{n_i - 2}\left(\mathrm{var}(y_i) - \beta_i^2 \mathrm{var}(x)\right) \]

This is the posterior mean of \(\beta_i\) under a normal prior \(N(\bar\beta, \tau^2)\). A precise estimate (\(s_i^2\) small against \(\tau^2\)) keeps its own beta; a beta from a short or noisy history moves most of the way to the average. Market Beta discusses when it helps.

A prior for each group. With groups (a group's name or any labels expression), \(\bar\beta\) and \(\tau^2\) are the mean and variance of the betas in the stock's own group that day: a utility's beta is pulled toward the utilities', not toward the market's average. A stock with no group has no value. A group with fewer than two betas that day has no spread to shrink by, so its stocks keep their own betas.

How qex tells the two optional arguments apart: a number written as a number is min_periods, and anything else (a quoted name, a labels expression) is groups. So groups comes last, after min_periods when there is one, and min_periods is written out (126, not 252 / 2); qex refuses either mistake with a message that says which.

ts_beta_shrunk(returns(1), returns(1, 'market'), 252)                  # toward the day's average beta
ts_beta_shrunk(returns(1), returns(1, 'market'), 252, 'sector')        # toward each sector's
ts_beta_shrunk(returns(1), returns(1, 'market'), 252, 126, 'sector')   # and a value only from 126 paired days

ts_alpha

ts_alpha(y, x1, ..., xk, period[, min_periods])

The intercept of the same regression: the \(\alpha\) of \(y_t = \alpha + \beta x_t + \varepsilon_t\) over the window. A least-squares line goes through the two means, so over the window's paired days

\[ \alpha = \bar y - \beta \, \bar x \]

Both means are over the days both series have a value, the same days as the beta. (Writing it by hand with two ts_means gets this wrong whenever y has a gap in the window that x doesn't.) Part of the window, with ts_beta's minimum. For a stock's daily return on the market's it is the daily alpha, Jensen's alpha without the risk-free rate; times 252 for a year.

ts_alpha(returns(1), returns(1, 'market'), 252) * 252     # annualized

Several regressors. Give more than one x and it is one joint least-squares fit over the window's days where y and every x have a value:

\[ y_t = \alpha + \sum_{j=1}^{k} \beta_j \, x_{j,t} + \varepsilon_t \qquad\Longrightarrow\qquad \alpha = \bar y - \sum_{j=1}^{k} \beta_j \, \bar x_j \]

with the \(\beta_j\) solving the window's normal equations, \(\mathrm{cov}(x, x)\,\beta = \mathrm{cov}(x, y)\). The alpha is net of all the regressors at once, which is not the same as taking them out one at a time when they are correlated. A window needs at least \(k + 2\) such days (and min_periods); regressors that duplicate each other get the minimum-norm fit. A multi-factor alpha, with factor returns supplied once per day for every stock (Data Format):

ts_alpha(returns(1), field('mkt'), field('smb'), field('hml'), 252) * 252

The shrunk versions take one regressor: shrinking several betas at once needs a prior across them, which is a model of its own.

ts_alpha_shrunk

ts_alpha_shrunk(y, x, period[, min_periods][, groups])

The intercept that goes with ts_beta_shrunk's slope. For a fixed slope, the least-squares intercept is still the line through the means:

\[ \alpha^{\text{shrunk}} = \bar y - \beta^{\text{shrunk}} \, \bar x \]

It differs from ts_alpha by exactly what the shrinkage took off the beta, times the average of x:

\[ \alpha^{\text{shrunk}} - \alpha = (\beta - \beta^{\text{shrunk}}) \, \bar x \]

In a rising market, a low beta shrunk up toward the average lowers the stock's alpha: part of what looked like alpha was low beta. groups is as in ts_beta_shrunk.

ts_alpha_shrunk(returns(1), returns(1, 'market'), 252, 'sector') * 252

ts_resid

ts_resid(y, x1, ..., xk, period[, min_periods])

Today's residual from the window's fit, with \(\alpha\) and the \(\beta_j\) from the window ending today:

\[ \varepsilon_t = y_t - \alpha - \sum_{j=1}^{k} \beta_j \, x_{j,t} \]

The part of today's y that x doesn't explain: for a stock's return on the market's, its idiosyncratic return. No value on a day y or x is missing. The fit includes today, as any rolling regression's last residual does; it uses nothing after today.

ts_sum(ts_resid(returns(1), returns(1, 'market'), 252), 20)         # last month, beyond beta
ts_resid(returns(1), field('mkt'), field('smb'), field('hml'), 252)    # beyond three factors

ts_resid_shrunk

ts_resid_shrunk(y, x, period[, min_periods][, groups])

Today's residual from ts_beta_shrunk's fit: \(y_t - \alpha^{\text{shrunk}} - \beta^{\text{shrunk}} x_t\). groups is as in ts_beta_shrunk.

-ts_sum(ts_resid_shrunk(returns(1), returns(1, 'market'), 252, 'sector'), 5)   # idiosyncratic reversal

Exponentially weighted

A day's weight halves every halflife days, so the effective look-back is about three half-lives. There is no warm-up: a value from the first observation, carried through missing days. With half-life \(h\), a value \(k\) days old has weight

\[ w_k = 2^{-k/h} \]

ewma

ewma(x, halflife)

The exponentially weighted mean, over the days \(x\) has a value:

\[ m_t = \frac{\sum_{k \ge 0} w_k \, x_{t-k}}{\sum_{k \ge 0} w_k} \]

Below, C keeps its average through Jan 04.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
ewma(close(), 2)
ABCD
Jan 0210.0020.0030.0040.00
Jan 0310.5919.4130.2941.17
Jan 0411.2318.7730.2941.09
Jan 0511.1418.6730.6741.84
Jan 0811.8018.0731.2542.61
Jan 0912.5417.3831.3443.74
ewma(returns(1), 10)                       # a 10-day half-life average return

ewma_var

ewma_var(x, halflife)

The exponentially weighted variance.

sqrt(ewma_var(returns(1), 20) * 252)       # an annualized volatility that adapts quickly

ewma_cov

ewma_cov(x, y, halflife)

The exponentially weighted covariance, over the days both have a value.

ts_beta_ewma

ts_beta_ewma(y, x, halflife)

The exponentially weighted version of ts_beta, over the days both have a value:

\[ \beta_t = \frac{\mathrm{cov}_h(y, x)}{\mathrm{var}_h(x)} \]
ts_beta_ewma(returns(1), returns(1, 'market'), 63)     # a beta that adapts within a quarter

Shape of the window

decay_linear

decay_linear(x, period)

A weighted mean with weights period, period − 1, …, 1, the newest heaviest. The whole window. With period \(p\):

\[ \text{decay}_t = \frac{\sum_{k=0}^{p-1} (p - k)\, x_{t-k}}{p(p+1)/2} \]

On Jan 04, A is (3 × 12 + 2 × 11 + 1 × 10) / 6.

close()
ABCD
Jan 0210.020.030.040.0
Jan 0311.019.030.542.0
Jan 0412.018.0·41.0
Jan 0511.018.531.043.0
Jan 0813.017.032.044.0
Jan 0914.016.031.546.0
decay_linear(close(), 3)
ABCD
Jan 02····
Jan 03····
Jan 0411.3318.67·41.17
Jan 0511.3318.42·42.17
Jan 0812.1717.67·43.17
Jan 0913.1716.7531.5844.83
decay_linear(returns(1), 10)               # a recency-weighted 10-day average

ts_skew

ts_skew(x, period)

The skewness of the window. The whole window.

-ts_skew(returns(1), 60)                   # prefer stocks without lottery-like upside

ts_kurt

ts_kurt(x, period)

The excess kurtosis of the window (0 for a normal distribution). The whole window.