A concise explanation of stationary ARMA models

Time series analysis 3 Minutes

Motivation
Stationary autoregressive moving average (ARMA) models play an important role in financial forecasting (we referred to it here and here). My experience is that online explanations of stationarity often are either very long or implicit about certain steps. Therefore, this post presents a fairly short but hopefully complete explanation of when an ARMA model is stationary.

Stationary ARMA models
Consider the following basic linear filter model:

z_t = a_t + \psi_1a_{t-1} + \psi_2a_{t-2} + \psi_3a_{t-3} + ...

where z_t is the value of a time series at time t, where \psi_1, \psi_2, ... are weights and a_t, a_{t-1}, a_{t-2}, ... are random terms, at different times, drawn from a probability distribution with a fixed mean and variance (we don’t bother with the level of the time series, which would equal a constant on the right-hand-side of the expression). A times series is called stationary whenever \sum_{j=0}^{\infty} |\psi_j| < \infty.

Now an ARMA model i just a special case of the linear filter model. To see this, consider the following ARMA model:

z_t(1- \phi_1B - \phi_2B^2) = a_t(1 - \theta_1B - \theta_2B^2)

where \phi_1, \phi_2 and \theta_1, \theta_2 are weights of the the AR and MA parts of the model, respectively, and B is the back-shift operator (i.e. Bz_t = z_{t-1}, Ba_t = a_{t-1}). For this particular ARMA model z_t depends on the two most recent values of the series, z_{t-1}, z_{t-2}, as well as three random terms, a_t, a_{t-1}, a_{t-2}. By re-arranging the expression we get:

z_t = a_t(1 - \theta_1B - \theta_2B^2) / (1- \phi_1B - \phi_2B^2)

Now we can compare this with the linear filter model (and using the back-shift operator for the linear filter model as well):

a_t(1 - \theta_1B - \theta_2B^2) / (1- \phi_1B - \phi_2B^2) = a_t (1 + \psi_1B + \psi_2B^2 + \psi_3B^3 + ...)

which is equal to:

a_t(1 - \theta_1B - \theta_2B^2) = a_t(1 + \psi_1B + \psi_2B^2 + \psi_3B^3 + ...)(1- \phi_1B - \phi_2B^2)

If we cancel a_t on both sides and compare the weights in front of B, B^2, B^3, ..., respectively, we find that:

-\theta_1 = -\phi_1 + \psi_1 \iff \psi_1 = \phi_1 - \theta_1 (for B)

-\theta_2 = -\phi_2 - \phi_1\psi_1 + \psi_2 \iff \psi_2 = \phi_1\psi_1 + \phi_2 - \theta_2 (for B^2)

0 = -\phi_2\psi_1 - \phi_1\psi_2 + \psi_3 \iff \psi_3 = \phi_2\psi_1 + \phi_1 \psi_2 (for B^3)

0 = -\phi_2\psi_2 - \phi_1\psi_3 + \psi_4 \iff \psi_4 = \phi_2\psi_2 + \phi_1\psi_3 (for B^4)

…

0 = -\phi_2\psi_{j-2} - \phi_1\psi_{j-1} + \psi_j \iff \psi_j = \phi_2\psi_{j-2} + \phi_1\psi_{j-1} (for B^j)

For \psi_j with j \geq 3 we have a second-order linear homogeneous recurrence relation (with initial conditions given by finite values \psi_1, \psi_2). What we want to do is to find an closed expression for \psi_j. We follow the standard approach when solving recurrence relations, by setting \psi_j = cr^j, where c, r \neq 0:

0 = -\phi_2\psi_{j-2} - \phi_1\psi_{j-1} + \psi_j = -\phi_2cr^{j-2} - \phi_1cr^{j-1} + cr^j

We divide by cr^{j-2} and get:

0 = -\phi_2 - \phi_1r + r^2.

This is a simple quadratic equation. If we solve for r we either get two distinct (independent) real roots, two complex roots or one repeated root. If we have two distinct roots r_1, r_2 we add them for the following general solution (this is a standard technique when solving recurrance relations):

\psi_j = c_1r_1^j + c_2r_2^j

For one repeated root r we instead have the general solution:

\psi_j = r^j(c_1 + c_2n).

We could use the initial conditions to determine the values of the constants c_1, c_2. But for stationarity it is sufficiant that the absolute values of the roots of the quadratic equation is less than one. This is because, when the roots are distinct, we have that:

\sum_{j=0}^{\infty} |\psi_j| = \sum_{j=0}^{\infty} |c_1r_1^j + c_2r_2^j| \leq \sum_{j=0}^{\infty} |c_1||r_1|^j + \sum_{j=0}^{\infty}|c_2||r_2|^j < \infty.

This follows from the triangle inequality and because the sum of infinite geometric series are finite when |r_1|, |r_2| < 1.

The cases will be similar if we have complex roots or one repeated root. Our method for determining stationarity also holds for other ARMA models as well (and of course for all AR and MA models since they equal ARMA models with some weights set to zero).

Published by Lofgren Finance

Published

Leave a comment