
Board of Governors of the Federal Reserve System
International Finance Discussion Papers
Number 895, May 2007--- Screen Reader
Version*
NOTE: International Finance Discussion Papers are preliminary materials circulated to stimulate discussion and critical comment. References in publications to International Finance Discussion Papers (other than an acknowledgment that the writer has had access to unpublished material) should be cleared with the author or authors. Recent IFDPs are available on the Web at http://www.federalreserve.gov/pubs/ifdp/. This paper can be downloaded without charge from the Social Science Research Network electronic library at http://www.ssrn.com/.
Abstract:
Since the seminal work of Mandelbrot (1963),
-stable distributions with infinite variance have been
regarded as a more realistic distributional assumption than the
normal distribution for some economic variables, especially
financial data. After providing a brief survey of theoretical
results on estimation and hypothesis testing in regression models
with infinite-variance variables, we examine the statistical
properties of the coefficient of determination in models with
-stable variables. If the regressor and
error term share the same index of stability
, the coefficient of determination has a
nondegenerate asymptotic distribution on the entire [0, 1] interval, and the density of this distribution is
unbounded at 0 and 1. We provide closed-form expressions
for the cumulative distribution function and probability density
function of this limit random variable. In contrast, if the indices
of stability of the regressor and error term are unequal, the
coefficient of determination converges in probability to
either 0 or 1, depending on which variable has the
smaller index of stability. In an empirical application, we revisit
the Fama-MacBeth two-stage regression and show that in the
infinite-variance case the coefficient of determination of the
second-stage regression converges to zero in probability even if
the slope coefficient is nonzero.
Keywords: Regression models,
-stable distributions, infinite variance, coefficient
of determination, Fama-MacBeth regression, Monte Carlo simulation,
signal-to-noise ratio, density transformation theorem.
JEL classification: C12, C13, C21, G12
Granger and Orr (1972) begin their article, " 'Infinite variance' and research strategy in time series analysis," by questioning the uncritical use of the normal distribution assumption in economic modelling and estimation:
Due in part to the influential seminal work of Mandelbrot
(1963),
-stable distributions are often
considered to provide the basis for more realistic distributional
assumptions for some economic data, especially for high-frequency
financial time series such as those of exchange rate fluctuations
and stock returns. Financial time series are typically fat-tailed
and excessively peaked around their mean--phenomena that can be
better captured by
-stable distributions
with
rather than by the normal
distribution, for which
.4 The
-stable distributional assumption with
is thus a generalization of rather
than an alternative to the Gaussian distributional assumption. If
an economic series fluctuates according to an
-stable distribution with
, it
is known that many of the standard methods of statistical analysis,
which often rest on the asymptotic properties of sample second
moments, do not apply in the conventional way. In particular, as we
demonstrate in this paper, the coefficient of determination--a
standard criterion for judging goodness of fit in a regression
model--has several nonstandard statistical properties if
.
The linear regression model is one of the most commonly used and basic econometric tools, not only for the analysis of macroeconomic relationships but also for the study of financial market data. Typical examples for the latter case are estimation of the ex-post version of the capital asset pricing model (CAPM) and the two-stage modelling approach of Fama and MacBeth (1973). Because of the prevalence of heavy-tailed distributions in financial time series, it is of interest to study how regression models perform when the data are heavy-tailed rather normally distributed.
The first purpose of the present paper is to survey theoretical results of estimation and hypothesis testing in regression models with infinite-variance distributions, and the second is to establish that infinite variance of the regression variables has important consequences for the statistical properties of the coefficient of determination and tests of the hypothesis that this coefficient is equal to zero. Third, we revisit the Fama-MacBeth two-stage regression approach and demonstrate that infinite variance of the regression variables can affect decisively the interpretation of the empirical results.
The rest of our paper is structured as follows. In
Section 2 we provide a
brief summary of the properties of
-stable
distributions and of aspects of estimation, hypothesis testing, and
model diagnostic checking in regression models with
infinite-variance regressors and disturbance terms.
Section 3
provides a detailed analysis of the asymptotic properties of the
coefficient of determination in regression models with
infinite-variance variables. In our empirical application,
presented in Section 4, we revisit the
data used in Fama and French (1992), and we show that the
statistical and/or economic interpretation of their findings can be
quite different under the maintained assumption of
-stable distributions from an interpretation based on
the assumption of normal distributions. Section 5 summarizes the
paper and offers some concluding remarks.
A random variable
is said to have a stable
distribution if, for any positive integer
,
there exist constants
and
such that
, where
are independent copies
of
and
signifies equality in
distribution. The coefficient
above is
necessarily of the form
for some
(see Feller, 1971,
Section VI). The parameter
is called the
index of stability of the distribution, and a random
variable
with index
is
called
-stable. An
-stable
distribution is described by four parameters and will be denoted by
. Closed-form
expressions for the probability density functions of
-stable distributions are known to exist only for
three special cases.5 However, closed-form expressions for
the characteristic functions of
-stable
distributions are readily available. One parameterization of the
logarithm of the characteristic function of
is
The tail shape of an
-stable distribution is
determined by its index of stability
. Skewness is governed by
; the distribution is symmetric
about
if and only if
. The scale and location parameters of
-stable distributions are denoted by
and
, respectively. When
, the log characteristic function
given by equation (1) reduces to
, which is that
of a Gaussian random variable with mean
and variance
. For
and
, the tail properties of
an
-stable random
variable
satisfy
i.e., both tails of the probability density function (pdf)
of
are asymptotically Paretian. For
and
(
), the distribution is maximally
right-skewed (left-skewed) and only the right (left) tail is
asymptotically Paretian.6 The term
in
equations (2)
and (3) is
given by
Because
, it follows that
for
and
for
if
is
-stable with
.8 Only moments of order
up to but not including
are finite if
, and a non-Gaussian stable
distribution's index of stability is also equal to its maximal
moment exponent.9 In particular, if
, the variance is infinite but
the mean exists. For
, it follows that
; in addition, for
,
is equal to the
distribution's mode and median irrespective of the value
of
, justifying the use of the term
``central location parameter'' for
in the
finite-mean or symmetric cases. In addition, for
, one can show that
.10 We make use of this property below
in the derivations of Theorem 1 and
Remark 3.
The class of
-stable distributions is an
interesting distributional candidate for disturbances in regression
models because (i) it is able to capture the relative frequencies
of extreme vs.observations in the economic variables, (ii) it has
the convenient statistical property of closure under convolution,
and (iii) only
-stable distributions can
serve as limiting distributions of sums of independent and
identically distributed (iid) random variables, as proven in
Zolotarev (1986). The latter two properties are appealing for
regression analysis, given that disturbances can be viewed as
random variables which represent the sum of all external effects
not captured by the regressors. For more details on the properties
of
-stable distributions, we refer to
Gnedenko and Kolmogorov (1954), Feller (1971), Zolotarev (1986),
and Samorodnitsky and Taqqu (1994). The role of the
-stable distribution in financial market and
econometric modelling is surveyed in McCulloch (1996) and Rachev
et al.1999).
Let
and
be two jointly
symmetric
-stable (henceforth,
) random variables with
,
i.e., we require
and
to have
finite means. Our main reason for concentrating on the case
lies in its empirical relevance.
Estimated maximal moment exponents for most empirical financial
data, such as exchange rates and stock prices, are generally
greater than 1.5; see, for example, de Vries (1991) and
Loretan and Phillips (1994). An econometric (purposeful) reason for
studying the case
is that, for
-stable distributions with
, regression analysis that is based on sample
second moments, such as least squares, is still asymptotically
consistent for the regression coefficients, even though the limit
distributions of these regression coefficients are
nonstandard.11 Suppose that the regression of a
random variable
on a random
variable
is linear, i.e., there exists a
constant
such that
with
where
is the scale parameter of the
variable
and
in the numerator is
covariation (covariance in the Gaussian case), which can be
calculated as
, for all
with
.
For estimation and diagnostics, the relation (5) can be written as a regression model with a constant term,
where the maintained hypothesis is that
is iid
, with
. The econometric issues of
interest are to estimate
properly, to test the
hypothesis of significance for the estimated parameter, usually
based on the
-statistic, as well as to compute model
diagnostics, such as the coefficient of determination, the
Durbin-Watson statistic, and the
-test of parameter
constancy across subsamples.
The effects of infinite variance in the regressor and
disturbance term can be substantial. If the variables share the
same index of stability
, the ordinary
least squares (OLS) estimate of
is
still consistent, but its asymptotic distribution is
-stable with the same
as the
underlying variables. Furthermore, the convergence rate to the true
parameter is
, smaller than the rate
which applies in the finite-variance
case. If
, OLS loses its best linear
unbiased estimator (BLUE) property, i.e., it is no longer the
minimum-dispersion estimator in the class of linear estimators
of
. In addition, the asymptotic
efficiency of the OLS estimator converges to zero as the index of
stability
declines to
.
Blattberg and Sargent (1971) (henceforth, BS) derived the BLUE
for
in (6) if the value
of
is known. The BS estimator is given
by
|
(7) |
which coincides with the OLS estimator if
.
Kim and Rachev (1999) prove that the asymptotic distribution of the
BS estimator is also
-stable. Samorodnitsky
et al.2007) consider an optimal power estimate based on the BS
estimator for unknown
, and they
also provide an optimal linear estimator of the regression
coefficients for various configurations of the indices of stability
of
and
. Other
efficient estimators of the regression coefficients have been
studied as well; Kanter and Steiger (1974) propose an unbiased
-estimator, which excludes very large
shocks in its estimation to avoid excess sensitivity due to
outliers. Using a weighting function, McCulloch (1998) considers a
maximum-likelihood estimator which is based on an approximation to
a symmetric stable density.
Hypothesis testing is also affected considerably when the
regressors and disturbance terms have infinite-variance stable
distributions. For example, the
-statistic,
commonly used to test the null hypothesis of parameter
significance, no longer has a conventional Student-
distribution if
. Rather, as established
by Logan et al.1973), its pdf has modes at
and
; for
these modes are infinite. Kim (2003) provides
empirical distributions of the
-statistic for
finite degrees of freedom and various values of
by simulation. The usual applied goodness-of-fit test
statistics, such as the likelihood ratio, Lagrange multiplier, and
Wald statistics, also no longer have the conventional asymptotic
distribution, but have a stable
distribution, a term that was
introduced by Mittnik et al.1998).
In time series regressions with infinite-variance innovations,
Phillips (1990) shows that the limit distribution of the augmented
Dickey-Fuller tests for a unit root are functionals of Lévy
processes, whereas they are functionals of Brownian motion
processes in the finite-variance case. The
-test
statistic for parameter constancy that is based on the residuals
from a sample split test has an
-distribution in
the conventional, finite-variance case. Kurz-Kim et al.2005)
obtain the limiting distribution of the
-test if the
random variables have infinite variance. As shown by the authors,
as well as by Runde (1993), the limiting distribution of the
-statistic for
behaves completely differently from the Gaussian
case: whereas in the latter case the statistic converges to 1
under the null as the degrees of freedom for both numerator and
denominator of the statistic approach infinity, in the former case
the statistic converges to a ratio of two independent, positive,
and maximally right-skewed
-stable distributions. This result
is used below to derive closed-form expressions for the pdf and
cumulative distribution function (cdf) of the limiting distribution
of the
statistic if the regressor and
disturbance term share the same index of stability
.
Moreover, commonly used criteria for judging the validity of
some of the maintained hypotheses of a regression model, such as
the Durbin-Watson statistic and the Box-Pierce
-statistic, would be inappropriate if one were to rely on
conventional critical values. Phillips and Loretan (1991) study the
properties of the Durbin-Watson statistic for regression residuals
with infinite variance, and Runde (1997) examines the properties of
the Box-Pierce
-statistic for random variables
with infinite variance. Loretan and Phillips (1994) and Phillips
and Loretan (1994) establish that both the size of tests of
covariance stationarity under the null and their rate of divergence
of these tests under the alternative are strongly affected by
failure of standard moment conditions; indeed, standard tests of
covariance stationarity are inconsistent if population
second moments do not exist.
For the general asymptotic theory of stochastic processes with stable random variables, we refer to Resnick (1986) and Davis and Resnick (1985a, 1985b, 1986). Our results in this section are, in large part, an application of their work to the regression diagnostic context.
The maintained assumptions are:
The fourth assumption, that the regressor and the error term
have the same index of stability, is rather strong, and its
validity may be difficult to ascertain in empirical applications.
In Corollary 2
below, we examine the consequences of having unequal values for the
indices of stability for
and
for the asymptotic properties of the coefficient of
determination.
The coefficient of determination measures the proportion of the total squared variation in the dependent variable that is explained by the regression:
Because
and
, where
and
are the
respective sample averages of
and
, and because
=0 by
construction, the coefficient of determination may be written as
Since
and
are
in the normal domain of attraction of a stable distribution with
index of stability
, norming by
rather than
by
is required to obtain non-degenerate
limits for the sums of the squared variables. Because
by the assumption
of consistent estimation, an application of the law of large
numbers to
, the continuous mapping
theorem, and the results of Davis and Resnick (1985b) yield the
following expression for the joint limiting distribution of the
elements in equation (9):
For
, the random variables
and
are
independent, maximally right-skewed, and positive stable random
variables with index of stability
,
,
,13
, and log characteristic function
We therefore conclude that, under the five maintained
assumptions of this section, the
statistic of
the regression model (8) has the
following asymptotic distribution.
Thus, for
and
, the coefficient of determination does not
converge to a constant but has a nondegenerate asymptotic
distribution on the interval
. This
contrasts starkly with the standard, finite-variance result, which
is stated here for completeness.
In the finite-variance case, the model's asymptotic
signal-to-noise ratio,
, is
constant, as is therefore the limit of the coefficient of
determination. In contrast, in the infinite-variance case the
model's limiting signal-to-noise ratio is given by
, where
and
, and is therefore a random
variable even asymptotically; it is this feature that causes the
randomness of
. We postpone a
fuller discussion of the intuition that underlies this result to
the end of this section, after we provide a detailed analysis of
the statistical properties of
.
Before doing so, however, we note that the fourth maintained
assumption, i.e., that the indices of stability of the regressor
and error term in (8) be the same, is
crucial for obtaining the result that the asymptotic distribution
of
is nondegenerate. Indeed, if
the two indices of stability differ, the asymptotic properties of
the
statistic are as follows.
Corollary 2
Suppose that the maintained assumptions of
Theorem 1 apply
except that
, i.e., suppose that
the indices of stability of the regressor and error term are
unequal. Let
to rule out the trivial case from
further consideration. Then,
Thus,
converges to
in probability if
, and it converges
to 0 in probability if
.
|
Heuristically, if
and
, the limiting distribution of the
statistic is degenerate at 0
or 1 because the model's asymptotic signal-to-noise ratio is
either zero (if
) or infinite (if
). From an
examination of the proof of this corollary, we can also deduce that
if
, the fifth
maintained assumption--that the regression coefficients are
estimated consistently--could be relaxed, to require merely that an
estimation method be employed that guarantees
; the result that
converges either to 0 or 1
would continue to hold in this case.
Returning to the main case of
, we note that
the random variable
is defined for all values of
, even though in a regression
context one would typically assume that
. We now establish some
important qualitative properties of
.
|
Thus,
is equal to the non-random limit
of
in the finite-variance case. Since
and
are positive
a.s., we also have
,
i.e., the median of
is equal to 1,
regardless of the value of
. As we will
demonstrate rigorously later in this paper, the probability mass
of
is highly concentrated around 1 for
values of
close to 2. Conversely, for
small values of
,
is unlikely to be close to 1; instead, it is very
likely that one will obtain a draw of
that is
either very small, i.e., close to 0, or very large. A small or
large draw of
has a crucial effect on the
model's signal-to-noise ratio,
, and
therefore also on
. This suggests that an
informal measure of the effect of infinite variance in the
regression variables on the value of
in
a given sample may be based on the difference between the
model's coefficient of determination and a consistent estimate of
its median
, say
, where
. The larger the difference between
and
, the more important the effect is of
having obtained a small (or large) value of
.
The following remark shows that a finite-variance property
of
for
,
viz.,
, carries over in
a natural way to
.
|
The symmetry of
about
for
follows immediately from this result
and the fact that the distribution's support is the
interval [0, 1].
Next, as the following remark shows, the pdf of
has infinite modes
at 0 and
, i.e., at the
endpoints of its support.
|
where the joint pdf
is nonzero on
. The case
can occur only if
; if
, however, the random
variables
and
are
perfectly dependent, their joint pdf is nonzero only on the
positive
-halfline, and the joint pdf
reduces to
,
.
Hence, for
we find
|
By Remark 2, we have
as well. The continuity
of the cdf of
on
for
follows from the continuity of the
cdfs of
and
on
and the fact that their pdfs
are equal to zero at the origin. For example, one finds that
; the result
then follows from
Remark 2.
The fact that the probability density function of
has infinite singularities
may seem unusual. However, the presence of singularities is a
regular feature of pdfs that are based on ratios of stable
random variables. For example, Logan et al.1973) and Phillips
and Hajivassiliou (1987) showed that if
,
the density of the
-statistic has infinite
modes at
and
; similarly,
Phillips and Loretan (1991) demonstrated that if
, this feature is also present in the asymptotic
distributions of the von Neuman ratio and the normalized
Durbin-Watson test statistic.
The remarks in the preceding subsection provide important
qualitative information about some of the distributional properties
of
. However, they do not
address issues such as whether the distribution has modes beyond
those at 0 and 1, whether the discontinuity of the pdf at
the endpoints is simple or if
diverges--and, if so,
at which rate--as
or
, or how much of the
distribution's mass is concentrated near the endpoints of the
support. To examine these issues, we provide expressions for the
cdf and pdf of
in this subsection.
It is possible to do so because
is a continuously
differentiable and invertible function of the ratio of two
independent, maximally right-skewed, and positive
-stable random variables, and because closed-form
expressions for the cdf and pdf of this ratio are known. The latter
expressions are provided in the following proposition.
The cdf of the random variable
is shown in
Figure 2 for various values of
between 1.98 and 0.25.18The random variable
has several interesting properties. First,
note that
and that
the rate of divergence to infinity of
as
is given by
; thus, the pdf
of
has a one-sided infinite singularity
at 0. Second, as
,
for a suitable
constant
. This result, along with
, implies that
lies in the normal domain of attraction of a positive
stable distribution, say
, with index of stability
and
, the
same parameters as that of the variables
and
.19 Hence, the mean
of
is infinite for all values of
. Third, in the special case of
,
and
are
each distributed as a Lévy
-stable
random variable, which is well known to be equivalent to the
inverse of a
random variable. For
, then, the pdf of
reduces to
, which is
also the pdf of an
distribution; see
Runde (1993).
As was noted earlier, the median of
is
equal to 1 for all values of
. The regression model's
signal-to-noise ratio is given by the random
variable
if
,
whereas it is given by the constant
in
the standard, i.e., finite-variance case. The fact that the random
variable which multiplies
has a median
of 1 helps to develop further the intuition that underlies the
result of Remark 1, viz.,the
median of
,
, is the same in both the
finite-variance and the infinite-variance cases. Finally, an
inspection of equation (13) reveals that
and
; put
differently,
. The
probability mass of
therefore becomes perfectly
concentrated at 1 as
, even though, of course, its
mean remains infinite as long as
.
From Theorem 1, we have
, say.
Note that
satisfies the conditions
of Proposition 1 and that the
function
is continuously differentiable and strictly increasing in the
interior of its domain. We are therefore able to provide the
following expressions for the cdf and pdf of
by an application of the
density transformation theorem.20
The pdf of
for
is given by
The probability density functions and cumulative distribution
functions of
for values of
between 0.25 and 1.98 are
graphed in Figures 3 and 4. (In all cases, we have set
.) The pdfs in Figure 3 are shown
with a logarithmic scale on the ordinate. Since we know that
, we
graph the functions only for
. The
graphs show that
A heuristic summary of these properties of
is straightforward. We begin
by recalling that the multiplicative term
, shown in equation (4) and
Figure 1, affects the probability of tail-region values of the
random variables in question, and that the rate of decline in the
tail areas of density of
-stable random
variables increases as
. Suppose first
that
is very close to 2; then,
is close to 0, and the
fraction of observations of
and
that fall into the respective
Paretian-tail regions is therefore very low; moreover, given the
fairly rapid decay of the density's tails for
close to 2, the likelihood of obtaining a very
large draw, conditional on obtaining a draw from the Paretian tail
area, is also low. As a result, the probability of observing large
observations of
and
is
quite low. This, in turn, makes it unlikely to observe a very large
draw of either
or
and thus
of observing a value of
that is either
close to 0 or very large. Therefore, if
is very close to 2,
is likely
close to its median of 1, and most of the mass of
is concentrated near its
median,
. Next, as
moves down and away from 2, say to
around 1.5,
increases rapidly, leading
to a higher frequency of observing tail-region draws
for
and
. In
addition, as the density in the tail region declines more slowly
for smaller values of
, it is much
more likely of obtaining very large draws of the regressor and
error term than if
is close to 2. In
consequence, if
is around 1.5, it is
quite likely to obtain draws of
that are
either very close to zero or very large, and thus more of the
probability mass of
is located near the edges of
its support. Conversely the interior mode of
is considerably less
pronounced than if
is close to 2.
Finally, as
decreases further,
rises further, and both the frequency of tail
observations and the likelihood that any draws from the tail areas
will be very large increase. Therefore, it is very likely that the
largest few observations of
or
will dominate the realization
of
and therefore the realization of
. As a result, if
is small the central mode of
vanishes entirely and
almost all of its probability mass is located very close to the
endpoints of the distribution's support. In the limit, as
,
converges to a Bernoulli
random variable, for which all of the probability mass is located
at 0 and 1.
Fama and MacBeth (1973) proposed the so-called Fama-MacBeth
regression to test the hypothesis of a linear relationship between
risk and risk premium in stock returns in a cross-sectional
setting. Let
be the return on market
portfolio
at time
,
where
and
; denote the average return of
portfolio
as
;
denote the average portfolio return at time
as
; and
denote the average portfolio return across all time periods by
. The
first-stage Fama-MacBeth regression is an ex post CAPM,
where
,
, and
is iid
the same index
as
.
We may assume that the distribution of
has a finite mean and variance,
say,
and
. Denote the OLS
estimates of the regression coefficients in equation (17) by
and
. The second-stage
Fama-MacBeth regression is given by
where
is iid
the same index
as
,
, and
.
The
statistic of the second-stage
Fama-MacBeth regression is given by
This statistic has the following asymptotic properties.
Thus, if
,
, at a rate that is
proportional to
.
This result does not conflict with the one provided in
Theorem 1, as the present case is one of an unbalanced
regression design: the regressor has an asymptotically finite
variance, whereas the error term has infinite variance, implying
that the asymptotic signal-to-noise ratio is zero. Instead, this
result is closely related to the one provided in Corollary 2,
which examined the asymptotic limit of
if
. We note that even
if
is fixed (as is generally taken to be the
case in Fama-MacBeth regressions), the dispersion of
will likely be quite a bit
smaller than that of
, indicating that the
model's signal-to-noise ratio,
, and
hence the median of
, in the