Probing the link between financial markets and chaotic states, fractals, and related topics dates to the 1960s, when mathematician Benoit Mandelbrot effectively launched this line of research with his seminal study of cotton prices that documented “recurring patterns at every scale in data,” notes Wikipedia. The subject of chaos and how to exploit it for profit is an ongoing arena for study, albeit with mixed results. The Hurst exponent, for instance, attracted a lot of interest a number of years ago, in part due to an influential and widely read review of the indicator by Edgar Peters in his 1991 book Chaos and Order in the Capital Markets: A New View of Cycles, Prices, and Market Volatility.

There’s still plenty debate about how one might apply the Hurst exponent and related metrics in the day-to-day business of money management. Yet the field of chaos theory remains an intriguing and potentially rewarding area for research. My own interest in this corner has waxed and waned over the years, but I was intrigued anew when I read David Varadi’s brief but thought-provoking essay in March on “Using a Self-Similarity Metric with Intraday Data to Define Market Regimes.” As he explains,

Self-dissimilar objects are associated with complexity and chaos. Several mathematicians (including Mandelbrot) have observed that markets are clearly non-gaussian or non-normal. Markets exhibit “fat-tails” and have a distribution that shares more in common with a Levy distribution than the normal distribution which is used frequently in quantitative finance. But the market does not have a constant distribution- at times the market behavior is fairly normal in character while at other times the market is wild and unpredictable. The question is how we can effectively determine which regime the market is in so that we can apply the appropriate trading strategies to mitigate risk.

Varadi offers a simple but potentially useful answer with a measure of self-similarity based on analyzing the market’s intraday range and how it changes in the short term relative to a longer-run window. As a basis for further exploration, let’s take a stab at translating Varadi’s instructions by generating the data in R (see code below) for analyzing the US stock market via the S&P 500.

Varadi advises that “values above 0.5 indicate that the market is in a ‘chaos’ regime and is much less predictable and non-stationary; values below 0.5 indicate that the market is stable and much more predictable.”

Here’s a look at the indicator over the past year:

Note that this metric has been bouncing around the 0.5 mark recently (red line in chart above), which suggests an equilibrium/neutral condition. That’s in contrast with the stable regime (below 0.5) that prevailed in late-2014 through this past January.

Why is any of this relevant? Because most of the gains in the stock market through time are generated during the stable regime. As an illustration, let’s reproduce a chart in Varadi’s post, where he separates the stable regime from its chaotic counterpart and generates wealth indexes for each data set. The key message, he notes, is that “the market performs quite poorly in ‘chaos’ conditions, and seems to make all of its long-term returns in the ‘stable’ regime.”

The notion that trending/momentum factors are valuable sources of risk premia is old news, of course. The challenge is figuring out when market conditions are ripe for earning a profit from these factors… and when they’re not.

Varadi’s indicator is hardly the last word on modeling the market’s evolution between chaos and stability, but it’s a useful first approximation. As such, it’s a good benchmark for additional research and looking for possibilities for enhancing this signal. (For additional perspective on this indicator, take a look at his follow-up piece here.)

*R Code to generate the data in the charts above:*

library(TTR) library(zoo) library(quantmod) # download daily S&P 500 data sp500 <-getSymbols('^gspc',from='1990-12-31',auto.assign=FALSE) # Varadi outlines 6 steps for calculating his market-regime indicator: # 1. "Find the high minus the low for each day going back 10 days." # 2. "Take the sum of these values (sum of the pieces)." hi.lo <-Hi(sp500)-Lo(sp500) hi.lo.10 <-rollapply(hi.lo,10,sum) # 3. "Find the 10-day range by taking the 10-day maximum (including the highs) # and subtracting out the 10-day minimum (whole range)." hi <-Hi(sp500) lo <-Lo(sp500) hi.10 <-rollapply(hi,10,max) lo.10 <-rollapply(hi,10,min) hi.lo.diff <-hi.10-lo.10 # 4. "Divide the sum of the pieces by the whole range-- # this is a basic measure of fractal dimension/complexity." frac.dim.complex <-hi.lo.10/hi.lo.diff # 5. "Take the 60-day average of the 10-day series of the complexity values-- # this is the "chaos/stability" metric." frac.dim.complex.60 <-na.omit(SMA(frac.dim.complex,60)) # 6. "Use either the 252-day normsdist of the z-score # or the percentile ranking of the chaos/stability metric." #regime <-na.omit(pnorm(scale(frac.dim.complex.60))) regime <-na.omit(runPercentRank(frac.dim.complex.60,252)) # values above .5 indicate that the market is in a "chaos" regime # and is much less predictable and non-stationary, # values below .5 indicate that the market is stable # and much more predictable. # Separate chaotic data from stable data # and generate wealth indexes # calculate S&P 500 daily % return sp500.ret <-na.omit(ROC(Cl(sp500),1,"discrete")) # create wealth index for stable regime periods regime.stable.signal <-ifelse(regime < 0.5,1,0) regime.stable.ret <-regime.stable.signal*tail(sp500.ret,length(regime.stable.signal)) regime.stable.index <-cumprod(1+regime.stable.ret) # create wealth index for chaotic regime periods regime.chaos.signal <-ifelse(regime > 0.5,1,0) regime.chaos.ret <-regime.chaos.signal*tail(sp500.ret,length(regime.chaos.signal)) regime.chaos.index <-cumprod(1+regime.chaos.ret) # combine wealth indexes for charting regimes.2 <-cbind(regime.stable.index,regime.chaos.index) ### END

John HallInteresting post. I had seen the paper, but I hadn’t given it a thorough reading. I did the calculations by myself as far as Yahoo has intra-day high/low data (January 1962).

I noticed a very strong change in your variable frac.dim.complex.60 that seems to take place in July 1981 (when S&P 500 futures begin trading).

Based on the chart showing the performance of the strategy, I wanted to evaluate if it was just the mean that was different (if above 0.5 or not). I was concerned that the strategy was actually picking up market regimes based on volatility, so that the periods identified had a higher mean because they were lower volatility. However, when I ran the numbers, the mean is much more different than the volatility. That would suggest that the stable strategy has a much higher Sharpe ratio. Worth looking into further.

Pingback: Parsimony | CSSA

RickHi James. What is the last date in your S&P 500 data and what is the value of the indicator?

James PicernoPost authorRick,

As of the publication date, the data was through 4/22/2015 with the following values:

Market regime indicator (top chart): 0.474

The values in the 2nd chart:

Stable regime wealth index: 6.477

Chaotic regime wealth index: 0.812

–JP

RickThanks. I programmed in excel and I also checked manually by looking at all 252 instances of the stability metric for April, 22, 2015. I get a different value, about 0.58, above 0.5. Some others in Varadi’s blog also claimed to be getting different results. Some things are too confusing with this indicator. If you looks at his second blog on this you get the idea. Best.

martin bauerHi James,

excellent work and all credit to you and Varadi.

Lets step back for a moment and look for some pitfalls

1) the high and a low for an index isn’t trade able and also dividends are not reinvested, so let substitute GSPC with a instrument like SPY – the performance is similar so far so good

2) Lets look for an European Performance Index i.e. GDAXI

Suddenly the performance of this strategy isn’t that impressive

3) Lets try a different asset class like volatility ie VIX

now it is ugly

4) ok Volatility is to special lets try Bonds ie TLT

oooops

5) Finally and that is the interesting part lets look at GLD

here we would expect what – the same behaviour

stable good performance and chaos bad performance ?

it isn’t it is exactly the opposite

The biggest problem I see, as soon an indicator is only working for a single equity market you have to be very careful not introducing a bias!!!

James PicernoPost authorMartin,

Good points. Running the analysis across all the major asset classes certainly makes sense. The first question: how do the numbers compare through time and what, if any, clues about market behavior do we discover? Sounds like a fertile line of inquiry.

–JP

RickGood points by Martin. I found this analysis by Michael Harris with similar conclusions after a google search for “cssa regime indicator”

http://www.priceactionlab.com/Blog/2015/04/performance-in-stable-and-chaotic-markets-based-on-the-cssa-regime-indicator/

However I cannot still get the same results for April 22, no matter how hard I try. There must be a difference in the ranking function implemented in R.