Diversifying across asset classes (otherwise known as asset allocation) is the foundation on which prudent, long-term successful investment strategies are built. One of the essential issues for allocating among the various asset classes is carefully choosing those pieces that will offer the most diversification bang for the buck, and then weighting the asset accordingly. Modern portfolio theory advises that there are three primary variables that feed into this decision: volatility of returns, expected returns for each asset, and the correlation of returns among the assets.

Focusing on the latter for the moment reveals several interesting trends for the strategically minded investor. (We’ll be publishing more on correlations going forward, but for the moment here’s a taste of what we’re tracking.) Let’s start with the classic stock/bond mix, for which we crunched the data based on rolling 36-month trailing correlations for monthly total returns between the Russell 3000 and the Lehman Aggregate Bond Index, plotted monthly, starting in January 2001 and running through last month.

As the chart below reveals, the sharply negative correlation that defined equities and fixed-income in recent years is giving way to something less. To be sure, stocks and bonds still post slightly negative correlation, and so the diversification factor remains potent for owning both asset classes. But if the trend in recent years keeps up, investors may want to re-examine diversification expectations for the classic stock/bond mix. (Note: 1.0 indicates perfect correlation, 0 is no correlation, and -1.0 is perfect negative correlation).

Meanwhile, the correlation trend in domestic/foreign stocks has been improving, although only marginally so. In fact, the diversification value of domestic/foreign equities has been of relatively limited value in recent years, as the chart below shows. Thanks to globalization and enhanced liquidity across borders, the Russell 3000 and the MSCI EAFE ($) returns correlation has been high in recent years. Ditto for U.S. stocks and emerging markets equities, as tracked by MSCI EM ($). Is that high correlation status in the process of changing? Perhaps, as the fall in correlation between the two indices of late suggests (see chart below). If the trend continues, allocating a greater portion of assets to foreign stocks may be warranted in the future. Stay tuned.

If there’s hope for the domestic/foreign equity mix as a diversification tool, something less inhabits the realm of large- and small-cap domestic stock blends. Or so the trend of late indicates. As the chart below illustrates, the correlation of monthly returns between the Russell 1000 and Russell 2000 indices (large- and small-cap proxies, respectively) has been on the rise. In fact, the correlation between large- and small-caps has been fairly lofty in recent years, and that status doesn’t show any sign of changing. As a result, the diversification benefit of owning large and small equities within the U.S. has been uninspiring.

The correlation between monthly returns for REITs and U.S. stocks is on the rise too. The low correlations of 2002 have been fading for some time. Although REITs still have roughly a 0.5 correlation (based on the Wilshire REIT and Russell 3000), the diversification effect has sharply diminished in recent years.

If nothing else, the above charts remind that while finding low and negatively correlated assets is essential for building a diversified portfolio, the definition of what constitutes low and negative is always evolving.

David StevensLarge vs small. Is the Russell 3000 that good a proxy for large cap? It includes the Russell 2000, so of course its correlation should be fairly high.

I think the Russell 1000 would be a better index to represent large cap. I suspect that the correlation is still higher than it has been in the past, so your conclusion is probably valid. But an independent variable would be a better measuring stick.

quintsquarryI would be very interested in seeing some other things that may not correlate, like commodities vs stocks,or maybe foreign exchange rates.

James PicernoDavid,

You’re quite right. Russell 1000 is the better proxy for large caps. But it doesn’t change much, probably because Russell 3000 is dominated by the same large caps that prevail in R1000. In any case, for clarity, I ran the correlation numbers for R1000/R2000 and posted a new chart.

James PicernoQuintsquarry,

Your wish is my command. I’m crunching a variety of indices. In the near future, I’ll publish more results.

Bill ConerlyInteresting charts, but remind me: does the correlation of two asset classes over the last 36 months have more predictive value going forward than long-term correlations?

James PicernoBill,

You raise a great question. Alas, I have less than a great answer. I’m not aware of any study that says which trailing period offers greater predictive ability for estimating future correlations.

The past, of course, generally holds only limited value as a window on the future, whether it’s correlations, volatility or returns. That said, longer-term data is probably more representative in terms of history, but not necessarily more practical for making assumptions about the next three years, for instance.

I’m doing a bit of research on correlations these days and I’ll be talking with some of the best minds in the business on the subject. Hopefully, I can post some insight in the near future that sheds light on your question.

Jeff DunnI’m not sure how many of you are aware of robust mean variance optimization so I thought it would be prudent to bring it up in this conversation.

While undoubtedly one of the most important frameworks developed for risk management, the Markowitz mean variance optimization has major flaws. Bill brings up a great point about the use of historical correlations to predict future behavior. In this study, the author is using a 36 month window; why not use a 60 month or a 12 month or perhaps use weekly information. Each of these methods will produce a new correlation that you are assuming will be indicative of the behavior of these two assets into the future. The time period that you chose is indicative of a certain part of the economic cycle whereas assets can have different correlations in up markets as opposed to down markets. (remember, correlations come from markets prices and I think most of us can agree that investor psychology, and therefore behavior, changes over a market cycle.)

Another time that investor psychology, and in turn behavior, changes is following a major market event. For example, following the russion default crisis there was a flight to quality in american tresuries. Foriegn markets experienced massive cash outflows as investors tried to limit their exposures. This massive worldwide selling of foreign assets, not based strictly on the fundamental picture of each individual market but on a psychological flight to quality, caused the correlation between foriegn stock markets to increase very quickly. Markets that normally traded on different information and did not track so closely were now all “shittin the bed” on the same news and as such correlations were moving towards one (ie much less diversification benefit). Ask the brianiacs at LCTM about problems of using historical correlations.

The real issue here is that mean variance optimization not only uses historical information, including correlations and beta for expected returns, but inherently maximizes the error of your estimations. Robust Markowitz optimization is part of a new field of optimizations called inverse conic optimizations. In this type of optimzation you recognize that your covariance matrix (correlations adjusted for variability) and your expected returns have errors so you introduce a certain error term in your optimization. This stops the model from maximizing your estimation error and instead looks at each correlation as a distribution of possible correlations.

This method of optimization is still not perfect but I believe it is far superior to the regular mean variance optimization because it looks at the realities of projecting returns and correlations into the future. While having run robust optimizations, there is one major problem that us non-mathematicians run into very quickly with this process and that is that you need to come up with an error term. So… does anyone out there have a process for doing so?

PS. Great work on the blog…

JJSorry for the late comment on this… Great charts. Are you aware of (or capable of producing)updated charts on these correlations? It would be interesting to see, for each pair of correlations, both long-term (50 year?) correlations as well as more recent (5 year, rolling 36 month) correlations, for example. Even better, would be some sort of tool where the user can input what indices and time periods he/she would like to see. Keep up the good work!

Jim CahillIn reading JP’s comment above from March, 2006, is there a tool where the user can create correlation charts with indices that they specify or pick from a menu?