By

 Robert A. Haugen

Introduction

In the Appendix to his book The Little Book that Beats the Market[i], Joel Greenblatt claims that his "magic formula" beats even the model based on "some of the best work done to date on sophisticated factor models"[ii] i.e., the multi-factor model used by Haugen Custom Financial Systems.  Since our model processes much more information with sophisticated statistical analysis, it would seem that it should exhibit superior performance. Greenblatt's claim that it doesn't is easily back-tested, so we rose to the challenge.  In addition, we performed a series of back-tests of different investment strategies closely related to his.

We find that Greenblatt's ranking procedure leads him into stocks that are, at the same time, relatively cheap and relatively profitable.  In the investment business, this style of investing is known as Growth At a Reasonable Price or GARP.  For reasons stated below in our concluding section, GARP investors usually out-perform the broad market indexes.  However, there are many ranking procedures that will take you to GARP.  Our results show that there is nothing special about the Greenblatt formula or his exclusion of finance and utility stocks from the investable population. 

Our results also show that using a process that employs many factors and sophisticated statistical analysis results in higher expected return lower volatility of return and a much lower probability of underperforming the broad market index.

                               Methodology

For the most part, we do not attempt here to replicate the results published in The Little Book that Beats the Market.[iii]  While we use our own "point in time database" (all information used in a strategy is known to have been publicly available at the time of investment), our database differs from Greenblatt's due to (a) differences in minimum data coverage requirements for including stocks, (b) periodicity in refreshing the database with new stocks, (c) exclusion of hundreds of penny stocks that are not investable by our clients and (d) differences in our sources of data (Bloomberg, Compustat, etc.).  For the tests presented here our population generally consists of the 1000 largest stocks in the United States , based on the market value of common stock.  All our portfolios initially contain the top 50 stocks, based on the individual ranking methodology employed in each test.  The portfolio rebalancing procedure is the same across all our tests.  In each month following initial portfolio construction, the four lowest ranked stocks are sold, and the proceeds are invested equally in the four combined highest ranked stocks in the population. (See Detailed Example of One Back-test.) 

For each ranking methodology, 12 portfolios are formed and tracked - one for each of the months of 1996 through 2005. The average performances of the 12 portfolios are presented in the tables.  Only the January portfolio will have a full year of performance in 2005.  The performance of the remaining 2005 monthly portfolios is annualized for that year.

Our trading rules result in an approximate annual turnover in the portfolios of 100%.  It is easy to calculate the performances net of trading costs.  For example, if you believe the round-trip transactions costs for the top 1000 stocks averages 2%, simply subtract 2% from all the performance numbers.

It is important to remember that, across our tables, the performances of the various strategies are directly comparable.  However, performances numbers between our tables and the Greenblatt book are not directly comparable.  Differences arise from differences in databases, the number of stocks held, ranking procedures, and rebalancing strategy.

While all ranking procedures tests focus on various measures of profitability and cheapness in price, none of our tests strictly employ the Greenblatt "magic formula".  For example, the measure of cheapness closest to Greenblatt's employs what is widely known as the earnings yield -- the ratio of latest reported 12-month trailing earnings per share to the market price of the common stock at the beginning of the month.

Instead, Greenblatt employs the ratio of income available for distribution to both debt and equity to the market value of equity plus the face value of debt.  Presumably this measures the cheapness of a combined investment in the firm's debt and equity.  However, Greenblatt follows the returns on an investment in the equity alone.  Why use an indicator of the combined cheapness of debt and equity when you're investing only in the equity?

Setting taxes aside, Greenblatt's indicator of cheapness can actually be seen as a composite of two ratios.  If E is income available for distribution to stockholders, I is interest paid on debt, P is the market value of the stock, and D is the face value of the debt, then Greenblatt's indicator of cheapness is (E+I)/(P+D).  The first ratio is E/P, the earnings yield, or the inverse of the price-to-earnings ratio - the most commonly used indicator of cheapness in the investment business.  The second ratio, I/D, isn't an indicator of cheapness at all.  It's the ratio of interest expense to the face value of debt.  The size of this ratio is determined by:  (a) the credit worthiness of the company, (b) the term of the debt when issued[iv], and (c) the general level of interest rates when the debt was originally issued. How does adding I/D to the process help in finding inexpensive stocks?  It would seem to only add more fog than clarity.  The relative importance of the two ratios in the composite is determined by the relative amount of debt in the firm's capital structure - the more debt the more the second ratio dominates.

To see where Greenblatt's indicator of cheapness will go wrong, suppose you have a company with lots of debt that is either low-grade or originally issued when interest rates were higher.  The company's stock might be wildly overvalued (P very high relative to E), but it still may seem cheap based on the magic formula.[v]

The measure of profitability closest to Greenblatt's is our return on assets (latest reported trailing 12-month operating income to the latest reported value for total assets).  Greenblatt actually uses return on net working capital plus net fixed assets.  While our return on assets and earnings yield are not exactly the same as Greenblatt's ratios, there is no denying that these are the closest to Greenblatt of any of the other ratios employed in our tests.

Results

Average Total Realized Return

Our first group of tests focuses on the question, "Does removing the financial and utility sectors provide a reliable way of improving performance?"  (See Back-tests with Alternative Two-sector Constraints.)  The answer seems to be no.  Removing these sectors actually reduces performance.  Removing other sector pairs seems to have no significant impact on average total realized return.  In the lower panel of this spread sheet, we show the yearly differences between each strategy on the S&P 500 stock index.  Note that the strategies tend to out-perform the S&P.

Our second group focuses on "Does the ranking methodology that is closest to Greenblatt's perform better than the others tested?"  (See Back-tests with Alternative two-factor Pairings.)  The answer appears to be no here as well.  In fact the methodology that is closest to Greenblatt's is near the bottom in terms of its performance.

There doesn't seem to be anything "magical" about Greenblatt's choice of ratios or exclusion of sectors.

We have also included the Haugen Systems model in the tests.  Here the same trading rules are employed, but stocks are ranked by their expected returns coming from the 65 factors in the Haugen model.  Note that the performance of this methodology is superior to all of the other models tested.  Its superiority derives from its complex use of more information -- it uses many more factors and it dynamically weights these factors based on the history of their payoffs.

Some have alleged that the Greenblatt results are the product of "data mining".  Did he run many models and publish the best result?  The answer from these tests seems to be no, since most of the other methodologies perform better than the one closest to the one he published.

Relative Risk

We now focus on the relative risk of the alternative strategies.  In the bottom rows of each panel, we show first the standard deviation or volatility of the year-to-year rates of return to the strategies. Note that in terms of the all-important excess return relative to the S&P, the Haugen multi-factor model is of much lower risk than any of the other models tested.  This is because the multi-factor model is much more diversified in terms of the factors that affect its signal.  The strength of its signal is much more stable than any of the models tested.

Even though the numbers come from a different data-base, and are not directly comparable, we have also included the Greenblatt results from the top 1000 stocks as reported in his book.  Greenblatt's strategy has much higher volatility than the multi-factor model.  In addition, the Greenblatt book model shows higher volatility and a greater probability of seeing a yearly negative return than any model tested.

In the excess return panels, below volatility, we show the information ratios of each strategy - the average out-performance relative to the S&P divided by the year-to-year standard deviation of the out-performance.  The multi-factor model has a much stronger information ratio than all other strategies including Greenblatt's results from his book.  In the next row the "T" statistics are reported for the average excess return.  A "T" statistic in excess of approximately 2.00 would indicate that the out-performance of a strategy of truly positive at a 95% level of confidence.  The out-performance of none of the simulated strategies are statistically significant.  The Greenblatt book strategy is barely significant, but the Haugen, multi-factor strategy is overwhelmingly significant.

Finally, in the bottom row of each panel, we show the probability of underperforming the S&P in any given year.  For the simulated two-factor strategies the probability is uniformly in excess of 30%.  For the Greenblatt book results it is 21%.  For the multi-factor strategy, the probability of under-performing the S&P in any given year is only 8%.

 A Rosetta Stone

In the tests presented above, we were unable to exactly duplicate Greenblatt's formula because some of the numbers in the formula are not included in our data-base.  This raises the possibility that subtle differences between the formula and our proxy may account for the comparatively poor performance of the proxy.  To address this issue we were able to obtain the exact fields used by Greenblatt for the period 1997 through 2002.  The top 1000 stocks were ranked by (a) earnings before interest and taxes to net operating assets and by (b) earnings before interest and taxes to enterprise value.  The rankings were combined and the top 30 stocks were selected.  The process was repeated and the portfolios were rebalanced after each 12-month period.  Separate tests were carried out for each of the 12 months of 1997.  The results for the 12 back-tests are averaged and presented in Rosetta Stone Back-tests .

As with our return on assets and earnings yield proxy, the magic formula doesn't distinguish itself.  This strongly indicates that the differences in the returns reported here and the returns reported in Greenblatt's book are attributable to differences in coverage in the data-bases.

Conclusion

A compelling question remains unanswered.  Why do all the methodologies tested, on average, out-perform the S&P 500 stock index? (See Graph of the Performance of Different Portfolio Strategies.)

All the methodologies build portfolios of stocks that are selling at relatively inexpensive prices (relative to current earnings, cash flow, etc.) and they are the stocks of relatively profitable companies. 

As discussed in the book The New Finance - Overreaction, Complexity and Uniqueness[vi], the stock market tends to overestimate the length of time it takes competition in lines of business to drive abnormally profitable and unprofitable companies to normal levels.  Competitive entry and exit from lines of business create mean reversion in profitability that is much faster than participants in the stock market generally believe.  Profitable and unprofitable companies tend to become average faster than the market thinks.  Consequently, the market tends to over-value currently profitable companies and under-value currently unprofitable ones.  As a result the under-valued, cheap stocks tend to produce higher rates of return in the future.

As discussed in The Inefficient Stock market - What Pays Off and Why[vii], the stock market also prices with imprecision, assigning different prices to stocks that have essentially the same earning potential.  It also assigns similar prices to stocks with widely differing earnings potential.  If this is true, given that two stocks are selling at the same price, buy the one that is more profitable and you'll get a higher future return. 

All of the methodologies tested bring you to portfolios of stocks that are both cheap and profitable.  In the investment business this type of investing is called GARP (Growth At a Reasonable Price).  For the reasons stated above GARP investing is likely to produce superior returns.  In our opinion, the merit of the Greenblatt book is not to present a "magic formula".  Rather, it serves to publicize the power of GARP investing.

In terms of risk as well as return, the multi-factor model dominates all of its two-factor counterparts.  Its use of a widely diversified number of signal sources, which vary in their individual strengths from time to time, results in a more stable overall signal that maximizes the probability of out-performance over the broad market index.

In summary, our conclusions are:

1.     We find no evidence that the Greenblatt results are a product of data mining.

2.     The Greenblatt results are dominated by most of the other strategies tested here in terms of the probability of getting poor results (negative return or negative excess return).

3.     Removing selected sectors from consideration doesn't seem to improve expected performance.

4.     There are reasons to suspect the Greenblatt formula as a method of screening into cheap stocks.

5.     As documented elsewhere, profitable stocks that sell cheaply in the market have relatively high, expected returns.

6.     Using a comprehensive list of factors creates enhances strength and creates stability in a factor model's signal.