Researchers name top quant analytics

PortfolioConstruction Forum asked the research houses: Which are the top five quant analytics that practitioners should always consider, and why? And how much credence does your firm believe quantitative measures should be given in analysing a fund?

Lonsec

The top five analytics used by Lonsec are:

• Absolute Return - the point-to-point change in a fund's total value (assuming cash distributions are reinvested) over a defined period. For periods greater than one year, the geometric average (compound annual growth rate) is calculated. 
• Excess Return - the difference between a fund's absolute return and its benchmark return for a given period. Often referred to as alpha, positive excess return implies outperformance versus the benchmark.
• Standard Deviation - calculates how much the intra-period (typically monthly) absolute return varies from its average over a defined period (annualised for periods greater than one year). It is commonly used to represent the risk associated with a fund.
• Outperformance Ratio - the percentage of months over a defined period (one year or more) where excess return is positive. Also calculated for 'up markets' and 'down markets' (defined by the monthly movement in the fund's benchmark).
• Worst drawdown - the largest cumulative 'peak to trough' decline in a fund's total value over a defined period (one year or more). Unlike absolute return, the value is not annualised. Its analysis provides an indication of a fund's riskiness and the extent and frequency to which investors may have to tolerate drawdowns.

Qualitative factors generally account for 80 per cent of the Lonsec rating for most mainstream asset classes, while quantitative factors account for 20 per cent. Quantitative analysis allows greater insight into how funds perform in various markets and whether they have performed according to stated investment objectives.

We consider additional quantitative factors to those above, and each factor's contribution to the 20 per cent weighting may vary depending on asset class or strategy.

Mercer

In researching the quantitative aspects of a fund’s performance, we think the following data analysis should be considered:

• Return data – gauges how a manager's process performs under different market conditions and how a manager may vary its process. This includes factors such as monthly or quarterly excess return, rolling one, three, five and 10-year returns, up/down market returns, returns during specific periods of market stress and sector or style influence.
• Risk data – gauges how well a manager understands and controls various sources of risk in a portfolio and, combined with return data, provides a better understanding of the overall risk efficiency.
  How risk data is used depends on how the portfolio will combine with others in a client’s overall scheme. Examples include volatility, tracking error, beta, sector/stock contribution to active risk, duration, spread duration, net exposure, gross exposure, leverage, portfolio turnover, days to trade to cash and currency exposure.
• Portfolio data - particularly useful in sizing up how well the investment process described matches up with the resulting portfolio and what risk and style factors are at play.
  Clients should consider analysis of equity portfolios firstly, to quantify how aggressively the manager positions the portfolio relative to its benchmark; secondly, to identify what types of bets the manager takes in an effort to outperform its benchmark, and the relative importance of these bets; and, thirdly, to quantify the different types of style biases in the portfolio.
  Examples include: stock, sector, style or market risk exposures, consistency of style or risk exposures versus itself and peers. 

Quantitative analysis is necessary, but it is not sufficient to analyse a fund on its own. Mercer devotes most of its research effort to the qualitative aspects of the analysis. Essentially, quantitative analysis is used to answer questions that arise during the qualitative review of a fund.

Standard & Poor’s Fund Services

The most critical quantitative metrics for advisers are those that relate to the manager's achievement of its investment objective:

The top five measures Standard & Poor’s Fund Services (S&P) considers are detailed below (all use rolling three or five-year periods to avoid ‘end point bias’). 

• Standard Deviation – the degree of fluctuation in a portfolio's return. The higher the standard deviation, the greater the magnitude of fluctuations from average return. Standard deviation assumes that returns are normally distributed, which limits its use for investments with unusual return distributions.
• Tracking Error – the standard deviation of excess return. Tracking error assumes returns are normally distributed. It combines upside and downside risk.
  Consider an index fund that has no excess return relative to its benchmark when measured over a long period, but that produces an annualised tracking error of 10 basis points (0.1 per cent). If the benchmark returns 10 per cent per year, the fund's return should be between 9.9 per cent and 10.1 per cent (10 basis points on either side of the 10 per cent benchmark return) in 68 per cent of the observed one-year time periods (one standard deviation).
  Tracking error is also commonly used to assess an index fund’s success in matching its target index. Active managers that are closely tied to a benchmark might describe expected deviation from the benchmark in terms of tracking error.
  It is less directly relevant for actively managed funds, although used in calculating the information ratio, which is often employed in comparisons of active managers. For benchmark-unaware funds, standard deviation relative to peers or market is more appropriate.
• Information Ratio – the risk-adjusted return versus a benchmark, being excess return divided by tracking error relative to the benchmark. This is typically used to measure a manager's skill versus peers.
  An actively managed fund that has 100 basis points of excess return and 200 basis points of tracking error relative to its benchmark would have an information ratio of 0.5. All else being equal, higher information ratios are preferable. 
• Sharpe Ratio – how much return is being obtained for each theoretical unit of risk, being an asset's excess return versus a risk-free asset divided by the standard deviation of returns.
  Sharpe ratios can be negative if the asset underperforms the risk-free asset. In the longer-term, it generally falls in a range from 0 to +1 - the higher, the better. This can be used to compare investments across asset classes with similar liquidity and valuation characteristics.
• Value At Risk – a portfolio's worst results over a given period, derived from a fixed percentage of the worst observations – the worst 1 per cent or 5 per cent, for example – or a fixed number of those observations. S&P uses it to assess strategies in the alternatives sector.
  If the worst annual return for the stock market in the past 50 years was -45 per cent, the stock market’s one-year value at risk based on roughly the worst 1 per cent of observations is -45 per cent. It’s a useful metric to quantify potential worst loss over a specified time period.

While past performance is important, S&P Fund Services considers how a manager is positioned for future success to be more significant. We use a variety of quantitative metrics to assess what a manager has done well and what they haven't – in effect, creating a quantitative, results-oriented assessment of a manager’s skill-set to support our qualitative research.

In determining our forward-looking ratings, we do not use a fixed weighting to quantitative metrics as it could incorrectly lead to higher ratings on funds that have performed well over recent history, but we believe may not necessarily continue to do so.

Morningstar

Morningstar’s research team studies a host of quantitative analytics when analysing a fund. To restrict the list to only five is a tall order, but the following are some of our favoured metrics (in no particular order).

• Risk-adjusted Rolling Returns – risk-adjusted returns reward consistency and penalise downside risk. They are the bedrock of the Morningstar Rating (star rating) of a fund based on the after-fee returns over three, five and 10-year periods, and assessed against peer groups.
  Importantly, it penalises downside risk against cash more than upside risk.
• Information Ratio – this is another risk-adjusted measure taking into account a fund’s excess return after fees above the market, and the risk taken to achieve it. An information ratio above 0.6 should see a fund in the top quartile of its peers.
• Holdings-based style analysis – this holdings-based score based on the size and value/growth orientation of the underlying stocks in a fund is a great way to understand a fund’s essential portfolio characteristics, especially for blending purposes, but also gives a deeper understanding as to how the fund has evolved through time.
  Analysing the characteristics of a portfolio today, relative to peers, is likely to tell more about future potential risks than five years of return data.
• Indirect Cost Ratio – fees are a constant, which erode returns over time. Many studies have shown that high-fee funds tend to underperform peers and the market. It’s very important to understand the cost equation, which also includes understanding the cost from excessive turnover - both significant headwinds to a fund’s future outperformance.
• Portfolio Manager Investment – while managers in Australia are not required to disclose their investment in a fund, they are in the US, and it’s a very interesting quantitative metric to observe.
  Our US team has analysed the performance of funds with a high portfolio manager investment, and those that do invest in their own funds tend to outperform those that do not, and have lower fees and turnover costs.

While we assess a large number of quantitative metrics when analysing a fund, our qualitative research process which results in a five-point rating scale is 100 percent qualitative. It is designed to be forward-looking, and we do not believe in putting a number on the influence that quantitative metrics have in that process.

It is the subjective analysis of fundamental factors and whether performance matches what’s expected that drives our final view of a fund.

Zenith Investment Partners

While Zenith reviews a wide range of quantitative factors when assessing the quality and appeal of a fund, the top five quantitative factors that practitioners should focus on, in our opinion, are:

• Absolute Return (3+ year timeframe) – ultimately, investors expect a return on their investment. The current environment is an excellent case study in the thought process of retail investors.
  If they feel they can get a competitive return on their money via a cash account or term deposit, they need to be convinced to take on some investment risk, and they must be adequately compensated by a more attractive return. Investors are not so concerned about outperformance to a benchmark if a fund has provided them with an attractive absolute return above that they could have earned in the bank.
  This is not to say that outperformance of the fund’s market benchmark is not important and an important component of its assessment and rating – it is. But ultimately, an investor wants a positive return, irrespective of market conditions over the longer term, so this should be the first assessment criteria.
• Standard Deviation – another critical concern to investors is the volatility of their investment so assessing a fund’s standard deviation is important. It is a more absolute measure of risk, whereas measures such as tracking error mean very little to the end investor and, often, their adviser.
  While downside deviation and/or the Sortino ratio are specific measures of downside volatility, and therefore can be better measures of the kind of volatility that concerns investors, standard deviation is well understood. High levels of standard deviation signal high levels of volatility with which investors are usually not comfortable.
• • Income & Growth Return Split – as a general rule, the industry does not place enough focus on the breakdown of income and growth in a fund’s return. It can tell much about the level of portfolio activity (trading).
  A manager with high levels of portfolio activity is likely to produce higher levels of realised capital gains, which are distributed within income distributions. This is unlikely to suit high marginal tax rate investors where they will pay tax on this income, whereas this type of manager could be appropriate for a super fund investor where the after tax return is not as highly impacted given the super fund’s lower tax rate. This factor is extremely important when selecting appropriate funds for different tax rate investors.
• Cumulative Return Chart – consistency of returns is also important to investors and, while there are a myriad of statistics, one of the easiest is to ‘eyeball’ its cumulative return chart.
  In an ideal world, the shape and slope of the return chart should be upward from left to right without large peaks and troughs. This would indicate the fund has generated consistent, positive returns over time.
• Sharpe Ratio – one of the more commonly known and understood risk/return measures, Sharpe ratio is important in assessing how a fund’s return is generated. It is arguably preferable that a fund generates attractive returns with lower volatility than higher volatility, and Sharpe ratio provides a good measure of this in an absolute sense.
  It reflects the increase in return above the risk free rate of return (cash), relative to the risk of the fund where risk is measured as standard deviation of returns. A Sharpe ratio above 1.0 is an excellent result as it indicates the fund is delivering a greater amount of outperformance for every additional unit of risk.

van Eyk Research

Quant measures are important tools in analysing the performance of fund managers. They enable investors to determine not only how well a manager is performing but what is actually driving that performance, how much of it can be attributed to the skill of the manager, and whether that success is likely to continue. 

The quantitative tools that van Eyk believes are the most important are:

• Three-year rolling excess returns – a period of three years is usually long enough to capture performance covering different conditions in economic and market cycles. Fund managers may exhibit strong or poor relative returns over short periods of time when their style is in or out of favour.
  For instance, a value manager with a quality bias may underperform significantly during a strong upward lift in markets driven by speculation. Three years is generally enough time to assess performance over a variety of market conditions.
  While you can’t necessarily extrapolate past performance into the future, past returns are clearly a strong indicator of manager skill. We find three-year rolling excess returns more useful than cumulative excess returns in deriving more conclusions about the consistency of alpha generation and how this changes over time.
  It is also useful to track how alpha generation has evolved over time where funds under management have significantly increased. 
• Information ratio – the excess return produced by a manager divided by the tracking error (where the latter is a measure of how much returns are likely to differ from the manager’s benchmark). The higher the information ratio, the better the manager has efficiently converted the level of risk taken in a portfolio into returns. Again, returns are best measured over rolling three year periods.
• Attribution analysis – Shows the proportion of a manager’s excess returns from being overweight the right sectors and underweight the wrong sectors (and for international equity managers, the right and wrong regions) and what stocks have contributed the most to that performance. It shows where the manager’s strengths lie and shows whether performance has been broad based or driven by one or two stocks, on the basis that broad-based performance is probably more repeatable.
• Style analysis – examines the underlying portfolio to determine whether positions reflect what you’d expect from the investment process. It can highlight anomalies between what the manager claims is its style and the reality of what is in its portfolio.
  Analysis of active sector positions and market capitalisation biases explains the extent to which performance has been due to structural biases in a portfolio as opposed to active investment decisions.
  If, for example, a manager has a structural bias towards small cap stocks and these have outperformed, that needs to be taken into account. It gives you a feel for whether the manager’s choices are a reflection of considered investment decisions or whether, for example, there might be dominant personalities on the investment team, whose opinions are being given excessive weight, leading to consistent overweights and underweights in certain sectors.
  Deviating from style is not always a bad thing, however, if it can be justified. 
• Assets under management – many investment managers with strong track records attract significant funds under management. But, owning over 0.5 per cent of the free float value of a market seriously impacts a manager’s ability to be nimble or flexible in its investment decisions.
  Often where they have a competitive edge in stock selection, they may only be able to buy or sell just a small portion of what they planned before the pricing inefficiency has been corrected. It is much more difficult to achieve a strong information ratio with high levels of funds under management. 

It is important to remember that quant measures should not be used in isolation. You cannot analyse a fund in full solely using data from your computer screen. Quant measures are most useful when they are used in conjunction with qualitative measures, which must include talking in depth to the people who run the fund.