The Messenger: How do you measure the quality of advice?

I’m not sure what it is about financial planning that makes us subject to far more attempts to measure quality than, say, the legal, accounting or medical professions.

TheAustralian Securities and Investments Commission(ASIC), theFinancial Planning Association(FPA), both with the Certified Financial Planner (CFP) and its quality assessment program, the Australian Consumers’ Association (ACA), reportedly Standards Australia, and a commercial enterprise Adviser Ratings, all want to play a role in this.

While lifting industry standards is critical, we need to ensure that we do not become subject to a range of separate, time-consuming or costly procedures all designed to achieve the one end.

Without doubt it would be a very good thing for both the investing public and superior planners if there was a widely available, objective measure of the quality of advisers.

On the other hand, it would be undesirable if an assessment purported to measure quality but, in reality, failed to do so. It would also be undesirable for any business to emerge as an industry gatekeeper, imposing another clip on the ticket.

Rainmakeris planning to launch a SelectAdviser service, designed to refer investors to advisers and charging a fee for being in the service. Adviser Ratings also offers this.

In fact, Adviser Ratings attempts to do more than this. It ranks advisers as ‘Not Rated’, ‘Preferred’ and ‘Premium’, that is, it forms value judgments, which it attempts to make “objective and independent”.

The question for all attempts to assess quality is whether financial planning quality be objectively measured?

The ASIC/ACA survey evaluated initial plans but ignored ongoing service — a very partial test.

There are many elements to financial planning. Obtaining a sound, long-term investment return is only one.

There is no objective quality measure of investment advice (other than hindsight). However, many consumers will imagine that highly rated or qualified advisers should be superior in this. Yet being a CFP did not stop some planners from recommending technology funds in 2000.

Adviser Ratings, for example, states that it does not attempt to second-guess the quality of advice, but rather “the infrastructure (personal and corporate) that surrounds an adviser”, that is, the means at their disposal to deliver superior advice.

While this is as much as can be reasonably attempted, it can’t provide much certainty about the prospects of superior or inferior investment returns, which consumers may rank as vital. What needs to be managed, therefore, by all of these organisations, is consumer expectation about what their accreditation means.

David Williams is reported as stating that Standards Australia will be attempting to establish reasonable benchmarks in which consumers can be confident. In some areas this should be simple, like disclosure. In many others it would seem to be impossible, such as processes to develop investment advice or superannuation strategy.

Organisations like Standards Australia have traditionally dealt with goods rather than services, where establishing quality standards is easier, which is not to say no assessment of services is possible. (Note that Standards Australia doesn’t rank product providers, but sets benchmarks.)

One danger for all qualitative services is if subjective opinion becomes included among “objective” criteria. Paul Resnik, one of the key individuals in Adviser Ratings, is also part of the ProQuest risk profiling business. Risk profiling is considered by Adviser Ratings. The merits of this practice is a hotly debated subject and I am one of the skeptics. This highlights the difficulties in making “objective” evaluations.

There are several reasons why planners might choose to qualify for an accreditation or ranking. If it involves training (for example, CFP) it can enhance skills. Some may feel it adds credibility in the eyes of prospects to receive “external verification”. Others will not feel the need for this.

Should it ever happen that any accreditation or standard obtained such clout in the marketplace that it was necessary for business success, we would all need to achieve it. That accreditation would have become an industry gatekeeper.

The prospect of any rating or accreditation gaining significant public recognition is probably a function of marketing budget. In this respect, the FPA would appear most likely to have success with the CFP. The challenge for that organisation is to overcome the perception that some planners who achieve it give poor advice and service. The same challenge will develop for all accreditations over time, because it is so difficult to measure quality.

The problem of commercial gatekeepers has been highlighted by the role of research houses. This still exists today, but was especially strong in the early days of their existence. Several had an enormous influence over fund manager inflow. Hindsight has shown that, in many cases, the research houses’ rankings did not objectively reflect fund management quality.

Many quality managers expressed frustration that their acceptance by advisers was dominated by research house judgments, which they considered to be either arbitrary or influenced by conflicts of interest.

It would be highly undesirable for the financial planning industry to find itself in a similar position. Should this occur, one would expect the gatekeeper to impose a toll, that is, to extract a slice of the industry’s revenue.

An obvious rationale for being part of a referral service is to obtain quality leads. It will be a simple matter for participants to measure whether the business generated justifies the costs. Presumably any prospective investors will prefer the highest rated advisers, so it remains to be seen how many leads might be generated for others.

In my view there are certain measurable factors that would indicate a superior financial planner. Yet others would disagree with my criteria. Other aspects simply cannot be objectively evaluated, such as whether a planner has the personal skills to reassure a client during volatility. This is an unavoidable reality for any evaluation process.

What influence an accreditation may have on prospective clients will depend on how it is described by the planner. Where the organisation does not believe its accreditation guarantees good advice, they will need to police misrepresentation.

This raises a possible legal risk for the rater. They will not want any aggrieved investor suing them on the grounds that a reasonable person would be entitled to expect that a high ranking meant they would get good advice. This is more a danger where any accreditation implies planners are superior, more than where it simply reflects training and experience levels.

The challenge for all accreditations, including the CFP, is this: if it is not an indicator of the likelihood of achieving better investment returns, strategic advice, value for money — what is it?

One issue in regard to accreditors is whether they imply that a decision to not use their service or meet their standards reflects badly on a planner.

As a business that has not seen value in our planners being rated, I must say I feel uncomfortable with the Adviser Ratings’ web site treatment of this.

According to its site: “We know there are companies that are bothered by the questions that advisers are required to answer. They would rather the public were not aware of the answers to some questions.”

While there may be dealers about whom this suggestion is true, it is not the only possible explanation for choosing not to buy a service.

First-rate financial planners have absolutely no fear of high standards and attempts to measure this can play a positive role — but measuring real quality is not easy.