Asking the right questions

With the introduction of financial services reform, the ‘know your client’ obligation for financial planners is more critical than ever before.

Yet, despite the legislative and common law requirements to delve deeper into the financial needs and investment behaviour of clients, much of the advice industry continues to rely on somewhat basic risk profiling techniques.

In addition, assessing an individual’s capacity for risk, rather than their attitude to risk, is still given greater priority.

The US Journal of Financial Planning (JFP) recently published a paper on this subject entitled Insights from Psychology and Psychometrics on Measuring Risk Tolerance, which was co-authored by Michael Roszkowski and John Grable, both US-based research academics, and Geoff Davey, provider of a risk-profiling system in Australia.

The report said: “Unfortunately, questionnaires commonly used by financial planners do not adhere to psychometric standards. They are generally too brief (a reliability problem) and contain too many ‘bad’ questions (a validity problem).”

While the authors have not issued a “blanket condemnation of the questionnaire as a method for measuring risk tolerance”, they certainly believe there is room for improvement among the questionnaires most commonly available.

The advent of risk testing

According to Davey, co-founder and chief executive officer of Finametrica, prior to the development of any form of risk profiling techniques, investment advice had “very much the supermarket shopping cart type of approach”.

“It was very product driven — ‘here have one of these, one of those, and one of something else’,” he explains.

“And the idea behind questionnaires, which was quite good really, was that people should be following an investment strategy, and there will be different levels of risk that suit different people.”

The first risk tolerance questionnaires were published in the mid-1980s and were subsequently widely adopted. However, despite their now widespread use, the format has come under scrutiny.

The report noted: “The problem with nearly all so-called risk tolerance questionnaires is that they have been constructed without regard to psychometrics.

“Years ago, it was not uncommon to find questions relating to physical risk tolerance in questionnaires designed to measure financial risk tolerance. Today, the more prevalent problem is that many risk tolerance questionnaires deal with financial matters that are not really part of the construct of risk tolerance,” it added.

Packaged offerings

Currently, the majority of advisers use the risk tolerance questionnaires that are included in their financial planning software package, which then allows them to integrate the results into other functions of the system.

Marisa Broome, director and adviser at Wealth Advice, explains: “We had a standard test on the software we were using — it was very ordinary and very short. And, in fact, I think it was just lip service to the ‘know your client’ rule rather than actually having anything that was at all meaningful.”

According to the report: “Modelling packages often have ‘risk tolerance questionnaires’ built into them, but in most cases they do a shabby job of measuring this construct. Generally, the questionnaires are simplistically short, or they require a level of investment-risk understanding beyond the vast majority of clients.”

There has been some progress among technology providers, however, with organisations such as VisiPlan now offering a psychometric risk tolerance questionnaire as part of its financial planning software package.

The standard model

Wes McMaster, adjunct professor of financial planning at RMIT and expert witness, says standard industry questionnaires are “relatively primitive”.

The main criticism of standard risk tolerance testing is that there is too great a focus on risk capacity, rather than risk tolerance.

“The capacity for risk tolerance is a function of your relative wealth, whereas the behavioural characteristics of people who are placed in different risk circumstances is quite different. And you may have, for example, an extremely wealthy person who has low risk tolerance,” he says.

Davey adds: “Capacity is someone’s financial capacity to bear the loss that might result from taking a particular level of risk. And that’s quite different from psychological capacity. They are both constraints, and need to be kept separate.”

Psychometric tests, on the other hand, predict how a client will actually behave, rather than examine the balance of their bank account.

Standard reports often provide a score for a client, categorising them into one of five risk groups, but offer very little additional information for the adviser.

“You do get a lot more information out of a psychometric report, so you could conceivably talk about the results in the report, and that helps you to know the client,” Davey says.

The science of psychometrics

Psychometric testing, often used in high-level recruitment to determine the behavioural characteristics of a potential candidate, was developed in the late 19th century as a means of assessing a person’s psychology using statistical analysis.

There are now internationally agreed psychometric standards, which ensure that tests:

~ have undergone a rigorous development process;

~ produce results that are both reliable and valid;

~ include questions that can be clearly understood by respondents; and

~ include the proper scoring algorithm, where questions are correctly weighted according to their importance.

McMaster says: “If you are genuinely trying to measure risk tolerance, then I think you have to use a questionnaire that is based on behavioural characteristics, and typically we call that a psychometric-type questionnaire.”

Psychometric risk tolerance tests tend to be longer than the industry standard questionnaires — generally around 25 questions.

The report states: “Think about it — on a five-question test, each question constitutes 20 per cent of the total score. Changing just one answer could put the client into an entirely different risk tolerance category.”

McMaster adds: “If you are looking at matching someone to an asset allocation model, you might be able to do that in five questions, but that doesn’t actually measure their behavioural characteristics when faced with different risk circumstances.”

Compliance support

Regulatory compliance is possibly the single most important reason for using a psychometric-based risk tolerance system.

McMaster, who also acts as an expert witness in financial planning court cases, says: “There are regulatory rules which require them [advisers] to undertake some kind of risk tolerance assessment. ASIC [Australian Securities and Investments Commission] has put out guidelines saying that.”

McMaster firmly believes that using a psychometric test could potentially help an adviser in litigation defend the ‘know your client’ requirement.

“In almost all of the cases [where he has acted as an expert witness] the appropriateness of the advice was an issue. And the appropriateness of advice is determined by a number of characteristics, one of which is the measurement of risk tolerance,” he says.

“So what I would say is that most of the litigation in which I have given an opinion and been involved, the issue of risk tolerance has been material.”

For instance, McMaster says recent Financial Wisdom case was based on the fact that advisers had recommended very high-risk investments, without having measured the risk tolerance of the clients. “However, the characteristics of the clients were such that, had he measured risk tolerance, it is likely that they would have been very conservative investors. My job was to advise the court that in my opinion that was so, and then justify why I thought that was so.

“Now I wasn’t in a position to put those people through a risk tolerance questionnaire, but there were facts about those people that pointed to that. So that’s an example of how the courts typically look at it.”

Broome adds: “I think if a client came to you because their portfolio has been an absolute dog, and they wanted to sue you because they think you have put them in all the wrong investments, that you could actually say ‘we put you in these investments based on this discussion, and these reasons, and this outcome from the testing’. I think you’ve got a better leg to stand on.”

Benefits for advisers

Broome currently uses Finametrica’s online risk profiling system, which was developed in conjunction with the University of New South Wales’ applied psychology unit. She says: “Even if the outcome on both types of tests [industry standard and psychometric] was exactly the same, to get a client to sit down and think about money in the way that the number of questions in Finametrica’s test makes them do is a good thing.

“At least you are starting off by making them focus for a lot longer. I find with a lot of the other industry standard tests that you literally just ask three questions, tick a box, and that’s where you end up.”

In many cases, advisers are forced to complete the tests with their clients, as the questions need to be explained to the individual. Because psychometric tests have been designed in ‘plain English’, clients can complete them on their own. This also ensures the results are based on the individual’s answers, not the adviser’s interpretation of their responses.

It is also a valuable information source for Broome, as it can highlight a client’s gradual increase in financial knowledge.

“I re-test my clients because I use it as an education tool. It’s actually a pretty good way of just seeing where clients are developing,” she says.

And with a young client base, made up of individuals “keen to know markets and get to know and understand how to be an investor”, Broome says her clients “like doing the test, they like seeing how they have progressed, and whether things have actually changed”.

Finding the time

Because psychometric tests tend to be much longer than an average industry standard questionnaire, financial advisers have been reluctant to adopt them, fearing lengthy testing will place more burdens on their already time-strapped day.

But because the tests are portable, in that clients can take them home to complete, advisers could actually spend less time on risk profiling.

In addition, Davey says: “There’s certainly less time spent subsequently because the adviser has a better understanding of the client, the client can see that’s the case, and therefore it can be quicker to get a properly informed commitment to the plan.

“You can get a positive decision quicker, and there’s less servicing because the clients understand better what it is they’ve done — there’s not that follow-up phone call or meetings.”

The report states: “Financial planners who seek a five to 10 question test that is 100 per cent accurate will be disappointed, because no such instrument can ever be developed.”

The future of risk profiling

So will we see a more scientific approach to risk tolerance testing in the future?

Broome is not convinced.

“I would hope so, but I think the argument falls on deaf ears still. I still don’t think there is a belief that there is any value in it. Every time we go to a conference it comes up as another debate topic, and really, I think we should be getting on to deeper things now.”

Davey is more confident, however, perhaps because he has seen increased interest in psychometric testing in the US.

“The recent bear market was much more severe in the US, so it really woke people up to issues about risk. Also, the CFP Board exam in the US is, in fact, a psychometric test, and so the planners who have worked on any CFP Board committees are aware of that,” he says.

Davey believes we will see more psychometric solutions available on the market — partly driven by increasing compliance regulations, but also by an increased understanding that financial planning “is about the person, not just the person’s money”.

He adds: “It should be about the journey, not just the destination, so you need to have an understanding of how your clients are going to feel about things, and be able to demonstrate to prospective clients that that’s the way you operate.”