Expert opinion (further discussion)

After having determined that there is a clear need to rely upon expert opinion, the following are some key points to contemplate when encountering those who profess expertise (or others do so on their behalf) in discussions about a particular claim or claims in the public domain:

1.     Public figures and others who almost certainly are not experts: well-known or newsworthy people presenting themselves (or being presented) as being able to speak with expertise about a claim, for example, a ‘super-model’ or ‘movie star’ is used in an advert in such a way as to intimate that the celebrity is in a position to make authoritative statements about the effects of the maker’s product or a charity’s or government’s community assistance programme (and the prospects of these helping consumers’ well-being in some way), when the model or movie star holds no credentials which qualify them to speak with expertise on the matter;

2.     Credentialed people whose expertise is not in a field relevant to the claim, for example, an award-winning research chemist makes statements about taking vitamins to cure or act ‘prophylactically’ against viral illnesses but we are offered no good reason to think that this person with expertise in a different field of inquiry has secured the knowledge and skill required to speak with authority in the area relevant to the current claims – on some occasions there appears to be a distinct disingenuousness perpetrated in as much as a person may be paraded as ‘Dr’. So-and-so where the required expertise pertinent to the claim calls for, say, a (particular kind of) specialist medical doctor, when in fact the person is not a medical doctor (rather absurdly they might be a dentist or a doctor of philosophy), or at least does not even hold a licence to practice in the discipline to which the claim relates;

3.     Retired or ‘lapsed’ professionals who may have in the past possessed expertise in the required area but whose expertise no longer seems current and is therefore now questionable, that is, subsequent findings and new or recent developments have changed the complexion of the field to the extent, and especially with reference to the particular claims of interest to us, that the person’s old experience – they may have lost contact with the field for a significant period while pursuing other interests – or education completed years before may not now provide sufficient weight in favour of their expertise regarding the claims – it is not at all clear whether the person is up-to-date and conversant with the latest discoveries and progress in the area;

4.     Experts paid for their testimony in favour of a claim or claimant, for example, a research scientist engaged by a tobacco company to undertake studies into risks concerned with cigarette-smoking who is also willing to spruik his company’s claims to the effect that smoking cigarettes is not significantly harmful to the health of consumers – even if a researcher’s own results suggest to him or her that this might be the case the genuine expert very carefully considers the wider picture of research findings and also whether sufficient research has been undertaken to test and substantiate the company’s or industry’s claims, also being at pains to take note of and to understand what peer review has to say about the adequacy and validity of their own research and to formulate credible responses to these reviews and criticisms – a key element of what is being discussed here is dependability, unfortunately this trait may be tarnished or obliterated by being ‘bought’ – such paid or ‘interested’ opinions should be treated with caution;

5.     Those who have developed extensive knowledge of a subject but who retain distinct biases†: people can feel strongly about particular issues and having such strong feelings will also have their points-of-view on those issues – expert opinion will only be credible to the extent to which the person expressing it constrains any biases they may harbour on the matter(s) from unduly distorting their judgement – intense biases are likely to have an adverse bearing on the person’s ability to deal with claims in such a way as to be judged competent and in possession of genuine expertise – however, it may not always be clear or apparent immediately that such biases are ‘in play’ so this component should be borne in mind when we weigh the expertise of advocates for various (contentious) public policies, so that we clearly understand whether they should be considered to possess genuine expertise in the subject area and that we don’t (lazily) attribute to the positions they are recommending or supporting more credibility than is really warranted.

So:

£Is the person recognized as having expertise in the field in which he or she is offering testimony? Have we adequately identified the fields to which the claim relates? What precisely are the ‘expert’s’ credentials and in what way explicitly do they constitute expertise with this particular claim and subject?

£Is the expert an independent party to the issue or claim with which we are concerned?

£Does the person possess expertise in a separate field of knowledge, and not in the area which is required for the matter at hand?

£Has the person applied any widely-accepted and validated methods of analysis from the relevant domain in their assessment of the issue and the opinion offered?

For someone to be considered an expert they must have demonstrated a facility to correctly interpret all of the applicable data and to have reached conclusions about the evidence which were justified, taking into account such factors as the completeness or adequacy of the available evidence – their ‘track record’ should be scrutinized and assessed.

The reliability of expert opinion has a relation to the degree of consensus in the field, which at least in part turns upon how much controversy there is in the domain about the issue or claims involved. Where there is widespread disagreement among experts or there is considerable uncertainty in the field relative to the claim or claims we should exercise restraint – in these circumstances it invariably spells trouble to rely upon a ‘maverick maven’s’ opinion in the face a conspicuous absence of agreement among independent experts – especially if the claim is one which has been hotly contested in the domain.

When we see ‘experts’ discussing topics on television we need to ascertain or identify the standard of expertise which is being applied. It may be that the television network has a more lenient interpretation of what constitutes relevant expertise for a particular claim or subject than do we, and indeed the same network may apply differing standards across a variety of subject areas (or news programmes) or different units of the network may have a disparate understanding of expertise even when dealing with the same subject matter, and of course competing or rival networks may well adhere to starkly dissimilar benchmarks for determining ‘expertise’ – so beware! And then there is the internet...

To state what seems to be obvious: it is not always easy to evaluate the credentials of experts, and what will be good credentials will vary from field to field, and so with it what constitutes genuine expertise for the particular claims we are interested in on any given occasion. Still if we don’t undertake some modest efforts with regard to identifying real expertise we will find ourselves ‘taking advice’ from people whose opinion may be little better than our own unaided (and unqualified?) judgement, while deceiving ourselves with “well, this is what the experts say!”

(see also: self-correction, pseudosymmetry, Expert Opinion, Internet and information, what is 'expert opinion'?, intelligence and belief, belief and bias)

† for instance, a researcher may have accumulated considerable knowledge about Hitler's Third Reich - and in fact may even be a 'fan' - and show bias to the extent that they deny the Holocaust took place.