Summary of journal club discussion - Coste and Pouchot, 'A grey zone for quantitative diagnostic and screening tests'

Thanks to those who contributed to today's journal club. Here is a short summary for anyone who couldn't make it.

We started the discussion by considering what clinicians do when they conduct a quantitative test. In general practice, it is rare for GPs to make a diagnosis based on one test result; often clinicians use multiple sources of information like signs, symptoms or other tests. In some cases, such as PSA testing, where there is a large grey zone, the doctor may sometimes ask what the patient wants to do based on a test reading (or series of readings).

One of the ideas we had was the the grey zone depicts the unacceptable levels of false positives or false negatives. This needs to be balanced against the harms occuring from each test/diagnosis based on the false positives or negatives. This led to the idea of the grey zone as giving us an idea of the value of a test; i.e. the proportion of people for whom you can get a conclusive result or not.

We thought a bit about how you would construct the grey zone for a specific diagnostic or screening test, and wondered if this could be achieved through clinical Delphi consensus, a systematic review of the literature, etc. One member said that she was involved in a study where patients were asked to set the grey zone, and to trade off how many false negatives were acceptable in order to gain one true positive. We agreed that a patient-set grey zone would be different than a clinician's, as patients tend to be more tolerant of false positives.

Beth Shinkins kindly prepared some discussion questions to go with this paper which are posted below:

Discussion Points
1) Coste and Pouchot extend the standard binary positive/negative test result framework to include an intermediate range of values where the diagnostic test is unable to determine disease status with any certainty. This is based on the argument made by Feinstein (1990) that the use of a single threshold is not representative of the reality of clinical decision making.
a.Statement by Battaglia and Pewsner (2003): “In practice clinicians hardly ever interpret results of continuous tests as being only ‘normal’ or ‘pathological’. They always take into consideration ‘how positive’ or ‘how negative’ the result is.” Battaglia and Pewsner (2003) – Do you agree with this statement? If so, why do we insist on using a binary framework if these interpretations do not translate into day-to-day practice?
b.Do you agree with the concept of classifying people as diseased or not diseased? Or are we talking more about people who require treatment/monitoring/therapy etc. as opposed to those who don’t?

2)“Pre-test probabilities may vary according to the epidemiological context, the care facility, information already gathered about diagnostic risk factors, and other factors; furthermore ‘subjective probabilities’ produced by clinicians or experts may be unreliable”
a.Does this completely undermine the clinical usefulness of the methodology?
b.How comfortable do you think clinicians are in using measures such as likelihood ratios to describe the accuracy of a test?

3)Excluding those in the ‘grey’ zone from diagnostic accuracy measures such as sensitivity and specificity are meaningless in isolation and potentially very misleading.
a.Do you agree? How could this be overcome?
b.How can we determine what proportion of the sample falling into the grey zone is acceptable?

4)Coste and Pouchot spend very little time explaining how a doctor should proceed in the face of an intermediate test result.
a.What do you think a doctor should do/is currently being done? Repeat the test? Use an alternative test?
b.Lemoine (2009) questions the assumption that a patient is either ‘diseased’ or ‘not diseased’ claiming that “the uncertainty is therefore always attributed to the measurement or to the test procedure itself” rather than “the qualitative complexity of a biological situation”. Coste and Pouchot do not discuss the possibility that an intermediate test result may be an indicator of a subclinical or early stage of disease – is this an important oversight?

5)“The proposal to enrich the interpretation of test results by measuring continuous parameters in shades of grey is an important step in the right direction. The introduction of different shades of grey may help to improve the interpretation of diagnostic test results and, more importantly, improve clinical outcomes.” Battaglia and Pewsner (2003)
a.Do you agree? How would this translate into clinical practice?