The covariance decomposition of the probability score and its use in evaluating prognostic estimates. SUPPORT Investigators.

Arkes HR, Dawson NV, Speroff T, Harrell FE, Alzola C, Phillips R, Desbiens N, Oye RK, Knaus W, Connors AF
Med Decis Making. 1995 15 (2): 120-31

PMID: 7783572 · DOI:10.1177/0272989X9501500204

The probability score (PS) or Brier score has been used in a large number of studies in which physician judgment performance was assessed. However, the covariance decomposition of the PS has not previously been used to evaluate medical judgment. The authors introduce the technique and demonstrate it by analyzing prognostic estimates of three groups: physicians, their patients, and the patients' decision-making surrogates. The major components of the covariance decomposition--bias, slope, and scatter--are displayed in covariance graphs for each of the three groups. The decomposition reveals that whereas the physicians have the best overall estimation performance, their bias and their scatter are not always superior to those of the other two groups. This is primarily due to two factors. First, the physicians' prognostic estimates are pessimistic. Second, the patients place the large majority of their estimates in the most optimistic category, thereby achieving low scatter. The authors suggest that the calculational simplicity of this decomposition, its informativeness, and the intuitive nature of its components make it a useful tool with which to analyze medical judgment.

MeSH Terms (12)

Bias Data Interpretation, Statistical Decision Making Discriminant Analysis Humans Judgment Patients Physicians Probability Prognosis Sensitivity and Specificity Survival Analysis

Connections (1)

This publication is referenced by other Labnodes entities: