IPT Journal - Appendix A

Appendix A

Description of the MnSOST-R Scales

The Minnesota Sex Offender Screening Tool – Revised (MnSOST-R) is an example of what are called actuarial scales. Actuarial scales are most commonly associated with insurance companies as statistical devices for assessing risks of a particular insured. Statistical models and computer programs sort through a myriad of data on, for example, automobile drivers (e.g., age, sex, occupation, type of car, locale, income level, etc.) to determine what factors relate to or predict auto accidents and with what strength. Thus, insurance companies know that being male and under the age of 25 are high predictors of auto accidents, while other factors are less useful in predicting accidents (such as grades in school).

Psychologists have similarly sought models that predict human behavior and have, for over 40 years, known that statistical models match or, more frequently, exceed, clinical judgment in accuracy about human behavior.¹ Not surprisingly, with states seeking to identify which sex offenders are at greatest risk to reoffend, some psychologists undertook to analyze statistically viable predictors of sex offending behavior. The MnSOST-R is the most comprehensive and reliable of these attempts. The researchers who developed the MnSOST-R (Douglas L. Epperson, Ph.D. of Iowa State University, Stephen J. Huot, M.Ed. and James D. Kaul, Ph.D. both of the Minnesota Department of Corrections) collected criminal and psychological records of 256 released sex offenders.² From the data obtained, Epperson, et al generated 30 factors on which each offender was rated. Next, computers statistically analyzed how well each factor discriminated between the 90 released sex offenders who subsequently were rearrested for a new sex offense within 6 years, and the 166 who committed no new offenses.

Based on several computer runs through the data, 16 items were eventually included in the scale as most discriminatory. The analysis also permitted the researchers to assign relative weights to some factors which were shown to have higher predictive validity than others.

Notwithstanding the extensive number-crunching used to develop the scale, the computer models were unable to generate a configuration of factors that accounted for any more than 20% of the factors that contribute to recidivism. Thus, 80% of what contributes to reoffending has yet to be identified. In the end, the MnSOST-R, while the most accurate actuarial scale available for assessing reoffense risk, yields an r³ value of .45. While this r value is three times better than another commonly used scale, the RRASOR (which, on the same sample of 256 offenders, yielded an r of .13 — a rate barely better than chance), and more than 4 times higher than the r of .10 for clinical judgment found by Hanson and Bussière (1998), it nevertheless is considered to fall in the moderate range of predictive accuracy.

The primary risk associated with actuarial scales of less than high predictive validity is the problem of false positives. A false positive occurs when one classifies an individual who would not have committed a new sex offense as likely to commit a sex offense. The risk of false positives is quite high, even with the MnSOST-R. The difficulty in using the MnSOST-R for predictions is complicated by the fact that its base rate of 35% recidivism, as compared to a base rate in the range of 15% as found by Hanson and Bussière (1998) and others, greatly underestimates the risk of false positives. Assuming a 15% recidivism rate and a population of 1000 released sex offenders, and using the cut score of 8 on the MnSOST-R that NJ currently employs, will result in classifying appropriately 100 of the 150 actual recidivists as SVPs, but will false identify as SVPs, 120 of the 850 who would not reoffend. This is a false positive rate in excess of the true positive rate (see Figure 1 below).

Actual Recidivism

1000 Offenders — 15% reoffense rate

MnSOST-R
Prediction

(Prediction based on obtaining a score of 8 or higher)

	Reoffenders	Non-Reoffenders
Predicted Reoffenders	True Positive 97	False Positive 123	220⁴
Predicted Non-Reoffenders	False Negative 53	True Negative 727	780
	150	850

Figure 1
MnSOST-R Predictions
Based on Cut Score of 8

NOTES ON FIGURE 1: Assuming a 15% Recidivism Rate, Epperson, et al predicts that 44% of those evaluated on the MnSOST-R who obtain a score of 8 or higher will reoffend. Approximately 22% of the MnSOST-R norm sample obtained a score of 8 or above. Using a hypothetical sample of 1000 soon-to-be-released sex offenders, one can then determine that 220 of them (22% of 1000) will obtain a score of 8 or higher. Thus, 44% of that group of 220 offenders will reoffend, or 97 individuals. That leaves 123 individuals (or 56%) who have been identified by the MnSOST-R, but who will not reoffend: the False Positive Group. Since we know that 150 offenders will actually reoffend (based on the 15% recidivism rate), we can subtract the True Positive Group of 97 from 150 to determine that the MnSOST-R will fail to identify 53 individuals who will reoffend. Finally, knowing that 850 offenders will not reoffend, we can subtract the False Positive rate of 123 from 850 to determine that 727 individuals were correctly identified by the MnSOST-R as not likely to reoffend. The resulting comparison of True to False Positives shows that for every 3 individuals the state correctly commits, it will erroneously commit 4 individuals.

Thus, the risk of false positives is unacceptably high (56% of those identified). In fact, inasmuch as the 15% recidivism rate is more in line with actual recidivism research than the 35% used on the MnSOST-R, Figure 1 more accurately reflects the risk of error with the MnSOST-R as it is used in New Jersey.

A cut score of 13 applied to the same population, again assuming a 15% recidivism rate, while decreasing the false positive rate, still only brings it back to the false positive rate achieved using the cut score of 8 and a 35% recidivism rate. As Figure 2 illustrates, the MnSOST-R will still incorrectly classify 30% of the offenders as recidivists, albeit better than the 56% error rate at the cut score of 8, but still unacceptably high.

Actual Recidivism

1000 Offenders — 15% reoffense rate

MnSOST-R
Prediction

(Prediction based on obtaining a score of 13 or higher)

	Reoffenders	Non-Reoffenders
Predicted Reoffenders	True Positive 42	False Positive 18	60⁵
Predicted Non-Reoffenders	False Negative 108	True Negative 832	9480
	150	850

Figure 2
MnSOST-R Predictions
Based on Cut Score of 13

NOTES ON FIGURE 2: Assuming a 15% Recidivism Rate, Epperson, et al predicts that 70% of those evaluated on the MnSOST-R who obtain a score of 13 or higher will reoffend. Approximately 6% of the MnSOST-R norm sample obtained a score of 13 or above. Using a hypothetical sample of 1000 soon-to-be-released sex offenders, one can then determine that 60 of them (6% of 1000) will obtain a score of 13 or higher. Thus, 70% of that group of 60 offenders will reoffend, or 42 individuals. That leaves 18 individuals (or 30%) who have been identified by the MnSOST-R, but who will not reoffend: the False Positive Group. Since we know that 150 offenders will actually reoffend (based on the 15% recidivism rate), we can subtract the True Positive Group of 42 from 150 to determine that the MnSOST-R will fail to identify 108 individuals who will reoffend. Finally, knowing that 850 offenders will not reoffend, we can subtract the False Positive rate of 18 from 850 to determine that 832 individuals were correctly identified by the MnSOST-R as not likely to reoffend. The resulting comparison of True to False Positives shows that for every 7 individuals the state correctly commits, it will erroneously commit 3 individuals.

The MnSOST-R represents a major advance in the ability to quantify the factors that predict sex reoffending. Notwithstanding its improved reliability over clinical judgments, uninformed and uneducated acceptance of the scores it yields can easily lead to overconfidence in the decisions derived from it. Recalling that the MnSOST-R has only identified 20% of the factors that go into predicting sexual reoffending behavior, the legal system must recognize the limitations of the scale when relying upon it to make life altering decisions.

This holds even more true when less reliable scales are utilized such as the Static 99 or the RRASOR. Especially in consideration of the more reasonable recidivism rate of 15%, using the cut score of 8, as preferred by the State of New Jersey, will result in more individuals who would not reoffend being committed, than actual high risk offenders. Use of appropriate cut scores and consideration of appropriate base rates can go a long way to reducing the risk of erroneously committing individuals who are ready to return to society.

Footnotes

¹ Grove, W. M. & Meehl, P.E. (1996). Comparative efficiency of formal (mechanical, algorithmic) and informal (subjective, impressionistic) prediction procedures: The clinical/statistical controversy. Psychology, Public Policy, and Law, 2, 293-323. [Back]

² The sample was disproportionately loaded with a higher percentage of high-risk offenders and offenders who were known to have reoffended since release, to ensure that enough reoffenders were present in the sample to meet the needs of the statistical analysis. [Back]

³ r is a statistical coefficient of reliability that ranges from –1 to 1 where either –1 or 1 equals perfect prediction and 0 indicates chance or random prediction. The closer to 0, the less reliable the predictions. [Back]

⁴ Based on MnSOST-R prediction that 22% of a screened population will obtain a score of 8 or higher. The MnSOST-R predicts that 44% of this group will actually reoffend, assuming a 15% recidivism rate. [Back]

⁵ Based on MnSOST-R prediction that 6% of a screened population will obtain a score of 8 or higher. The MnSOST-R predicts that 70% of this group will actually reoffend, assuming a 15% recidivism rate. [Back]