The ROC curve characterizes the choices available to the doctor. They may set the criterion anywhere, but any choice that they make will land them with a hit and false alarm rate somewhere on the ROC curve. Notice also that for any reasonable choice of criterion, the hit rate is always larger than the false alarm rate, so the ROC curve is bowed upward. Figure 4: Internal response probability of occurrence curves and ROC curves for different signal strengths.
When the signal is stronger there is less overlap in the probability of occurrence curves, and the ROC curve becomes more bowed.
The role of information: Acquiring more information makes the decision easier. Unfortunately, the radiologist does not have much control over how much information is available. In a controlled perception experiment the experimenter has complete control over how much information is provided. Having this control allows for quite a different sort of outcome.
If the experimenter chooses to present a stronger stimulus, then the subject's internal response strength will, on the average, be stronger. Pictorially, this will have the effect of shifting the probability of occurrence curve for signal-plus-noise trials to the right, a bit further away from the noise-alone probability of occurrence curve.
Figure 4 shows two sets of probability of occurrence curves. When the signal is stronger there is more separation between the two probability of occurrence curves. When this happens the subject's choices are not so difficult as before. They can pick a criterion to get nearly a perfect hit rate with almost no false alarms.
Ultimately, if the signal is really strong lots of information , then the ROC curve goes all the way up to the upper left corner all hits and no false alarms. Varying the noise: There is another aspect of the probability of occurrence curves that also determines detectability: the amount of noise. Less noise reduces the spread of the curves. For example, consider the two probability of occurrence curves in Figure 5. The separation between the peaks is the same but the second set of curves are much skinnier.
Clearly, the signal is much more discriminable when there is less spread less noise in the probability of occurrence curves. So the subject would have an easier time setting their criterion in order to be right nearly all the time. Figure 5: Internal response probability of occurrence curves for two different noise levels. When the noise is greater, the curves are wider more spread and there is more overlap.
In reality, we have no control over the amount of internal noise. But it is important to realize that decreasing the noise has the same effect as increasing the signal strength.
Both reduce the overlap between the probability of occurrence curves. Discriminability index d' : Thus, the discriminability of a signal depends both on the separation and the spread of the noise-alone and signal-plus-noise curves. Discriminability is made easier either by increasing the separation stronger signal or by decreasing the spread less noise. In either case, there is less overlap between the probability of occurrence curves. To write down a complete description of how discriminable the signal is from no-signal, we want a formula that captures both the separation and the spread.
The most widely used measure is called d-prime d' , and its formula is simply:. This number, d' , is an estimate of the strength of the signal. Its primary virtue, and the reason that it is so widely used, is that its value does not depend upon the criterion the subject is adopting, but instead it is a true measure of the internal response.
Estimating d' : To recap Increasing the stimulus strength separates the two noise-alone versus signal-plus-noise probability of occurrence curves. This has the effect of increasing the hit and correct rejection rates. Shifting to a high criterion leads to fewer false alarms, fewer hits, and fewer surgical procedures.
Shifting to a low criterion leads to more hits lots of worthwhile surgeries , but many false alarms unnecessary surgeries as well. The discriminability index, d' , is a measure of the strength of the internal response that is independent of the criterion. But how do we measure d'? The trick is that we have to measure both the hit rate and the false alarm rate, then we can read-off d' from an ROC curve. Figure 4 shows a family of ROC curves.
As the signal strength increases, the internal response increases, the ROC curve bows out more, and d' increases. The only statistical knowledge that is required on behalf of the student is a basic understanding of z-scores and the normal distribution.
An alternative exercise for students is to formulate their own research designs that would allow them to investigate pseudoscientific principles using SDT with the following examples as a framework. Either approach would provide students with valuable hands-on experience for using SDT to objectively assess human decision-making. Electronic voice phenomena EVP is a claim by parapsychologists that spirit voices can be detected in random radio noise.
James conducts an experiment with a spiritualist and a skeptic to determine if EVP can be reliably detected by the spiritualist. Each subject is presented with five-second sound clips, where 50 of the sound clips contain a very weak voice signal and the other 50 clips contain random noise.
In each trial, the subjects report whether the sound clip contained a voice. Did Dr. James find that the spiritualist was more sensitive than the skeptic? The proportion of hits made by the spiritualist is 0. Thus the sensitivity of the spiritualist is:. The proportion of hits by the skeptic is 0. Therefore, the spiritualist was not more sensitive to EVP than the skeptics, but did demonstrate a more liberal bias than the skeptic, who demonstrated a more conservative bias.
A psychic detective is consulted by the Metro city police department for multiple missing person cases. Of these cases, 50 bodies are discovered. Before discovery, the psychic claimed that 31 of the bodies would be discovered close to water. After the bodies were discovered, it was determined that 17 of the bodies the psychic predicted would be close to water were within m of a body of water.
On the other hand, 11 of the bodies that the psychic did not claim were close to water were also within m of a body of water. Was the psychic sensitive to the location of the bodies? The proportion of bodies the psychic correctly determined were close to water was 0. Therefore the sensitivity of the psychic is:.
Thus, the psychic was not sensitive in predicting the location of the bodies, but did demonstrate a liberal bias for reporting that bodies would be discovered close to water. The author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
The author would like to thank Dr. Kathleen Corrigall and the reviewer for useful comments and suggestions for earlier versions of this manuscript. Allan, L. A signal detection theory analysis of the placebo effect. Health Prof. Banks, W. Signal detection theory and human memory. Google Scholar. Fisher, C. Using spreadsheets to teach signal detection theory.
Spreadsheets Educ. PubMed Abstract. Garfield, J. How students learn statistics revisited: a current review of research on teaching and learning statistics. Goldstein, E. Sensation and Perception, 9th Edn. Belmont, CA: Wadsworth. Green, D. Signal Detection Theory and Psychophysics. New York, NY: Wiley. Huang-Pollock, C. A primer of signal detection theory.
Readers may have difficulty locating this book. Swets, J. Signal detection theory and ROC analysis in psychology and diagnostics: Collected papers. John Swets, who passed away in , was arguably the most influential proponent of SDT in psychology. This collection of twelve articles he wrote or cowrote over a period of twenty years provides an introduction to the theory that is surprisingly bereft of mathematical details.
Numerous examples of SDT applications in a wide variety of fields are also included. Evaluation of diagnostic systems: Methods from signal detection theory. This book was written to enable comparisons of the relative accuracy of diagnostic devices, particularly those used in medical settings. Thus, it emphasizes issues that are more relevant to engineering and medicine than to psychology. However, the applicability of the material to behavioral concerns is obvious, and the practical examples used to discuss SDT may help readers who struggle with the more abstract approaches used in other general SDT books.
Wickens, T. Elementary signal detection theory. New York: Oxford Univ. As the title indicates, this book explores relatively few extensions of SDT. It aims to provide an introduction to SDT that is thorough and mathematically grounded, but at the same time relatively accessible.
Readers who struggle with mathematics will probably find this text easier to understand than Green and Swets and Macmillan and Creelman , but more challenging than McNicol Users without a subscription are not able to see the full content on this page. Please subscribe or login. Oxford Bibliographies Online is available by subscription and perpetual access to institutions.
For more information or to contact an Oxford Sales Representative click here. Not a member? Sign up for My OBO. Already a member?
Publications Pages Publications Pages. Subscriber sign in You could not be signed in, please check and try again. Username Please enter your Username.
0コメント