Comments on: Alternative measures to the AUC for rare-event prognostic models http://blog.markus-breitenbach.com/2010/02/16/alternative-measures-to-the-auc-for-rare-event-prognostic-models/ AI, Data Mining, Machine Learning and other things Sun, 20 May 2012 12:21:20 +0000 http://wordpress.org/?v=2.2.1 By: Markus http://blog.markus-breitenbach.com/2010/02/16/alternative-measures-to-the-auc-for-rare-event-prognostic-models/#comment-26753 Markus Fri, 26 Feb 2010 17:40:09 +0000 http://blog.markus-breitenbach.com/2010/02/16/alternative-measures-to-the-auc-for-rare-event-prognostic-models/#comment-26753 The x-axis is just the score-value. The plot is supposed to show that the two classes overlap and can not be perfectly separated by the classifier. The x-axis is just the score-value. The plot is supposed to show that the two classes overlap and can not be perfectly separated by the classifier.

]]> By: steffen http://blog.markus-breitenbach.com/2010/02/16/alternative-measures-to-the-auc-for-rare-event-prognostic-models/#comment-26744 steffen Fri, 26 Feb 2010 10:47:44 +0000 http://blog.markus-breitenbach.com/2010/02/16/alternative-measures-to-the-auc-for-rare-event-prognostic-models/#comment-26744 Thanks for sharing this case study (including your thoughts). I have messed around with AUC and the Calibration-Refinement / Sharpness - measures also. A conclusion from various papers was, that Logistic Regression in general delivers well calibrated probabilities. Hence it was interesting to see the opposite can happen when only a small base rate is given. Side Note A: I did not understand the risk-score density plot. What is the x-axis ? Side Note B: http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf is an excellent study of roc and auc in general. thanks again for this post. Blogs with technical and detailed data mining content are hard to find. Steffen Thanks for sharing this case study (including your thoughts). I have messed around with AUC and the Calibration-Refinement / Sharpness - measures also.

A conclusion from various papers was, that Logistic Regression in general delivers well calibrated probabilities. Hence it was interesting to see the opposite can happen when only a small base rate is given.

Side Note A: I did not understand the risk-score density plot. What is the x-axis ?

Side Note B: http://home.comcast.net/~tom.fawcett/public_html/papers/ROC101.pdf is an excellent study of roc and auc in general.

thanks again for this post. Blogs with technical and detailed data mining content are hard to find.

Steffen

]]>