Comments on: ISC on the Future of Anti-Virus Protection http://blog.markus-breitenbach.com/2008/08/01/isc-on-the-future-of-anti-virus-protection/ AI, Data Mining, Machine Learning and other things Sun, 20 May 2012 11:53:00 +0000 http://wordpress.org/?v=2.2.1 By: Markus http://blog.markus-breitenbach.com/2008/08/01/isc-on-the-future-of-anti-virus-protection/#comment-9569 Markus Wed, 06 Aug 2008 04:11:21 +0000 http://blog.markus-breitenbach.com/2008/08/01/isc-on-the-future-of-anti-virus-protection/#comment-9569 Yes, in my opinion using the AUC for ill-balanced datasets represents the true performance the best. It summarizes it all in one number - how do you decide which model is better when comparing 2x2 matrices? A nice summary into all sorts of measures (AUC, Brier scores etc.) and how to determine how well a model actually works can be found here: http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html#Methods%20for%20dichotomous%20forecasts Yes, in my opinion using the AUC for ill-balanced datasets represents the true performance the best. It summarizes it all in one number - how do you decide which model is better when comparing 2×2 matrices?

A nice summary into all sorts of measures (AUC, Brier scores etc.) and how to determine how well a model actually works can be found here: http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html#Methods%20for%20dichotomous%20forecasts

]]> By: RB http://blog.markus-breitenbach.com/2008/08/01/isc-on-the-future-of-anti-virus-protection/#comment-9550 RB Tue, 05 Aug 2008 09:08:04 +0000 http://blog.markus-breitenbach.com/2008/08/01/isc-on-the-future-of-anti-virus-protection/#comment-9550 As an aside: Would you be suggesting AUC for evaluating performance on ill-balanced testsets in general? How does this information differ from the common 2x2 matrix of classified vs true labels? As an aside: Would you be suggesting AUC for evaluating performance on ill-balanced testsets in general? How does this information differ from the common 2×2 matrix of classified vs true labels?

]]>