- Advertising (1)
- Artificial Intelligence (AI) (13)
- Classification (3)
- Clustering (1)
- Coding / Programming (8)
- Cryptography (1)
- Data Mining (22)
- Economy / Investing (1)
- ewrt linux (2)
- Fixing Stuff (8)
- Machine Learning (32)
- Math (2)
- Politics (3)
- Predictive Modeling (5)
- Psychology (3)
- Ramblings (26)
- Random (9)
- Security (16)
- Society (13)
- Sociology (4)
- spam (3)
- Statistics (20)
- January 28, 2012 4:56 pm: Will 2012 be the year of Big Data?
- August 14, 2011 10:41 pm: UK plans to exempt data mining from copyright laws
- June 21, 2011 3:26 am: Risk Assessment of Rare Events in adversarial Scenarios
- March 26, 2011 7:57 pm: How Kinect body tracking works and how Machine Learning helped
- March 1, 2011 11:58 am: European Court of Justice ruling (indirectly) on what cannot be used in Insurance Risk Models
- December 11, 2010 8:35 pm: Mining of Massive Datasets
- December 4, 2010 2:28 pm: Ideas on communicating risks and probabilities to the general public
- October 17, 2010 5:48 pm: Birthday Paradox
- August 5, 2010 1:06 am: Elo Scores and Rating Contestants
- July 11, 2010 8:56 pm: GraphLab & Parallel Machine Learning
Blogroll
Uncategorized
Useful Links
- January 2012
- August 2011
- June 2011
- March 2011
- December 2010
- October 2010
- August 2010
- July 2010
- June 2010
- February 2010
- January 2010
- November 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- August 2008
- July 2008
- June 2008
- May 2008
- April 2008
- March 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- August 2007
- July 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
- August 2006
ISC on the Future of Anti-Virus Protection
An article on the Internet Storm Center discusses wether Anti-Virus software in the current state is a dead end. In my opinion it has been dead for quite a while now. Apart from the absolutely un-usable state that anti-virus software is in, I think it’s protecting the wrong things. Most attacks (trojans, spyware) nowadays come through web-browser exploits and maybe instant-messenger (see reports on ISC). So instead of scanning incoming emails, how about a behavior blocker for the web-browser and the instant messenger? There are a couple of freeware programs (e.g. IEController [German]) out there that successfully put Internet Explorer, etc. into a sandbox; whatever Javascript exploit - known or unknown - the browser won’t be able to execute arbitrary files or write outside its cache-directory. Why is there nothing like that in the commercial AV packages?
However, a few possibilities suggested in the article might be worth exploring. For example, they suggest Bayesian heuristics to identify threats. Using machine learning techniques might be a direction worth exploring. IBM AntiVirus (maybe not the current version anymore) has been using Neural Networks with 4Byte sequences (n-grams) for bootsector virus detection.
A couple things to keep in mind, though:
- Quality of the classifier (detection rate) should be measured with Area-under-ROC-Curve (AUC), not error-rate like most people tend to do in Spam-Filter comparisons. The base-rate of the “non-virus” class is pretty high; I have over 10.000 executables/libraries on my windows machine. All (most?) of them non-malicious.
- The tricky part with that is the feature extraction. While sequences of bytes or strings extracted from a binary might be a good start, advanced features like call-graphs or imported API-calls should be used as well. This is pretty tricky and time-consuming, especially when it has to be done for different types of executables (Windows scripts, x86-EXE files, .Net files etc.). De-obfuscation techniques, just like in the signature based scanners, will probably be necessary before the features can be extracted.
- Behavior blocking and sandboxes are probably easier, a better short-term fix, and more pro-active. This has been my experience with email-based attacks as well back in the Mydoom days when a special mime-type auto-executed an attachment in Outlook. Interestingly there are only two programs out there that sanitize emails (check mime-types, headers, rename executable attachments etc.) at the gateway-level - a much better pro-active approach than simply detecting known threats. The first is Mimedefang, a sendmail plugin. The other is impsec, based on procmail. CU Boulder was using impsec to help keep student’s machines clean (there were scalability issues with the procmail solution, though).
August 5, 2008 5:08 am at 5:08 am (August 5, 2008)
As an aside: Would you be suggesting AUC for evaluating performance on ill-balanced testsets in general? How does this information differ from the common 2×2 matrix of classified vs true labels?
August 6, 2008 12:11 am at 12:11 am (August 6, 2008)
Yes, in my opinion using the AUC for ill-balanced datasets represents the true performance the best. It summarizes it all in one number - how do you decide which model is better when comparing 2×2 matrices?
A nice summary into all sorts of measures (AUC, Brier scores etc.) and how to determine how well a model actually works can be found here: http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html#Methods%20for%20dichotomous%20forecasts