Security Suites Get Exploited

Traditional antivirus (AV) testing, such as that done by organizations such as AV-test.org and AV-comparatives.org, uses collections of malware to demonstrate the capabilities of security products. Secunia, on the other hand, focuses on exploits. For example, it has a scanner at its site that will search your system for unpatched vulnerabilities in products such as Microsoft Windows, Internet Explorer, or Firefox.

As Secunia's focus is on exploits, you might expect them to try to put AV solutjions to the test with exploits, rather than just already published malware, and that's what they did. They did a study (.PDF) by taking the following security suites:

• McAfee Internet Security Suite 2009
• Norton Internet Security 2009
• Windows Live OneCare
• ZoneAlarm Security Suite 8
• AVG Internet Security 8.0
• CA Internet Security Suite 2008
• F-secure Internet Security 2009
• TrendMicro Internet Security 2008
• BitDefender Internet Security Suite 2009
• Panda Internet Security 2009
• Kaspersky Internet Security 2009
• Norman Security Suite 7.10

and ran them against a library of exploit code (all developed in-house). As the report says:

The test cases are a mix of three different kinds of exploits:

• Proof of Concept (PoC) – The purpose of a PoC is to just trigger the vulnerability. It does not carry a payload. If a security product can reliably detect a PoC, then it can detect all attempts to exploit the vulnerability independent of the payload.
• GameOver PoC – The purpose of a GameOver PoC is to prove that code execution is possible by gaining control of the program flow, without actually launching any code.
• Exploit – Exploits carry a payload and will execute it if used against a vulnerable application.

In real life, an attacker would always use an exploit. However, if a security product can not detect a PoC it also can not detect an exploit reliably.

Of the 12 suites, Symantec did the best job. But it shouldn't be congratulated. It only detected 64 out of 300 exploits. Of course, that was 10x higher than second place, but still.

Of course, the study is already controversial, with some questioning its validity. However, questionable or not, it's still an interesting study, and it does make an important point: you need to keep your PC patched as well as have an up-to-date AV product on your system.

And after all, the most insecure part of your system --- is yourself. Don't click on unexpected attachments; don't visit risky sites, and things will be much better for your PC.