Currently, most comparative AV tests are signature-based. This type of test is analogous to what occurs when an antivirus product runs a hard drive scan—virus files with various signatures are scattered throughout the data set that's being checked and each product is rated on how many of those various files it managed to detect. Behavioral scans, on the other hand, are meant to replicate how a PC typically encounters malware, and they model a wide variety of scenarios from email virus detection to page redirects.
Companies that have signed on to work with the new group include Symantec, F-Secure, and Sunbelt Software (no McAfee yet). The AMTWG isn't just a vendors-only club; AV software evaluators are also participating, including AV-Test.org and Virus Bulletin. Although they aren't listed in the PC World article, I contacted AV-Comparatives.org. The company publishes a quarterly "report card" on AV products, and is considered to be an excellent evaluator of security products.
AV-Comparatives tells Ars that it has also been involved in some of these vendor meetings, and reports that the various companies involved seem genuinely interested in creating an effective standard for measuring product performance. Vendors also aren't putting any pressure on product testers to specifically adopt or abandon particular methods of testing, and have thus far emphasized that the goal of the task force is to create a good evaluation tool.
Antivirus vendors to agree on standard testing guidelines
Posted on Monday, Dec 10 2007 @ 03:05 CET by Thomas De Maesschalck
ARS Technica writes several AV vendors have agreed to create a set of software benchmarks that can conduct behavioral tests on multiple suits of security software. This will make it easier for consumers to compare products from antivirus makers.