June 7, 2007
Recent commentary in Virus Bulletin magazine describes the schizophrenic relationship between antivirus vendors and those who test their products. The article forewarns of a movement to do away with The WildList, the minimum bar to which all reputable antivirus scanners are held.
Briefly, the WildList Organization manages a collection of viruses, worms, and Trojans that have been deemed by at least two researchers - independently of one another - of having been found actively infecting computers 'in the wild' (i.e. actively infecting customers). It's not a perfect solution - the WildList excludes whole classes of threats. Coupled with reporting requirements and management issues, the end result is a WildList that includes only a small fraction (less than 800) of the hundreds of thousands of malware now known to exist.
Despite this shortcoming, the WildList remains an important resource. While some claim the tests serve only as marketing fodder for the vendors, and others claim the tests are easy to pass, neither argument could be further from the truth.
No grading curve
Threats are taken off the WildList and new threats added on each month. The exact threats removed or added aren't divulged until after the tests have been completed. In order to get and maintain WildList certification means the participating vendors must maintain constant vigilance, quickly responding to and incorporating new detections in their antivirus scanners. This need for constant vigilance and continual development not only constantly improves the protection of the reputable antivirus scanners, it also prevents rogue antivirus products from gaining the certification.
Consumer confidence
To the average consumer, the WildList certification then becomes the sole legitimate measure - the bar, if you will - of what constitutes a viable scanner. Thus, while it's true that the vendor gains some marketing exposure from passing WildList certification, the real benefit is to the user who now has some evidence of standardized competence upon which to base their buying decision. This is not to say that WildList certification should be the sole measure of a scanner's ability. It is, however, an important first step in choosing which of the scanners are worth considering for purchase.
Mileage may vary
The three main certifying agencies (Virus Bulletin, Checkmark, and ICSA Labs) all use the WildList to some extent in their testing. And the well-respected independent testing agency AV-Test.org also includes WildList testing as one part of a wide range of detection tests performed. These tests in turn become the basis for many of the independent antivirus reviews, including this site.
No antivirus scanner can claim to sweep all of the certifications and tests all of the time. But the due diligance required to even be able to submit to the tests is such that disreputable products will quickly be weeded out. In other words, the next time you see a flashing banner ad claiming your system is infected and instructing you to download the latest 'spy killer', check out the certification and legitimate review sites. You shouldn't be surprised these nefarious products don't even make the D list - WildList testing weeds them out.
In this manner, the WildList not only provides the consumer with a valuable starting point for antivirus products to consider - the omissions let the consumer know which products to avoid. Since many of the nefarious, rogue scanners are themselves dropping miscreant files on consumers' systems, this places the WildList in the position of serving not only as a bar of competence, but also as an extra layer of protection.
No one will argue the WildList is perfect, or that it's not in need of some fixing. And every vendor has struggled with failing at least some of the certification tests some of the time. It's understandable that these frustrations might cause some vendors to wish the WildList would just go away. But if the WildList goes, so goes the only standard bar of measurement - paving the way for even more rogue and nefarious products making unsubtantiated claims, deceiving the consumer and leading to even more infection and eventually disenchantment with the industry as a whole.
