Re: ClamAV effectiveness

On Sat, 12 Oct 2013 12:00:02 +0200
[email protected] wrote:

> Date: Fri, 11 Oct 2013 10:41:33 -0400
> From: Nick Johnson <[email protected]>

> (1) You're measuring effectiveness against your assumption that 99% of
> .exe files in email have malware.  Although I agree with that
> assumption, it should really be validated (perhaps with another AV
> program) before we accept it as truth and declare that clamav has 80%
> false negatives.

It's validated by eye; I look at the message subjects and they are obviously
viruses.

> (2) You are confusing two different metrics.  One is the % of .exe
> files which clamav declares clean.  The other is the % of malware
> which clamav declares clean.  These are different because one malware
> could appear in several .exe files.

It's of academic interest; Clam is leaking like a sieve and our customers
are not particularly interested in the reasons.

> When a new malware appears, there is a brief window during which
> signature-based detection schemes (from ANY vendor) cannot find it.

Absolutely.

> It's entirely possible that there is ONE new malware that appears in
> 137K .exe files sampled in 'a few days'.

Possible.

> In that case, clamav would
> identify all but one malware, yet the statistics look very bad because
> that ONE undetectable malware appeared 137K times.  So, I would ask:
> of these 137K .exe files, are they all identical?  Perhaps you could
> report the number of distinct file sizes or number of distinct
> md5sums.

I will have to run that analysis next week.  I suspect they are not all
identical, but I suspect too that there's a clump of a few or a few dozen
distinct viruses.

> From: Joel Esler <[email protected]>
> It helps the ClamAV tremendously if these files are submitted to the
> ClamAV team for analysis.

Do you have an efficient mechanism for submitting hundreds or thousands
of files?  I can dedupe them and submit, but it has to be something
semi-automated; please reply off-list if you have such a mechanism.

Regards,

David.
_______________________________________________
http://lurker.clamav.net/list/clamav-devel.html
Please submit your patches to our Bugzilla: http://bugs.clamav.net