Re: Bug with .fp file being ignored

Hi Micah,

Yes I did, and I submitted a patch back in July but there was another related issue which I wasn't able to fix.  I've copied my email below with the patches.

Best regards
Mark
---

The issue seems to have crept in with commits 3e42216cc and 28afc94c3 back in April/May 2017.

Attached are patches for devel/HEAD as well as the stable 0.101.2 

Tests show that the issue is fixed and doesn't appear to introduce any false negatives.....however, it does produce a duplicate output line - one listed the infection found, and the second line (honouring the FP file) saying "OK".  The "infected files" count is correct - see output below.

Does anyone know how to fix that duplicate output?

Cheers
Mark

virus-2009-04-13-id0007662101.zip: Osx.Worm.Leap-2 FOUND
virus-2009-04-13-id0007662101.zip: OK

----------- SCAN SUMMARY -----------
Known viruses: 6168730
Engine version: 0.101.2
Scanned directories: 0
Scanned files: 1
Infected files: 0
Data scanned: 0.02 MB
Data read: 0.00 MB (ratio 0.00:1)
Time: 33.865 sec (0 m 33 s)



> On 20 Sep 2019, at 10:29 am, Micah Snyder (micasnyd) <[email protected]> wrote:
> 
> Hi Mark,
> 
> Did you have any luck identifying the source of the bug?  I admit I bookmarked your email and failed to find time to look into it myself after that.  
> 
> -Micah
> 
> On 7/12/19, 6:09 PM, "clamav-devel on behalf of Mark Allan" <[email protected] on behalf of [email protected]> wrote:
> 
>    Hi,
> 
>    I think there's a bug with ClamAV not honouring the contents of a .fp file
>    within the database directory.
> 
>    I've tested 0.101.2 as well as previous versions of ClamAV going back to
>    0.99.4 and the issue seems to have appeared as of 0.100.0 onwards.
> 
>    To re-create the issue:
> 
>    Find a zip file which you know reports an infection when scanned.
>    Use sigtool --md5 to generate an FP sig of the zip file and save it in a
>    <filename>.fp file in the databse directory.
>    Use clamscan to scan the file and see that it still reports the file as
>    being infected.
> 
> 
>    The output from clamscan --debug shows the .fp file is being loaded, but it
>    just doesn't seem to be being honoured for some reason.
> 
>    I see the same thing when I build ClamAV on macOS as well as when using the
>    apt-get distribution on Ubuntu 18.04
> 
>    Lastly, it only appears to be an issue with archive filetypes eg .zip, .dmg
>    etc. Simple files are excluded as expected - similarly, if you generate an
>    FP sig of a simple file and put that file within an archive, it correctly
>    gets excluded.
> 
>    I'll clone the source from Git on Monday and have a dig through it myself
>    to see if I can fix the bug, but thought I'd mention it here in case
>    someone's already on it, or at least knows where I can start looking!
> 
>    Cheers
>    Mark
>    _______________________________________________
> 
>    clamav-devel mailing list
>    [email protected]
>    https://lists.clamav.net/mailman/listinfo/clamav-devel
> 
>    Please submit your patches to our Bugzilla: http://bugzilla.clamav.net
> 
>    Help us build a comprehensive ClamAV guide:
>    https://github.com/vrtadmin/clamav-faq
> 
>    http://www.clamav.net/contact.html#ml
> 
> 
> _______________________________________________
> 
> clamav-devel mailing list
> [email protected]
> https://lists.clamav.net/mailman/listinfo/clamav-devel
> 
> Please submit your patches to our Bugzilla: http://bugzilla.clamav.net
> 
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
> 
> http://www.clamav.net/contact.html#ml

_______________________________________________

clamav-devel mailing list
[email protected]
https://lists.clamav.net/mailman/listinfo/clamav-devel

Please submit your patches to our Bugzilla: http://bugzilla.clamav.net

Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

--- clamav-devel_clean/libclamav/scanners.c	2019-07-17 15:49:32.000000000 +0100
+++ clamav-devel/libclamav/scanners.c	2019-07-17 16:32:00.000000000 +0100
@@ -3399,6 +3399,7 @@
     ctx->recursion--;
 
     if (ret == CL_VIRUS && !SCAN_ALLMATCHES) {
+    	ret = cli_checkfp(hash, hashed_size, ctx);
         cli_bitset_free(ctx->hook_lsig_matches);
         ctx->hook_lsig_matches = old_hook_lsig_matches;
         return magic_scandesc_cleanup(ctx, type, hash, hashed_size, cache_clean, ret, parent_property);
@@ -3442,7 +3443,7 @@
                     res = CL_CLEAN;
                     // Fall through
                 case CL_VIRUS:
-                    ret = res;
+                    ret = cli_checkfp(hash, hashed_size, ctx);
                     if (SCAN_ALLMATCHES)
                         break;
                     cli_bitset_free(ctx->hook_lsig_matches);
@@ -3514,6 +3515,8 @@
             break;
     }
 
+	if(ret == CL_VIRUS)
+        ret = cli_checkfp(hash, hashed_size, ctx);
     ctx->recursion--;
     cli_bitset_free(ctx->hook_lsig_matches);
     ctx->hook_lsig_matches = old_hook_lsig_matches;

--- clamav-0.101.2_clean/libclamav/scanners.c	2019-03-13 21:13:01.000000000 +0000
+++ clamav-0.101.2/libclamav/scanners.c	2019-07-17 16:23:37.000000000 +0100
@@ -3710,6 +3710,7 @@
 
     if (ret == CL_VIRUS && !SCAN_ALLMATCHES)
     {
+    	ret = cli_checkfp(hash, hashed_size, ctx);
         cli_bitset_free(ctx->hook_lsig_matches);
         ctx->hook_lsig_matches = old_hook_lsig_matches;
         return magic_scandesc_cleanup(ctx, type, hash, hashed_size, cache_clean, ret, parent_property);
@@ -3750,7 +3751,7 @@
                 return magic_scandesc_cleanup(ctx, type, hash, hashed_size, cache_clean, res, parent_property);
             /* CL_VIRUS = malware found, check FP and report */
             case CL_VIRUS:
-                ret = res;
+                ret = cli_checkfp(hash, hashed_size, ctx);
                 if (SCAN_ALLMATCHES)
                     break;
                 cli_bitset_free(ctx->hook_lsig_matches);
@@ -3814,6 +3815,9 @@
         break;
     }
 
+    if(ret == CL_VIRUS)
+		ret = cli_checkfp(hash, hashed_size, ctx);
+
     ctx->recursion--;
     cli_bitset_free(ctx->hook_lsig_matches);
     ctx->hook_lsig_matches = old_hook_lsig_matches;