Re: Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''

Hello Adrian,

Although its not snort bug, but it still is affecting / will affect 
snort users, sooner or later.

That bug in hyperscan was introduced in February 2023 and reported on 
1st March 2023.

Although hyperscan code contributor stated that it will be fixed but it 
seems that for some unknown reason it is not a high priority bug for them.

Anyway, coming back to snort, it seems that if I undef 
HAVE_HS_COMPILE_LIT then snort will use hs_compile() instead of 
hs_compile_lit().
https://github.com/snort3/snort3/blob/master/src/helpers/hyper_search.cc#L67

I use this command for building / compiling snort:

|./configure_cmake.sh --prefix=/usr --enable-tcmalloc 
--with-daq-libraries=/usr/lib/daq/ --disable-static-daq ||make -C build|||

Full PKGBUILD is here: 
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=snort-nfqueue

This may fix the bug for now for me.

But I do not know how to undef HAVE_HS_COMPILE_LIT while building snort 
via configure_cmake.sh?

Can you give some tips?

Thank you,

Amish.

PS: I do not want to downgrade hyperscan.

On 23/06/23 22:22, Adrian Mamolea (admamole) wrote:
> Hello Amish,
>
> Glad it is sorted out. The documentation for hs_compile_lithttps://intel.github.io/hyperscan/dev-reference/api_files.html,
> is indeed clear: the special terminating character \0 should be allowed to appear in expression, and not treated as a terminator for a string..
>
> Adrian
>
> -----Original Message-----
> From: Amish<[email protected]>  
> Sent: Thursday, June 22, 2023 10:29 PM
> To: Adrian Mamolea (admamole)<[email protected]>;[email protected]
> Subject: Re: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''
>
> Ok this is indeed hyperscan bug.
>
> There are other people who complained too.
>
> https://github.com/intel/hyperscan/commit/9e17e8520f182ac81fb4f1092281a6969bf93083
> https://github.com/intel/hyperscan/issues/386
> https://github.com/intel/hyperscan/issues/388
>
> Regards
>
> Amish
>
> On 23/06/23 07:48, Amish wrote:
>> Hello Adrian,
>>
>> If that's the case then it would be Hyperscan bug, because it should
>> not use strcmp() but it should check if expLength is 0.
>>
>> https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420  
>>
>>
>> -    if (!strcmp(expression, "")) {
>> +    if (expLength == 0) {
>>
>> Because expression may start with NUL (hex 00) byte but still may not
>> really be empty.
>>
>> However I am neither snort expert nor hyperscan expert. I am just
>> guessing based on few lines of snort and hyperscan code that I read.
>>
>> I have no clue how these "content" (expressions) work.
>>
>> Please do let me know if this needs filing a bug report (either on
>> snort github or hyperscan github). I can do that.
>>
>> Thanks and regards,
>>
>> Amish.
>>
>> On 23/06/23 04:59, Adrian Mamolea (admamole) wrote:
>>> Hello Amish,
>>>
>>> Good info, the rule is tripping on the check for null string that was
>>> added to hyperscan since 5.4.1:
>>> https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420  
>>>
>>> Perhaps on the other host you have an older version? In that case you
>>> can try to downgrade hyperscan.
>>> I'll check how this is supposed to work and get back to you.
>>>
>>> Adrian
>>>
>>> -----Original Message-----
>>> From: Amish<[email protected]>
>>> Sent: Wednesday, June 21, 2023 10:14 PM
>>> To: Adrian Mamolea (admamole)<[email protected]>;
>>> [email protected]
>>> Subject: Re: [Snort-users] Hyperscan and ERROR:
>>> rules/snort.rules:3208 can't compile content ''
>>>
>>> Further to my previous message, I added a code to print error message
>>> given by Hyperscan.
>>>
>>> So here is the error message:
>>>
>>> ERROR: rules/snort.rules:2210 can't compile content '', error message:
>>> 'Pure literal API doesn't support empty string.'
>>>
>>> So it seems that snort is sending empty strings to Hyperscan on certain
>>> systems. (Or hyperscan is treating it as empty strings)
>>>
>>> Can snort developers please look into it?
>>>
>>> I can try to provide more information if required.
>>>
>>> Thank you,
>>>
>>> Regards
>>>
>>> Amish.
>>>
>>> On 22/06/23 06:52, Amish wrote:
>>>> Hello Adrian,
>>>>
>>>> Thank you for your reply.
>>>>
>>>> On 22/06/23 01:56, Adrian Mamolea (admamole) wrote:
>>>>> Hello Amish,
>>>>>
>>>>> For:
>>>>> ERROR: rules/snort.rules:3208 can't compile content ''
>>>>> I would expect the pattern between quotes. Did you remove it?
>>>> No, rules are fetched using pulledpork v3 and same rules work on
>>>> another machine with Intel Core 2 Duo.
>>>>
>>>> And yes till now even I was surprised why the content is empty in
>>>> ERROR message.
>>>>
>>>> But I just checked all the rules where snort gives error, all those
>>>> rules have content starting with "00", which gets treated as EMPTY
>>>> string as it begins with NUL byte.
>>>>
>>>> For example: (I use snort community rules)
>>>>
>>>> alert udp $EXTERNAL_NET any -> $HOME_NET 111 ( msg:"PROTOCOL-RPC
>>>> portmap SET attempt UDP 111"; flow:to_server; content:"|00 01 86
>>>> A0|",depth 4,offset 12; content:"|00 00 00 01|",within 4,distance 4;
>>>> content:"|00 00 00 00|",depth 4,offset 4; metadata:policy
>>>> max-detect-ips drop,ruleset community; service:sunrpc;
>>>> classtype:rpc-portmap-decode; sid:1950; rev:13; )
>>>>
>>>> alert udp $EXTERNAL_NET any -> $HOME_NET 500 ( msg:"SERVER-OTHER
>>>> ISAKMP second payload initial contact notification without SPI
>>>> attempt"; flow:to_server; content:"|0B|",depth 1,offset 28;
>>>> byte_jump:2,30; content:"|00 0C 00 00 00 01 01 00|`|02|",within
>>>> 10,distance -2; metadata:policy max-detect-ips drop,ruleset community;
>>>> reference:bugtraq,9416; reference:bugtraq,9417;
>>>> reference:cve,2004-0164; classtype:misc-attack; sid:2415; rev:16; )
>>>>
>>>> In some rules there are multiple contents, but atleast one content
>>>> starts with "00"
>>>>
>>>> In short, any rule where there is a content starting with "00", snort
>>>> fails and gives FATAL error.
>>>>
>>>> So it looks like (for certain specific systems) either its snort bug
>>>> somewhere which sends "empty" content to hyperscan OR its hyperscan
>>>> bug which treats strings starting with "00" as NUL.
>>>>
>>>>> Otherwise, since commenting out hyperscan made it work, I would
>>>>> suspect hyperscan issues on this host, and rules probably OK.
>>>>> Seehttps://github.com/snort3/snort3/issues/171  for similar issue.
>>>> Above link and other links given inside that link suggest:
>>>>
>>>> 1) Use physical core - but I am already using physical machine and not
>>>> virtual machine
>>>> 2) CPU should have ssse3 - I am using i3 9th generation and
>>>> /proc/cpuinfo indeed lists ssse3
>>>>
>>>> I am not a coder but will try to debug further on whatever I can but
>>>> just in case you can figure out the problem based on above, then
>>>> please do check.
>>>>
>>>> Thank you
>>>>
>>>>> Adrian
>>>>>
>>>>> -----Original Message-----
>>>>> From: Snort-users<[email protected]>  On Behalf Of
>>>>> Amish via Snort-users
>>>>> Sent: Tuesday, June 20, 2023 12:21 AM
>>>>> To:[email protected]
>>>>> Subject: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208
>>>>> can't compile content ''
>>>>>
>>>>> Hello,
>>>>>
>>>>> I am using Arch Linux with snort version as follows:
>>>>>
>>>>>       ,,_     -*> Snort++ <*-
>>>>>       o"  )~   Version 3.1.63.0
>>>>>        ''''    By Martin Roesch & The Snort Team
>>>>>                http://snort.org/contact#team
>>>>>                Copyright (C) 2014-2023 Cisco and/or its affiliates. All
>>>>> rights reserved.
>>>>>                Copyright (C) 1998-2013 Sourcefire, Inc., et al.
>>>>>                Using DAQ version 3.0.11
>>>>>                Using LuaJIT version 2.1.0-beta3
>>>>>                Using OpenSSL 3.1.1 30 May 2023
>>>>>                Using libpcap version 1.10.4 (with TPACKET_V3)
>>>>>                Using PCRE version 8.45 2021-06-15
>>>>>                Using ZLIB version 1.2.13
>>>>>                Using Hyperscan version 5.4.2 2023-04-22
>>>>>                Using LZMA version 5.4.3
>>>>>
>>>>> I am running it on Intel i3-9100T CPU (Dell Optiplex 3070)
>>>>>
>>>>> But there are hundreds of lines as follows:
>>>>>
>>>>> ERROR: rules/snort.rules:3208 can't compile content ''
>>>>>
>>>>> Snort crashes with following error:
>>>>> Jun 20 09:36:34 foo snort[27985]: FATAL ERROR: see prior 721 errors (0
>>>>> warnings)
>>>>>
>>>>> But this same version (with exact same package file) works fine on an
>>>>> older machine with Intel Core 2 Duo. Both machines have identical
>>>>> packages installed.
>>>>>
>>>>> I have following line in local.lua file
>>>>>
>>>>> -- Enable hyperscan for IPS, AppID, HTTP inspection, pcre/regex
>>>>> matches
>>>>> search_engine = { search_method = "hyperscan" }
>>>>> detection = { hyperscan_literals = true, pcre_to_regex = true }
>>>>>
>>>>> With this line it works fine on Intel Core 2 Duo machine but not
>>>>> work on
>>>>> Intel i3-9100T.
>>>>>
>>>>> If I comment above two config lines, then snort starts fine.
>>>>>
>>>>> What could be the issue? And how can I solve this?
>>>>>
>>>>> Thank you
>>>>>
>>>>> Amish.

_______________________________________________
Snort-users mailing list
[email protected]
Go to this URL to change user options or unsubscribe:
https://lists.snort.org/mailman/listinfo/snort-users

	To unsubscribe, send an email to:
	[email protected]

Please visit http://blog.snort.org to stay current on all the latest Snort news!

Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette