Re: Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''

Hello Amish,

Glad it is sorted out. The documentation for hs_compile_lit https://intel.github.io/hyperscan/dev-reference/api_files.html,
is indeed clear: the special terminating character \0 should be allowed to appear in expression, and not treated as a terminator for a string..

Adrian

-----Original Message-----
From: Amish <[email protected]> 
Sent: Thursday, June 22, 2023 10:29 PM
To: Adrian Mamolea (admamole) <[email protected]>; [email protected]
Subject: Re: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''

Ok this is indeed hyperscan bug.

There are other people who complained too.

https://github.com/intel/hyperscan/commit/9e17e8520f182ac81fb4f1092281a6969bf93083
https://github.com/intel/hyperscan/issues/386
https://github.com/intel/hyperscan/issues/388

Regards

Amish

On 23/06/23 07:48, Amish wrote:
> Hello Adrian,
>
> If that's the case then it would be Hyperscan bug, because it should 
> not use strcmp() but it should check if expLength is 0.
>
> https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420 
>
>
> -    if (!strcmp(expression, "")) {
> +    if (expLength == 0) {
>
> Because expression may start with NUL (hex 00) byte but still may not 
> really be empty.
>
> However I am neither snort expert nor hyperscan expert. I am just 
> guessing based on few lines of snort and hyperscan code that I read.
>
> I have no clue how these "content" (expressions) work.
>
> Please do let me know if this needs filing a bug report (either on 
> snort github or hyperscan github). I can do that.
>
> Thanks and regards,
>
> Amish.
>
> On 23/06/23 04:59, Adrian Mamolea (admamole) wrote:
>> Hello Amish,
>>
>> Good info, the rule is tripping on the check for null string that was 
>> added to hyperscan since 5.4.1:
>> https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420 
>>
>> Perhaps on the other host you have an older version? In that case you 
>> can try to downgrade hyperscan.
>> I'll check how this is supposed to work and get back to you.
>>
>> Adrian
>>
>> -----Original Message-----
>> From: Amish <[email protected]>
>> Sent: Wednesday, June 21, 2023 10:14 PM
>> To: Adrian Mamolea (admamole) <[email protected]>; 
>> [email protected]
>> Subject: Re: [Snort-users] Hyperscan and ERROR: 
>> rules/snort.rules:3208 can't compile content ''
>>
>> Further to my previous message, I added a code to print error message
>> given by Hyperscan.
>>
>> So here is the error message:
>>
>> ERROR: rules/snort.rules:2210 can't compile content '', error message:
>> 'Pure literal API doesn't support empty string.'
>>
>> So it seems that snort is sending empty strings to Hyperscan on certain
>> systems. (Or hyperscan is treating it as empty strings)
>>
>> Can snort developers please look into it?
>>
>> I can try to provide more information if required.
>>
>> Thank you,
>>
>> Regards
>>
>> Amish.
>>
>> On 22/06/23 06:52, Amish wrote:
>>> Hello Adrian,
>>>
>>> Thank you for your reply.
>>>
>>> On 22/06/23 01:56, Adrian Mamolea (admamole) wrote:
>>>> Hello Amish,
>>>>
>>>> For:
>>>> ERROR: rules/snort.rules:3208 can't compile content ''
>>>> I would expect the pattern between quotes. Did you remove it?
>>> No, rules are fetched using pulledpork v3 and same rules work on
>>> another machine with Intel Core 2 Duo.
>>>
>>> And yes till now even I was surprised why the content is empty in
>>> ERROR message.
>>>
>>> But I just checked all the rules where snort gives error, all those
>>> rules have content starting with "00", which gets treated as EMPTY
>>> string as it begins with NUL byte.
>>>
>>> For example: (I use snort community rules)
>>>
>>> alert udp $EXTERNAL_NET any -> $HOME_NET 111 ( msg:"PROTOCOL-RPC
>>> portmap SET attempt UDP 111"; flow:to_server; content:"|00 01 86
>>> A0|",depth 4,offset 12; content:"|00 00 00 01|",within 4,distance 4;
>>> content:"|00 00 00 00|",depth 4,offset 4; metadata:policy
>>> max-detect-ips drop,ruleset community; service:sunrpc;
>>> classtype:rpc-portmap-decode; sid:1950; rev:13; )
>>>
>>> alert udp $EXTERNAL_NET any -> $HOME_NET 500 ( msg:"SERVER-OTHER
>>> ISAKMP second payload initial contact notification without SPI
>>> attempt"; flow:to_server; content:"|0B|",depth 1,offset 28;
>>> byte_jump:2,30; content:"|00 0C 00 00 00 01 01 00|`|02|",within
>>> 10,distance -2; metadata:policy max-detect-ips drop,ruleset community;
>>> reference:bugtraq,9416; reference:bugtraq,9417;
>>> reference:cve,2004-0164; classtype:misc-attack; sid:2415; rev:16; )
>>>
>>> In some rules there are multiple contents, but atleast one content
>>> starts with "00"
>>>
>>> In short, any rule where there is a content starting with "00", snort
>>> fails and gives FATAL error.
>>>
>>> So it looks like (for certain specific systems) either its snort bug
>>> somewhere which sends "empty" content to hyperscan OR its hyperscan
>>> bug which treats strings starting with "00" as NUL.
>>>
>>>> Otherwise, since commenting out hyperscan made it work, I would
>>>> suspect hyperscan issues on this host, and rules probably OK.
>>>> See https://github.com/snort3/snort3/issues/171 for similar issue.
>>> Above link and other links given inside that link suggest:
>>>
>>> 1) Use physical core - but I am already using physical machine and not
>>> virtual machine
>>> 2) CPU should have ssse3 - I am using i3 9th generation and
>>> /proc/cpuinfo indeed lists ssse3
>>>
>>> I am not a coder but will try to debug further on whatever I can but
>>> just in case you can figure out the problem based on above, then
>>> please do check.
>>>
>>> Thank you
>>>
>>>> Adrian
>>>>
>>>> -----Original Message-----
>>>> From: Snort-users <[email protected]> On Behalf Of
>>>> Amish via Snort-users
>>>> Sent: Tuesday, June 20, 2023 12:21 AM
>>>> To: [email protected]
>>>> Subject: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208
>>>> can't compile content ''
>>>>
>>>> Hello,
>>>>
>>>> I am using Arch Linux with snort version as follows:
>>>>
>>>>      ,,_     -*> Snort++ <*-
>>>>      o"  )~   Version 3.1.63.0
>>>>       ''''    By Martin Roesch & The Snort Team
>>>>               http://snort.org/contact#team
>>>>               Copyright (C) 2014-2023 Cisco and/or its affiliates. All
>>>> rights reserved.
>>>>               Copyright (C) 1998-2013 Sourcefire, Inc., et al.
>>>>               Using DAQ version 3.0.11
>>>>               Using LuaJIT version 2.1.0-beta3
>>>>               Using OpenSSL 3.1.1 30 May 2023
>>>>               Using libpcap version 1.10.4 (with TPACKET_V3)
>>>>               Using PCRE version 8.45 2021-06-15
>>>>               Using ZLIB version 1.2.13
>>>>               Using Hyperscan version 5.4.2 2023-04-22
>>>>               Using LZMA version 5.4.3
>>>>
>>>> I am running it on Intel i3-9100T CPU (Dell Optiplex 3070)
>>>>
>>>> But there are hundreds of lines as follows:
>>>>
>>>> ERROR: rules/snort.rules:3208 can't compile content ''
>>>>
>>>> Snort crashes with following error:
>>>> Jun 20 09:36:34 foo snort[27985]: FATAL ERROR: see prior 721 errors (0
>>>> warnings)
>>>>
>>>> But this same version (with exact same package file) works fine on an
>>>> older machine with Intel Core 2 Duo. Both machines have identical
>>>> packages installed.
>>>>
>>>> I have following line in local.lua file
>>>>
>>>> -- Enable hyperscan for IPS, AppID, HTTP inspection, pcre/regex 
>>>> matches
>>>> search_engine = { search_method = "hyperscan" }
>>>> detection = { hyperscan_literals = true, pcre_to_regex = true }
>>>>
>>>> With this line it works fine on Intel Core 2 Duo machine but not 
>>>> work on
>>>> Intel i3-9100T.
>>>>
>>>> If I comment above two config lines, then snort starts fine.
>>>>
>>>> What could be the issue? And how can I solve this?
>>>>
>>>> Thank you
>>>>
>>>> Amish.
_______________________________________________
Snort-users mailing list
[email protected]
Go to this URL to change user options or unsubscribe:
https://lists.snort.org/mailman/listinfo/snort-users

	To unsubscribe, send an email to:
	[email protected]

Please visit http://blog.snort.org to stay current on all the latest Snort news!

Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette