Re: Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''

Mello Amish,

HAVE_HS_COMPILE_LIT is set by cmake based on ./cmake/sanity_checks.cmake:
       check_function_exists(hs_compile_lit HAVE_HS_COMPILE_LIT)
I think it is written to build/config.h  when you run configure_cmake.sh and it will be used when make is run,
You could remove the define before running make.

Adrian

From: Amish <[email protected]>
Sent: Friday, June 23, 2023 2:04 PM
To: Adrian Mamolea (admamole) <[email protected]>; [email protected]
Subject: Re: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''


Hello Adrian,

Although its not snort bug, but it still is affecting / will affect snort users, sooner or later.

That bug in hyperscan was introduced in February 2023 and reported on 1st March 2023.

Although hyperscan code contributor stated that it will be fixed but it seems that for some unknown reason it is not a high priority bug for them.

Anyway, coming back to snort, it seems that if I undef HAVE_HS_COMPILE_LIT then snort will use hs_compile() instead of hs_compile_lit().
https://github.com/snort3/snort3/blob/master/src/helpers/hyper_search.cc#L67

I use this command for building / compiling snort:

    ./configure_cmake.sh --prefix=/usr --enable-tcmalloc --with-daq-libraries=/usr/lib/daq/ --disable-static-daq

    make -C build

Full PKGBUILD is here: https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=snort-nfqueue

This may fix the bug for now for me.

But I do not know how to undef HAVE_HS_COMPILE_LIT while building snort via configure_cmake.sh?

Can you give some tips?

Thank you,

Amish.

PS: I do not want to downgrade hyperscan.
On 23/06/23 22:22, Adrian Mamolea (admamole) wrote:

Hello Amish,



Glad it is sorted out. The documentation for hs_compile_lit https://intel.github.io/hyperscan/dev-reference/api_files.html,

is indeed clear: the special terminating character \0 should be allowed to appear in expression, and not treated as a terminator for a string..



Adrian



-----Original Message-----

From: Amish <[email protected]><mailto:[email protected]>

Sent: Thursday, June 22, 2023 10:29 PM

To: Adrian Mamolea (admamole) <[email protected]><mailto:[email protected]>; [email protected]<mailto:[email protected]>

Subject: Re: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''



Ok this is indeed hyperscan bug.



There are other people who complained too.



https://github.com/intel/hyperscan/commit/9e17e8520f182ac81fb4f1092281a6969bf93083

https://github.com/intel/hyperscan/issues/386

https://github.com/intel/hyperscan/issues/388



Regards



Amish



On 23/06/23 07:48, Amish wrote:

Hello Adrian,



If that's the case then it would be Hyperscan bug, because it should

not use strcmp() but it should check if expLength is 0.



https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420





-    if (!strcmp(expression, "")) {

+    if (expLength == 0) {



Because expression may start with NUL (hex 00) byte but still may not

really be empty.



However I am neither snort expert nor hyperscan expert. I am just

guessing based on few lines of snort and hyperscan code that I read.



I have no clue how these "content" (expressions) work.



Please do let me know if this needs filing a bug report (either on

snort github or hyperscan github). I can do that.



Thanks and regards,



Amish.



On 23/06/23 04:59, Adrian Mamolea (admamole) wrote:

Hello Amish,



Good info, the rule is tripping on the check for null string that was

added to hyperscan since 5.4.1:

https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420



Perhaps on the other host you have an older version? In that case you

can try to downgrade hyperscan.

I'll check how this is supposed to work and get back to you.



Adrian



-----Original Message-----

From: Amish <[email protected]><mailto:[email protected]>

Sent: Wednesday, June 21, 2023 10:14 PM

To: Adrian Mamolea (admamole) <[email protected]><mailto:[email protected]>;

[email protected]<mailto:[email protected]>

Subject: Re: [Snort-users] Hyperscan and ERROR:

rules/snort.rules:3208 can't compile content ''



Further to my previous message, I added a code to print error message

given by Hyperscan.



So here is the error message:



ERROR: rules/snort.rules:2210 can't compile content '', error message:

'Pure literal API doesn't support empty string.'



So it seems that snort is sending empty strings to Hyperscan on certain

systems. (Or hyperscan is treating it as empty strings)



Can snort developers please look into it?



I can try to provide more information if required.



Thank you,



Regards



Amish.



On 22/06/23 06:52, Amish wrote:

Hello Adrian,



Thank you for your reply.



On 22/06/23 01:56, Adrian Mamolea (admamole) wrote:

Hello Amish,



For:

ERROR: rules/snort.rules:3208 can't compile content ''

I would expect the pattern between quotes. Did you remove it?

No, rules are fetched using pulledpork v3 and same rules work on

another machine with Intel Core 2 Duo.



And yes till now even I was surprised why the content is empty in

ERROR message.



But I just checked all the rules where snort gives error, all those

rules have content starting with "00", which gets treated as EMPTY

string as it begins with NUL byte.



For example: (I use snort community rules)



alert udp $EXTERNAL_NET any -> $HOME_NET 111 ( msg:"PROTOCOL-RPC

portmap SET attempt UDP 111"; flow:to_server; content:"|00 01 86

A0|",depth 4,offset 12; content:"|00 00 00 01|",within 4,distance 4;

content:"|00 00 00 00|",depth 4,offset 4; metadata:policy

max-detect-ips drop,ruleset community; service:sunrpc;

classtype:rpc-portmap-decode; sid:1950; rev:13; )



alert udp $EXTERNAL_NET any -> $HOME_NET 500 ( msg:"SERVER-OTHER

ISAKMP second payload initial contact notification without SPI

attempt"; flow:to_server; content:"|0B|",depth 1,offset 28;

byte_jump:2,30; content:"|00 0C 00 00 00 01 01 00|`|02|",within

10,distance -2; metadata:policy max-detect-ips drop,ruleset community;

reference:bugtraq,9416; reference:bugtraq,9417;

reference:cve,2004-0164; classtype:misc-attack; sid:2415; rev:16; )



In some rules there are multiple contents, but atleast one content

starts with "00"



In short, any rule where there is a content starting with "00", snort

fails and gives FATAL error.



So it looks like (for certain specific systems) either its snort bug

somewhere which sends "empty" content to hyperscan OR its hyperscan

bug which treats strings starting with "00" as NUL.



Otherwise, since commenting out hyperscan made it work, I would

suspect hyperscan issues on this host, and rules probably OK.

See https://github.com/snort3/snort3/issues/171 for similar issue.

Above link and other links given inside that link suggest:



1) Use physical core - but I am already using physical machine and not

virtual machine

2) CPU should have ssse3 - I am using i3 9th generation and

/proc/cpuinfo indeed lists ssse3



I am not a coder but will try to debug further on whatever I can but

just in case you can figure out the problem based on above, then

please do check.



Thank you



Adrian



-----Original Message-----

From: Snort-users <[email protected]><mailto:[email protected]> On Behalf Of

Amish via Snort-users

Sent: Tuesday, June 20, 2023 12:21 AM

To: [email protected]<mailto:[email protected]>

Subject: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208

can't compile content ''



Hello,



I am using Arch Linux with snort version as follows:



     ,,_     -*> Snort++ <*-

     o"  )~   Version 3.1.63.0

      ''''    By Martin Roesch & The Snort Team

              http://snort.org/contact#team

              Copyright (C) 2014-2023 Cisco and/or its affiliates. All

rights reserved.

              Copyright (C) 1998-2013 Sourcefire, Inc., et al.

              Using DAQ version 3.0.11

              Using LuaJIT version 2.1.0-beta3

              Using OpenSSL 3.1.1 30 May 2023

              Using libpcap version 1.10.4 (with TPACKET_V3)

              Using PCRE version 8.45 2021-06-15

              Using ZLIB version 1.2.13

              Using Hyperscan version 5.4.2 2023-04-22

              Using LZMA version 5.4.3



I am running it on Intel i3-9100T CPU (Dell Optiplex 3070)



But there are hundreds of lines as follows:



ERROR: rules/snort.rules:3208 can't compile content ''



Snort crashes with following error:

Jun 20 09:36:34 foo snort[27985]: FATAL ERROR: see prior 721 errors (0

warnings)



But this same version (with exact same package file) works fine on an

older machine with Intel Core 2 Duo. Both machines have identical

packages installed.



I have following line in local.lua file



-- Enable hyperscan for IPS, AppID, HTTP inspection, pcre/regex

matches

search_engine = { search_method = "hyperscan" }

detection = { hyperscan_literals = true, pcre_to_regex = true }



With this line it works fine on Intel Core 2 Duo machine but not

work on

Intel i3-9100T.



If I comment above two config lines, then snort starts fine.



What could be the issue? And how can I solve this?



Thank you



Amish.

_______________________________________________
Snort-users mailing list
[email protected]
Go to this URL to change user options or unsubscribe:
https://lists.snort.org/mailman/listinfo/snort-users

	To unsubscribe, send an email to:
	[email protected]

Please visit http://blog.snort.org to stay current on all the latest Snort news!

Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette