Re: Hyperscan and ERROR: rules/snort.rules:3208 can't compile content ''
Amish via Snort-users <[email protected]> Mon, 26 Jun 2023 10:03:19 +0530
| Newsgroups | gmane.comp.security.ids.snort.general |
|---|---|
| Message-ID | <[email protected]> |
Thanks Adrian, I did the same and snort works fine i.e. does not crash anymore. https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=snort-nfqueue&id=fbb1f40bf54b3118b56001df6bdb3ba238debee7#n49 Regards Amish. On 26/06/23 06:49, Adrian Mamolea (admamole) wrote: > > Mello Amish, > > HAVE_HS_COMPILE_LIT is set by cmake based on ./cmake/sanity_checks.cmake: > > check_function_exists(hs_compile_lit HAVE_HS_COMPILE_LIT) > > I think it is written to build/config.h when you run > configure_cmake.sh and it will be used when make is run, > > You could remove the define before running make. > > Adrian > > *From:* Amish <[email protected]> > *Sent:* Friday, June 23, 2023 2:04 PM > *To:* Adrian Mamolea (admamole) <[email protected]>; > [email protected] > *Subject:* Re: [Snort-users] Hyperscan and ERROR: > rules/snort.rules:3208 can't compile content '' > > Hello Adrian, > > Although its not snort bug, but it still is affecting / will affect > snort users, sooner or later. > > That bug in hyperscan was introduced in February 2023 and reported on > 1st March 2023. > > Although hyperscan code contributor stated that it will be fixed but > it seems that for some unknown reason it is not a high priority bug > for them. > > Anyway, coming back to snort, it seems that if I undef > HAVE_HS_COMPILE_LIT then snort will use hs_compile() instead of > hs_compile_lit(). > https://github.com/snort3/snort3/blob/master/src/helpers/hyper_search.cc#L67 > > I use this command for building / compiling snort: > > | .|/|configure_cmake.sh |--prefix=/|usr |--enable-tcmalloc --with-daq-libraries=/|usr|/|lib|/|daq|/||--disable-static-daq > ||make||-C|build| > > Full PKGBUILD is here: > https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=snort-nfqueue > > This may fix the bug for now for me. > > But I do not know how to undef HAVE_HS_COMPILE_LIT while building > snort via configure_cmake.sh? > > Can you give some tips? > > Thank you, > > Amish. > > PS: I do not want to downgrade hyperscan. > > On 23/06/23 22:22, Adrian Mamolea (admamole) wrote: > > Hello Amish, > > Glad it is sorted out. The documentation for hs_compile_lithttps://intel.github.io/hyperscan/dev-reference/api_files.html, > > is indeed clear: the special terminating character \0 should be allowed to appear in expression, and not treated as a terminator for a string.. > > Adrian > > -----Original Message----- > > From: Amish<[email protected]> <mailto:[email protected]> > > Sent: Thursday, June 22, 2023 10:29 PM > > To: Adrian Mamolea (admamole)<[email protected]> <mailto:[email protected]>;[email protected] > > Subject: Re: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208 can't compile content '' > > Ok this is indeed hyperscan bug. > > There are other people who complained too. > > https://github.com/intel/hyperscan/commit/9e17e8520f182ac81fb4f1092281a6969bf93083 > > https://github.com/intel/hyperscan/issues/386 > > https://github.com/intel/hyperscan/issues/388 > > Regards > > Amish > > On 23/06/23 07:48, Amish wrote: > > Hello Adrian, > > If that's the case then it would be Hyperscan bug, because it should > > not use strcmp() but it should check if expLength is 0. > > https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420 > > - if (!strcmp(expression, "")) { > > + if (expLength == 0) { > > Because expression may start with NUL (hex 00) byte but still may not > > really be empty. > > However I am neither snort expert nor hyperscan expert. I am just > > guessing based on few lines of snort and hyperscan code that I read. > > I have no clue how these "content" (expressions) work. > > Please do let me know if this needs filing a bug report (either on > > snort github or hyperscan github). I can do that. > > Thanks and regards, > > Amish. > > On 23/06/23 04:59, Adrian Mamolea (admamole) wrote: > > Hello Amish, > > Good info, the rule is tripping on the check for null string that was > > added to hyperscan since 5.4.1: > > https://github.com/intel/hyperscan/blob/v5.4.1/src/compiler/compiler.cpp#L420 > > Perhaps on the other host you have an older version? In that case you > > can try to downgrade hyperscan. > > I'll check how this is supposed to work and get back to you. > > Adrian > > -----Original Message----- > > From: Amish<[email protected]> <mailto:[email protected]> > > Sent: Wednesday, June 21, 2023 10:14 PM > > To: Adrian Mamolea (admamole)<[email protected]> <mailto:[email protected]>; > > [email protected] > > Subject: Re: [Snort-users] Hyperscan and ERROR: > > rules/snort.rules:3208 can't compile content '' > > Further to my previous message, I added a code to print error message > > given by Hyperscan. > > So here is the error message: > > ERROR: rules/snort.rules:2210 can't compile content '', error message: > > 'Pure literal API doesn't support empty string.' > > So it seems that snort is sending empty strings to Hyperscan on certain > > systems. (Or hyperscan is treating it as empty strings) > > Can snort developers please look into it? > > I can try to provide more information if required. > > Thank you, > > Regards > > Amish. > > On 22/06/23 06:52, Amish wrote: > > Hello Adrian, > > Thank you for your reply. > > On 22/06/23 01:56, Adrian Mamolea (admamole) wrote: > > Hello Amish, > > For: > > ERROR: rules/snort.rules:3208 can't compile content '' > > I would expect the pattern between quotes. Did you remove it? > > No, rules are fetched using pulledpork v3 and same rules work on > > another machine with Intel Core 2 Duo. > > And yes till now even I was surprised why the content is empty in > > ERROR message. > > But I just checked all the rules where snort gives error, all those > > rules have content starting with "00", which gets treated as EMPTY > > string as it begins with NUL byte. > > For example: (I use snort community rules) > > alert udp $EXTERNAL_NET any -> $HOME_NET 111 ( msg:"PROTOCOL-RPC > > portmap SET attempt UDP 111"; flow:to_server; content:"|00 01 86 > > A0|",depth 4,offset 12; content:"|00 00 00 01|",within 4,distance 4; > > content:"|00 00 00 00|",depth 4,offset 4; metadata:policy > > max-detect-ips drop,ruleset community; service:sunrpc; > > classtype:rpc-portmap-decode; sid:1950; rev:13; ) > > alert udp $EXTERNAL_NET any -> $HOME_NET 500 ( msg:"SERVER-OTHER > > ISAKMP second payload initial contact notification without SPI > > attempt"; flow:to_server; content:"|0B|",depth 1,offset 28; > > byte_jump:2,30; content:"|00 0C 00 00 00 01 01 00|`|02|",within > > 10,distance -2; metadata:policy max-detect-ips drop,ruleset community; > > reference:bugtraq,9416; reference:bugtraq,9417; > > reference:cve,2004-0164; classtype:misc-attack; sid:2415; rev:16; ) > > In some rules there are multiple contents, but atleast one content > > starts with "00" > > In short, any rule where there is a content starting with "00", snort > > fails and gives FATAL error. > > So it looks like (for certain specific systems) either its snort bug > > somewhere which sends "empty" content to hyperscan OR its hyperscan > > bug which treats strings starting with "00" as NUL. > > Otherwise, since commenting out hyperscan made it work, I would > > suspect hyperscan issues on this host, and rules probably OK. > > Seehttps://github.com/snort3/snort3/issues/171 for similar issue. > > Above link and other links given inside that link suggest: > > 1) Use physical core - but I am already using physical machine and not > > virtual machine > > 2) CPU should have ssse3 - I am using i3 9th generation and > > /proc/cpuinfo indeed lists ssse3 > > I am not a coder but will try to debug further on whatever I can but > > just in case you can figure out the problem based on above, then > > please do check. > > Thank you > > Adrian > > -----Original Message----- > > From: Snort-users<[email protected]> <mailto:[email protected]> On Behalf Of > > Amish via Snort-users > > Sent: Tuesday, June 20, 2023 12:21 AM > > To:[email protected] > > Subject: [Snort-users] Hyperscan and ERROR: rules/snort.rules:3208 > > can't compile content '' > > Hello, > > I am using Arch Linux with snort version as follows: > > ,,_ -*> Snort++ <*- > > o" )~ Version 3.1.63.0 > > '''' By Martin Roesch & The Snort Team > > http://snort.org/contact#team > > Copyright (C) 2014-2023 Cisco and/or its affiliates. All > > rights reserved. > > Copyright (C) 1998-2013 Sourcefire, Inc., et al. > > Using DAQ version 3.0.11 > > Using LuaJIT version 2.1.0-beta3 > > Using OpenSSL 3.1.1 30 May 2023 > > Using libpcap version 1.10.4 (with TPACKET_V3) > > Using PCRE version 8.45 2021-06-15 > > Using ZLIB version 1.2.13 > > Using Hyperscan version 5.4.2 2023-04-22 > > Using LZMA version 5.4.3 > > I am running it on Intel i3-9100T CPU (Dell Optiplex 3070) > > But there are hundreds of lines as follows: > > ERROR: rules/snort.rules:3208 can't compile content '' > > Snort crashes with following error: > > Jun 20 09:36:34 foo snort[27985]: FATAL ERROR: see prior 721 errors (0 > > warnings) > > But this same version (with exact same package file) works fine on an > > older machine with Intel Core 2 Duo. Both machines have identical > > packages installed. > > I have following line in local.lua file > > -- Enable hyperscan for IPS, AppID, HTTP inspection, pcre/regex > > matches > > search_engine = { search_method = "hyperscan" } > > detection = { hyperscan_literals = true, pcre_to_regex = true } > > With this line it works fine on Intel Core 2 Duo machine but not > > work on > > Intel i3-9100T. > > If I comment above two config lines, then snort starts fine. > > What could be the issue? And how can I solve this? > > Thank you > > Amish. > _______________________________________________ Snort-users mailing list [email protected] Go to this URL to change user options or unsubscribe: https://lists.snort.org/mailman/listinfo/snort-users To unsubscribe, send an email to: [email protected] Please visit http://blog.snort.org to stay current on all the latest Snort news! Please follow these rules: https://snort.org/faq/what-is-the-mailing-list-etiquette