Re: Snort++: how to get multithreading to work?
Russ <[email protected]>
| Newsgroups | gmane.comp.security.ids.snort.devel |
|---|---|
| Message-ID | <[email protected]> |
My bad ... didn't see all the output you provided before firing off that response. Currently load balancing must be done externally which means you get one packet thread per source. If you have just one interface, one packet thread is all you get. On 6/17/15 1:41 PM, Prude, Terrell (SCC) wrote: > > Hello folks, > > This is my first post. We’ve been running “regular” Snort since the > 2.9.5.x days and thought we’d give the new Snort 3.0.0 Alpha a whirl. > For us, the major attraction to Snort++ is the multithreading for > reasons of capacity. > > Unfortunately, I’m having some trouble figuring out how to get that to > work. So far, the Snort process looks like it’s still using only one > CPU. Snort itself seems to start right up and is “snorting” > packets, and we are getting output in the Unified2 format. > > Could someone point me in the right direction as to what I’m missing? > > Platform: > > ------------------------------------ > > Processor: Intel 4GHz quad-core w/ hyperthreading > > DRAM: 32 GB > > Disk space: 2TB, with about 1.9TB free > > NIC for Snorting: Intel X520-SR2 10Gbit fiber Ethernet > > NIC for management: Realtek 8169 built-in 1Gbit copper Ethernet > > OS: CentOS 7.1 > > Snort version: 3.0.0-a1-155 > > LuaJIT version: 2.0.4 > > DAQ version: 2.0.5 > > ------------------------------------ > > All the ./configure stuff uses the default paths, i. e. the /usr/local > tree. I tried to stay as plain-vanilla as I reasonably could that way. > > The configure statement: > > ------------------------------------ > > ./configure --disable-silent-rules --enable-ppm > --enable-perf-profiling --enable-large-pcap > > ------------------------------------ > > I then ran the make statement with -j8, per the directions, followed > by “make install”. That looked good. > > Command line to invoke Snort: > > ------------------------------------ > > /usr/local/bin/snort -D -i enp1s0f0 -c /usr/local/etc/snort/snort.lua > -l /var/log/snort -z 8 > > ------------------------------------ > > The log output from when Snort starts: > > ------------------------------------ > > Jun 17 04:07:47 p-its-idssnort2 snort[2984]: > -------------------------------------------------- > > Jun 17 04:07:47 p-its-idssnort2 snort[2984]: o")~ Snort++ 3.0.0-a1-155 > > Jun 17 04:07:47 p-its-idssnort2 snort[2984]: > -------------------------------------------------- > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading > /usr/local/etc/snort/snort.lua: > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: ips > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: active > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: classifications > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rpc_decode > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_tcp > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: binder > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: unified2 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_ip > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: event_queue > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: detection > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: network > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: normalizer > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: references > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: stream_udp > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: search_engine > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished > /usr/local/etc/snort/snort.lua. > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading rules: > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Loading > /usr/local/etc/snort/rules/local.rules: > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished > /usr/local/etc/snort/rules/local.rules. > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Finished rules. > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: > -------------------------------------------------- > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule counts > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: total rules loaded: 2304 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: text rules: 2304 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: option chains: 2304 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: chain headers: 2304 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: > -------------------------------------------------- > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: rule port counts > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: tcp udp icmp ip > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: any 2304 2304 > 2304 2304 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: nc 0 0 > 0 2304 > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: > -------------------------------------------------- > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: pcap DAQ configured to > passive. > > Jun 17 04:07:48 p-its-idssnort2 snort[2984]: Initializing daemon mode > > Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Daemon initialized, > signaled parent pid: 2984 > > Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Writing PID "2993" to > file "/var/log/snort/snort.pid" > > Jun 17 04:07:48 p-its-idssnort2 snort[2993]: Commencing packet processing > > Jun 17 04:07:48 p-its-idssnort2 snort[2993]: ++ [0] enp1s0f0 > > ------------------------------------ > > The log output after I kill the Snort process: > > ------------------------------------ > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ** caught term signal > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: == stopping > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: -- [0] enp1s0f0 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Packet Statistics > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: daq > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pcaps: 1 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: received: 1120415147 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: dropped: 1096740235 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: outstanding: 1096741527 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: allow: 23673620 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: idle: 1 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: codec > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: total: > 23673624 (100.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: other: > 555 ( 0.002%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: > 2609430 ( 11.023%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: auth: 769 ( 0.003%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: esp: 211987 ( 0.895%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: eth: 23673624 (100.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gre: 8574 ( 0.036%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4: > 2671 ( 0.011%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp4_ip: > 1277 ( 0.005%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: icmp6: > 26 ( 0.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv4: 23673624 (100.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6: 58 ( 0.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ipv6_no_next: > 31 ( 0.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ppp_encap: > 8574 ( 0.036%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp: 16224198 ( 68.533%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: teredo: > 58 ( 0.000%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp: 4838675 ( 20.439%) > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Module Statistics > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: bad checksum (ip4): 10418 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: binder > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 4399149 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: inspects: 4399149 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: ip flows: 2800 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp flows: 4296173 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: tcp prunes: 4165102 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: udp flows: 100176 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_ip > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: fragments: 220 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: reassembled: 4 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers added: 216 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers freed: 216 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes inserted: 220 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: nodes deleted: 220 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_tcp > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 4296173 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: discards: 170170 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: events: 3999594 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn trackers: 247856 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: syn-ack trackers: 6903 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: data trackers: 109476 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers created: 364235 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: trackers released: 364235 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs queued: 335355 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs released: 335355 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs split: 227 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: segs used: 56291 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt packets: 22755 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: rebuilt buffers: 42889 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: overlaps: 28 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: gaps: 73264 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max segs: 15128 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: max bytes: 137382 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: client cleanups: 73351 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: server cleanups: 66854 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: stream_udp > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: sessions: 100176 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: created: 100176 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: released: 100176 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: Summary Statistics > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: detection > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: analyzed: 23673620 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: process > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: signals: 1 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: > -------------------------------------------------- > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: timing > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: runtime: 01:25:16 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: seconds: 5116.16403 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: packets: 23673620 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: pkts/sec: 4627 > > Jun 17 05:33:04 p-its-idssnort2 snort[2993]: o")~ Snort exiting > > ------------------------------------ > > The “top” output while Snort++ is running: > > ------------------------------------ > > top - 05:20:58 up 2:15, 3 users, load average: 1.00, 1.01, 0.99 > > Tasks:*201 *total,* 1 *running,*200 *sleeping,* 0 *stopped,* 0 * > zombie > > %Cpu0 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu1 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*99.7 *id,*0.3 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu2 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu3 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu4 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu5 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu6 :* 0.0 *us,* 0.0 *sy,* 0.0 *ni,*100.0 *id,*0.0 *wa,* 0.0 * > hi,* 0.0 *si,* 0.0 *st > > %Cpu7 :*97.3 *us,* 0.0 *sy,* 0.0 *ni,* 0.0 *id,*0.0 *wa,* 0.0 * > hi,* 2.7 *si,* 0.0 *st > > KiB Mem :*32703168 *total,*31865964 *free,* 659032 *used,*178172 > *buff/cache > > KiB Swap:* 4092 *total,* 4092 *free,* 0 *used.*31846588 > *avail Mem > > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND > > 2993 root 20 0 379776 327360 4040 S 100.0 1.0 73:23.35 snort > > 1 root 20 0 56652 6728 3908 S 0.0 0.0 0:00.76 > systemd > > 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > kthreadd > > 3 root 20 0 0 0 0 S 0.0 0.0 0:00.02 > ksoftirqd/0 > > 5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 > kworker/0:0H > > 7 root rt 0 0 0 0 S 0.0 0.0 0:00.00 > migration/0 > > 8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh > > 9 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/0 > > 10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/1 > > 11 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/2 > > 12 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/3 > > 13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/4 > > 14 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/5 > > 15 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/6 > > 16 root 20 0 0 0 0 S 0.0 0.0 0:00.00 > rcuob/7 > > 17 root 20 0 0 0 0 S 0.0 0.0 0:00.22 > rcu_sched > > 18 root 20 0 0 0 0 S 0.0 0.0 0:00.09 > rcuos/0 > > 19 root 20 0 0 0 0 S 0.0 0.0 0:00.08 > rcuos/1 > > 20 root 20 0 0 0 0 S 0.0 0.0 0:00.01 > rcuos/2 > > 21 root 20 0 0 0 0 S 0.0 0.0 0:00.01 > rcuos/3 > > 22 root 20 0 0 0 0 S 0.0 0.0 0:00.03 > rcuos/4 > > 23 root 20 0 0 0 0 S 0.0 0.0 0:00.01 > rcuos/5 > > 24 root 20 0 0 0 0 S 0.0 0.0 0:00.03 > rcuos/6 > > 25 root 20 0 0 0 0 S 0.0 0.0 0:00.01 > rcuos/7 > > 26 root rt 0 0 0 0 S 0.0 0.0 0:00.00 > watchdog/0 > > 27 root rt 0 0 0 0 S 0.0 0.0 0:00.00 > watchdog/1 > > 28 root rt 0 0 0 0 S 0.0 0.0 0:00.00 > migration/1 > > 29 root 20 0 0 0 0 S 0.0 0.0 0:00.01 > ksoftirqd/1 > > ------------------------------------ > > And finally, what the NIC itself is reporting for traffic that it’s > seeing. We’re seeing it come in, all right. J So far, no errors, > collisions, or any other apparent nasties. > > ------------------------------------ > > $ ip -s link show enp1s0f0 > > 3: enp1s0f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state > UP mode DEFAULT qlen 1000 > > link/ether 90:e2:ba:85:28:74 brd ff:ff:ff:ff:ff:ff > > RX: bytes packets errors dropped overrun mcast > > 1865322070123 1892842032 0 0 0 8445 > > TX: bytes packets errors dropped carrier collsns > > 0 0 0 0 0 0 > > ------------------------------------ > > > > ------------------------------------------------------------------------------ > > > _______________________________________________ > Snort-devel mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/snort-devel > Archive: > http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel > > Please visit http://blog.snort.org for the latest news about Snort! ------------------------------------------------------------------------------ _______________________________________________ Snort-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/snort-devel Archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel Please visit http://blog.snort.org for the latest news about Snort!