Re: Bus errors and segmentation faults after upgrade to 2.9.7.3 and daq 2.0.5
Hui cao <[email protected]>
| Newsgroups | gmane.comp.security.ids.snort.devel |
|---|---|
| Message-ID | <[email protected]> |
Can you disable dce/rpc preprocessor in your configruation and restart snort? Best, Hui. On 06/04/2015 06:00 PM, [email protected] wrote: > > So, my other sensor, on which I disabled chroot and uid/gid change in > snort.conf to keep snort running as root, I now got a signal 10 and a > core > dumped. > > pid 3744 (snort), uid 0: exited on signal 10 (core dumped) > > > # gdb /usr/local/bin/snort snort.core > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and > you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for > details. > This GDB was configured as "amd64-marcel-freebsd"... > Core was generated by `snort'. > Program terminated with signal 10, Bus error. > Reading symbols from /usr/local/lib/libdnet.so.1...done. > Loaded symbols for /usr/local/lib/libdnet.so.1 > Reading symbols from /usr/local/lib/libpcre.so.1...done. > Loaded symbols for /usr/local/lib/libpcre.so.1 > Reading symbols from /lib/libm.so.5...done. > Loaded symbols for /lib/libm.so.5 > Reading symbols from /lib/libcrypto.so.7...done. > Loaded symbols for /lib/libcrypto.so.7 > Reading symbols from /lib/libpcap.so.8...done. > Loaded symbols for /lib/libpcap.so.8 > Reading symbols from /usr/local/lib/libsfbpf.so.0...done. > Loaded symbols for /usr/local/lib/libsfbpf.so.0 > Reading symbols from /lib/libz.so.6...done. > Loaded symbols for /lib/libz.so.6 > Reading symbols from /usr/lib/liblzma.so.5...done. > Loaded symbols for /usr/lib/liblzma.so.5 > Reading symbols from /lib/libthr.so.3...done. > Loaded symbols for /lib/libthr.so.3 > Reading symbols from /lib/libc.so.7...done. > Loaded symbols for /lib/libc.so.7 > Reading symbols from > /usr/local/lib/snort_dynamicengine/libsf_engine.so...done. > Loaded symbols for /usr/local/lib/snort_dynamicengine/libsf_engine.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so > Reading symbols from > /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so...done. > Loaded symbols for > /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so > Reading symbols from /libexec/ld-elf.so.1...done. > Loaded symbols for /libexec/ld-elf.so.1 > #0 0x0000000804c48193 in ?? () from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > [New Thread 815b9f800 (LWP 100714/snort)] > [New Thread 802806400 (LWP 100635/snort)] > (gdb) > (gdb) backtrace full > #0 0x0000000804c48193 in ?? () from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > No symbol table info available. > #1 0x0000000804c47e8f in ?? () from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > No symbol table info available. > #2 0x000000000052c967 in s5_paf_callback (ps=0x80ffbd350, > ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", len=16, > flags=128) at stream_paf.c:185 > bit = 128 > paf = PAF_ABORT > mask = 128 > update = false > i = 7 > #3 0x000000000052bf7f in s5_paf_eval (pc=0x80374d000, ps=0x80ffbd350, > ssn=0x8c2942bf0, port=5600, flags=128, fuzz=150, data=0x8c7c36f48 > "32142740|PONG\r\n", len=16, ft=0x7fffffffe050) at stream_paf.c:243 > No locals. > #4 0x000000000052bbdd in s5_paf_check (pv=0x80374d000, > ps=0x80ffbd350, ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", > len=16, total=32, seq=3414157013, port=5600, flags=0x7fffffffe2b0, > fuzz=150) > at stream_paf.c:437 > ft = FT_NOP > idx = 16 > shift = 28893 > cont = false > pc = (PAF_Config *) 0x80374d000 > #5 0x0000000000520766 in flush_pdu_ackd (config=0x80372c000, > ssn=0x80ffbd1e0, trk=0x80ffbd338, pkt=0xf18ad0, flags=0x7fffffffe2b0) > at snort_stream_tcp.c:9571 > flush_pt = 8 > size = 16 > end = 3414157029 > pos = 3414157013 > to_srv = true > srv_port = 5600 > total = 32 > seg = (StreamSegment *) 0x8c7c36ec0 > snort_ticks_start = 34512853664 > snort_ticks_end = 4237555046520209951 > #6 0x000000000051ff15 in CheckFlushPolicyOnAck (config=0x80372c000, > tcpssn=0x80ffbd1e0, talker=0x80ffbd338, listener=0x80ffbd1e0, > tdb=0x7fffffffe710, p=0xf18ad0) at snort_stream_tcp.c:9729 > flags = 128 > flush_amt = 8 > flushed = 0 > #7 0x0000000000518e0f in ProcessTcp (scb=0x8c2942bf0, p=0xf18ad0, > tdb=0x7fffffffe710, s5TcpPolicy=0x812c06000) at snort_stream_tcp.c:9260 > retcode = 0 > eventcode = 0 > ignore = 0 '\0' > got_ts = 0 > new_ssn = 0 > ts_action = 0 > tcpssn = (TcpSession *) 0x80ffbd1e0 > talker = (StreamTracker *) 0x80ffbd338 > listener = (StreamTracker *) 0x80ffbd1e0 > require3Way = 0 > snort_ticks_start = 12884901889 > snort_ticks_end = 0 > #8 0x0000000000514fd4 in StreamProcessTcp (p=0xf18ad0, > scb=0x8c2942bf0, s5TcpPolicy=0x812c06000, skey=0x7fffffffe7c0) at > snort_stream_tcp.c:5655 > tdb = {seq = 3659739341, ack = 3414157029, win = 5159, end_seq > = 3659739341, ts = 0} > rc = 0 > status = 15829712 > snort_ticks_start = 140737488348992 > snort_ticks_end = 5129361 > #9 0x00000000004dc96b in StreamProcess (p=0xf18ad0, context=0x0) at > spp_stream6.c:751 > key = {ip_l = {4294961120, 32767, 5559925, 0}, ip_h = > {1045822549, 0, 364268896, 8}, port_l = 59392, port_h = 65535, > vlan_tag = 32767, protocol = 0 '\0', pad = 0 '\0', mplsLabel = 4526162, > addressSpaceId = 0, addressSpaceIdPad1 = 256} > scb = (SessionControlBlock *) 0x8c2942bf0 > snort_ticks_start = 0 > snort_ticks_end = 18446744065119617024 > #10 0x000000000044f542 in DispatchPreprocessors (p=0xf18ad0, > policy_id=0, policy=0x802fb2000) at detect.c:136 > scb = (SessionControlBlock *) 0x8c2942bf0 > ppn = (PreprocEvalFuncNode *) 0x815b7aee0 > pps_enabled_foo = 3219496 > alerts_processed = true > #11 0x000000000044ef88 in Preprocess (p=0xf18ad0) at detect.c:234 > retval = 0 > policy_id = 0 > policy = (SnortPolicy *) 0x802fb2000 > pktcnt = 0 > snort_ticks_start = 34413820928 > snort_ticks_end = 2683929608 > #12 0x000000000043e9e8 in ProcessPacket (p=0xf18ad0, > pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "", ft=0x0) at snort.c:1873 > verdict = DAQ_VERDICT_PASS > #13 0x0000000000445608 in PacketCallback (user=0x0, > pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "") at snort.c:1718 > inject = 0 > verdict = DAQ_VERDICT_PASS > snort_ticks_start = 34896609280 > snort_ticks_end = 34896609306 > #14 0x000000000056dc6a in pcap_process_loop () > No symbol table info available. > #15 0x00000008014d0554 in pcap_platform_finddevs () from > /lib/libpcap.so.8 > No symbol table info available. > #16 0x000000000056d7d8 in pcap_daq_acquire () > No symbol table info available. > #17 0x000000000046b66b in DAQ_Acquire (max=0, callback=0x445420 > <PacketCallback>, user=0x0) at sfdaq.c:541 > err = 32767 > #18 0x000000000043e47c in PacketLoop () at snort.c:3268 > error = 0 > pkts_to_read = 0 > #19 0x000000000043d3d9 in SnortMain (argc=6, argv=0x7fffffffec90) at > snort.c:921 > tmp_ptr = 0x0 > intf = 0x8028527c8 "mon0" > daqInit = 1 > #20 0x000000000043d1f8 in main (argc=6, argv=0x7fffffffec90) at > snort.c:817 > No locals. > (gdb) > (gdb) info registers > rax 0x7669643c00000000 8532461177890930688 > rbx 0x15e0 5600 > rcx 0x1 1 > rdx 0x804c47e80 34439724672 > rsi 0x8a46f3b20 37118491424 > rdi 0x33 51 > rbp 0x7fffffffded0 0x7fffffffded0 > rsp 0x7fffffffdea0 0x7fffffffdea0 > r8 0x80 128 > r9 0x80ffbd360 34627900256 > r10 0x7fffffffe050 140737488347216 > r11 0x8c7c36f48 37711212360 > r12 0x821a3f77a 34924132218 > r13 0x821a3f760 34924132192 > r14 0x96 150 > r15 0x3c 60 > rip 0x804c48193 0x804c48193 <strchr@plt+40023> > eflags 0x10206 66054 > cs 0x43 67 > ss 0x3b 59 > ds 0x0 0 > es 0x0 0 > fs 0x0 0 > gs 0x0 0 > > (gdb) x/16i $pc > 0x804c48193 <strchr@plt+40023>: mov (%rax),%cl > 0x804c48195 <strchr@plt+40025>: mov %cl,-0x11(%rbp) > 0x804c48198 <strchr@plt+40028>: movsbl -0x11(%rbp),%edx > 0x804c4819c <strchr@plt+40032>: cmp $0x0,%edx > 0x804c481a2 <strchr@plt+40038>: jne 0x804c48209 <strchr@plt+40141> > 0x804c481a8 <strchr@plt+40044>: movzbl -0x1(%rbp),%eax > 0x804c481ac <strchr@plt+40048>: cmp $0x3a,%eax > 0x804c481b1 <strchr@plt+40053>: jne 0x804c481c6 <strchr@plt+40074> > 0x804c481b7 <strchr@plt+40059>: mov -0x10(%rbp),%rax > 0x804c481bb <strchr@plt+40063>: movl $0x2,(%rax) > 0x804c481c1 <strchr@plt+40069>: jmpq 0x804c48204 <strchr@plt+40136> > 0x804c481c6 <strchr@plt+40074>: mov $0x20000,%rsi > 0x804c481d0 <strchr@plt+40084>: movzbl -0x1(%rbp),%edi > 0x804c481d4 <strchr@plt+40088>: callq 0x804c485c0 <strchr@plt+41092> > 0x804c481d9 <strchr@plt+40093>: cmp $0x0,%eax > 0x804c481de <strchr@plt+40098>: jne 0x804c481ff <strchr@plt+40131> > (gdb) > (gdb) thread apply all backtrace > > Thread 2 (Thread 802806400 (LWP 100635/snort)): > #0 0x0000000804c48193 in ?? () from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > #1 0x0000000804c47e8f in ?? () from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > #2 0x000000000052c967 in s5_paf_callback (ps=0x80ffbd350, > ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", len=16, > flags=128) at stream_paf.c:185 > #3 0x000000000052bf7f in s5_paf_eval (pc=0x80374d000, ps=0x80ffbd350, > ssn=0x8c2942bf0, port=5600, flags=128, fuzz=150, data=0x8c7c36f48 > "32142740|PONG\r\n", len=16, ft=0x7fffffffe050) at stream_paf.c:243 > #4 0x000000000052bbdd in s5_paf_check (pv=0x80374d000, > ps=0x80ffbd350, ssn=0x8c2942bf0, data=0x8c7c36f48 "32142740|PONG\r\n", > len=16, total=32, seq=3414157013, port=5600, flags=0x7fffffffe2b0, > fuzz=150) > at stream_paf.c:437 > #5 0x0000000000520766 in flush_pdu_ackd (config=0x80372c000, > ssn=0x80ffbd1e0, trk=0x80ffbd338, pkt=0xf18ad0, flags=0x7fffffffe2b0) > at snort_stream_tcp.c:9571 > #6 0x000000000051ff15 in CheckFlushPolicyOnAck (config=0x80372c000, > tcpssn=0x80ffbd1e0, talker=0x80ffbd338, listener=0x80ffbd1e0, > tdb=0x7fffffffe710, p=0xf18ad0) at snort_stream_tcp.c:9729 > #7 0x0000000000518e0f in ProcessTcp (scb=0x8c2942bf0, p=0xf18ad0, > tdb=0x7fffffffe710, s5TcpPolicy=0x812c06000) at snort_stream_tcp.c:9260 > #8 0x0000000000514fd4 in StreamProcessTcp (p=0xf18ad0, > scb=0x8c2942bf0, s5TcpPolicy=0x812c06000, skey=0x7fffffffe7c0) at > snort_stream_tcp.c:5655 > #9 0x00000000004dc96b in StreamProcess (p=0xf18ad0, context=0x0) at > spp_stream6.c:751 > #10 0x000000000044f542 in DispatchPreprocessors (p=0xf18ad0, > policy_id=0, policy=0x802fb2000) at detect.c:136 > #11 0x000000000044ef88 in Preprocess (p=0xf18ad0) at detect.c:234 > #12 0x000000000043e9e8 in ProcessPacket (p=0xf18ad0, > pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "", ft=0x0) at snort.c:1873 > #13 0x0000000000445608 in PacketCallback (user=0x0, > pkthdr=0x7fffffffe9a0, pkt=0x821a3f77a "") at snort.c:1718 > #14 0x000000000056dc6a in pcap_process_loop () > #15 0x00000008014d0554 in pcap_platform_finddevs () from > /lib/libpcap.so.8 > #16 0x000000000056d7d8 in pcap_daq_acquire () > #17 0x000000000046b66b in DAQ_Acquire (max=0, callback=0x445420 > <PacketCallback>, user=0x0) at sfdaq.c:541 > #18 0x000000000043e47c in PacketLoop () at snort.c:3268 > #19 0x000000000043d3d9 in SnortMain (argc=6, argv=0x7fffffffec90) at > snort.c:921 > #20 0x000000000043d1f8 in main (argc=6, argv=0x7fffffffec90) at > snort.c:817 > > Thread 1 (Thread 815b9f800 (LWP 100714/snort)): > #0 0x000000080209a8ba in nanosleep () from /lib/libc.so.7 > #1 0x0000000801fd72ea in sleep () from /lib/libc.so.7 > #2 0x0000000801d5ec63 in sleep () from /lib/libthr.so.3 > #3 0x0000000000446448 in ReloadConfigThread (data=0x0) at snort.c:5695 > #4 0x0000000801d5c4f5 in pthread_create () from /lib/libthr.so.3 > #5 0x0000000000000000 in ?? () > #0 0x0000000804c48193 in ?? () from > /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so > (gdb) > (gdb) quit > > /Elof > > > On Thu, 4 Jun 2015, Hui Cao (huica) wrote: > >> Thanks! >> >> The issue happens on smtp preprocessor, but the so is not compiled with >> debug enabled. Can you recompile it with ―enable-debug ? >> >> Best, >> Hui. >> >> On 6/4/15, 12:10 PM, "[email protected]" <[email protected]> wrote: >> >>> >>> So I just had a signal 6... >>> >>> I assume I can't attach files to the mailing list, so here it is, >>> directly >>> in the mailbody. :-) >>> >>> >>> >>> >>> >>> gdb /usr/local/bin/snort 11057 >>> >>> GNU gdb 6.1.1 [FreeBSD] >>> Copyright 2004 Free Software Foundation, Inc. >>> GDB is free software, covered by the GNU General Public License, and >>> you >>> are >>> welcome to change it and/or distribute copies of it under certain >>> conditions. >>> Type "show copying" to see the conditions. >>> There is absolutely no warranty for GDB. Type "show warranty" for >>> details. >>> This GDB was configured as "amd64-marcel-freebsd"... >>> Attaching to program: /usr/local/bin/snort, process 11057 >>> Reading symbols from /usr/local/lib/libdnet.so.1...done. >>> Loaded symbols for /usr/local/lib/libdnet.so.1 >>> Reading symbols from /usr/local/lib/libpcre.so.1...done. >>> Loaded symbols for /usr/local/lib/libpcre.so.1 >>> Reading symbols from /lib/libm.so.5...done. >>> Loaded symbols for /lib/libm.so.5 >>> Reading symbols from /lib/libcrypto.so.6...done. >>> Loaded symbols for /lib/libcrypto.so.6 >>> Reading symbols from /lib/libpcap.so.8...done. >>> Loaded symbols for /lib/libpcap.so.8 >>> Reading symbols from /usr/local/lib/libsfbpf.so.0...done. >>> Loaded symbols for /usr/local/lib/libsfbpf.so.0 >>> Reading symbols from /lib/libz.so.6...done. >>> Loaded symbols for /lib/libz.so.6 >>> Reading symbols from /usr/lib/liblzma.so.5...done. >>> Loaded symbols for /usr/lib/liblzma.so.5 >>> Reading symbols from /lib/libthr.so.3...done. >>> [New Thread 815a59400 (LWP 100459/snort)] >>> [New Thread 802407400 (LWP 100375/snort)] >>> Loaded symbols for /lib/libthr.so.3 >>> Reading symbols from /lib/libc.so.7...done. >>> Loaded symbols for /lib/libc.so.7 >>> Reading symbols from >>> /usr/local/lib/snort_dynamicengine/libsf_engine.so...done. >>> Loaded symbols for /usr/local/lib/snort_dynamicengine/libsf_engine.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dce2_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dnp3_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_dns_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so...don >>> >>> e. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ftptelnet_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_gtp_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_imap_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so...done. >>> >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_modbus_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_pop_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so...do >>> >>> ne. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_reputation_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sdf_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_sip_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssh_preproc.so >>> Reading symbols from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so...done. >>> Loaded symbols for >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_ssl_preproc.so >>> Reading symbols from /libexec/ld-elf.so.1...done. >>> Loaded symbols for /libexec/ld-elf.so.1 >>> [Switching to Thread 815a59400 (LWP 100459/snort)] >>> 0x0000000801faa40c in nanosleep () from /lib/libc.so.7 >>> (gdb) set logging file gdb-snort.txt >>> (gdb) set logging on >>> Copying output to gdb-snort.txt. >>> (gdb) continue >>> Continuing. >>> >>> >>> >>> <...it has just been a few minutes when I receive a SIGABRT> >>> >>> >>> >>> Program received signal SIGABRT, Aborted. >>> [Switching to Thread 802407400 (LWP 100375/snort)] >>> 0x0000000801f2364c in thr_kill () from /lib/libc.so.7 >>> >>> (gdb) backtrace full >>> #0 0x0000000801f2364c in thr_kill () from /lib/libc.so.7 >>> No symbol table info available. >>> #1 0x0000000801fc7c4b in abort () from /lib/libc.so.7 >>> No symbol table info available. >>> #2 0x0000000801fab315 in __assert () from /lib/libc.so.7 >>> No symbol table info available. >>> #3 0x0000000805068395 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> No symbol table info available. >>> #4 0x0000000805068781 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> No symbol table info available. >>> #5 0x000000080506afd0 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> No symbol table info available. >>> #6 0x000000080506b85b in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> No symbol table info available. >>> #7 0x000000080506c150 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> No symbol table info available. >>> #8 0x000000080506cb27 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> No symbol table info available. >>> #9 0x00000000004423f7 in DispatchPreprocessors (p=0x8033a3e00, >>> policy_id=0, policy=0x802faa000) at detect.c:136 >>> scb = (SessionControlBlock *) 0x8a1aad2f0 >>> ppn = (PreprocEvalFuncNode *) 0x8033ff0a0 >>> pps_enabled_foo = 1123336 >>> alerts_processed = true >>> #10 0x000000000044286d in Preprocess (p=0x8033a3e00) at detect.c:234 >>> retval = 0 >>> policy_id = 0 >>> policy = (SnortPolicy *) 0x802faa000 >>> pktcnt = 0 >>> snort_ticks_start = 34413886976 >>> snort_ticks_end = 34413888664 >>> #11 0x00000000004e44b8 in _flush_to_seq (tcpssn=0x80811ce50, >>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4, >>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4352 >>> tmp_do_detect = 1 >>> tmp_do_detect_content = 1 >>> snort_ticks_start = 37073416192 >>> snort_ticks_end = 37069258752 >>> start_seq = 846966387 >>> stop_seq = 1940818286 >>> footprint = 3644 >>> bytes_processed = 3644 >>> flushed_bytes = 3644 >>> pkth = {ts = {tv_sec = 100375, tv_usec = 0}, caplen = 0, pktlen >>> = 0, >>> ingress_index = -1, egress_index = -1, ingress_group = -1, >>> egress_group = >>> -1, flags = 0, opaque = 8, priv_ptr = 0x8a1800000, flow_id = 535241216, >>> address_space_id = 0} >>> enc_flags = 2147483648 >>> snort_ticks_start = 51544732022 >>> snort_ticks_end = 113187 >>> #12 0x00000000004e3e06 in flush_to_seq (tcpssn=0x80811ce50, >>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4, >>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4493 >>> No locals. >>> #13 0x00000000004e4f8e in flush_ackd (tcpssn=0x80811ce50, >>> st=0x80811cfa8, >>> p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8, sp=39946, dp=6400, >>> dir=128) at snort_stream_tcp.c:4559 >>> bytes = 3644 >>> #14 0x00000000004e3bcb in flush_stream (tcpssn=0x80811ce50, >>> st=0x80811cfa8, p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8, >>> sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4588 >>> fm = (FlushMgr *) 0x80811cfb4 >>> #15 0x00000000004e60b4 in FlushQueuedSegs (scb=0x8a1aad2f0, >>> tcpssn=0x80811ce50) at snort_stream_tcp.c:5074 >>> p = (Packet *) 0x8033a4900 >>> flushed = 1926 >>> tmp_pcap_hdr = {ts = {tv_sec = 1433431165, tv_usec = 321125}, >>> caplen = >>> 94, pktlen = 94, ingress_index = 5004089, egress_index = 0, >>> ingress_group >>> = 38246208, egress_group = 8, flags = 4294960320, opaque = 32767, >>> priv_ptr = 0x4b4c81, flow_id = 0, address_space_id = 0} >>> #16 0x00000000004e61bd in TcpSessionCleanup (scb=0x8a1aad2f0, >>> freeApplicationData=1) at snort_stream_tcp.c:5115 >>> tcpssn = (TcpSession *) 0x80811ce50 >>> #17 0x00000000004e61ea in TcpSessionCleanupWithFreeApplicationData >>> (scb=0x8a1aad2f0) at snort_stream_tcp.c:5122 >>> No locals. >>> #18 0x00000000004e69e1 in StreamProcessTcp (p=0xee2a40, >>> scb=0x8a1aad2f0, >>> s5TcpPolicy=0x812807000, skey=0x7fffffffe6c0) at >>> snort_stream_tcp.c:5648 >>> sscc = {old_mem_in_use = 15788887, client_ip = {family = 2, bits >>> = 32, >>> ip = {u6_addr8 = "\nm\027L", '\0' <repeats 11 times>, u6_addr16 = >>> {27914, >>> 19479, 0, 0, 0, 0, 0, 0}, u6_addr32 = {1276603658, 0, 0, 0}}}, >>> server_ip >>> = {family = 2, bits = 32, ip = {u6_addr8 = "\nm\026\024", '\0' <repeats >>> 11 times>, >>> u6_addr16 = {27914, 5142, 0, 0, 0, 0, 0, 0}, u6_addr32 = >>> {337014026, 0, 0, 0}}}, client_port = 39946, server_port = 6400, >>> lw_session_state = 200, lw_session_flags = 4284679, app_proto_id = 0} >>> tdb = {seq = 1940818286, ack = 2349672268, win = 64032, end_seq = >>> 1940818325, ts = 0} >>> rc = 0 >>> status = 4512282 >>> snort_ticks_start = 34397587520 >>> snort_ticks_end = 140737488348864 >>> #19 0x00000000004b5a14 in StreamProcess (p=0xee2a40, context=0x0) at >>> spp_stream6.c:751 >>> key = {ip_l = {0, 0, 4216431, 0}, ip_h = {0, 2, 362856224, 8}, >>> port_l = >>> 59136, port_h = 65535, vlan_tag = 32767, protocol = 0 '\0', pad = 0 >>> '\0', >>> mplsLabel = 5328944, addressSpaceId = 0, addressSpaceIdPad1 = 0} >>> scb = (SessionControlBlock *) 0x8a1aad2f0 >>> snort_ticks_start = 140737488348960 >>> snort_ticks_end = 34722594592 >>> #20 0x00000000004423f7 in DispatchPreprocessors (p=0xee2a40, >>> policy_id=0, >>> policy=0x802faa000) at detect.c:136 >>> scb = (SessionControlBlock *) 0x8a1aad2f0 >>> ppn = (PreprocEvalFuncNode *) 0x815b61340 >>> pps_enabled_foo = 1123336 >>> alerts_processed = true >>> #21 0x000000000044286d in Preprocess (p=0xee2a40) at detect.c:234 >>> retval = 0 >>> policy_id = 0 >>> policy = (SnortPolicy *) 0x802faa000 >>> pktcnt = 0 >>> snort_ticks_start = 0 >>> snort_ticks_end = 6059431713369489410 >>> #22 0x00000000004351b3 in ProcessPacket (p=0xee2a40, >>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?, ft=0x0) at >>> snort.c:1873 >>> verdict = DAQ_VERDICT_PASS >>> __func__ = "ProcessPacket" >>> #23 0x0000000000434ccd in PacketCallback (user=0x0, >>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?) at snort.c:1718 >>> inject = 0 >>> verdict = DAQ_VERDICT_PASS >>> snort_ticks_start = 34894979584 >>> snort_ticks_end = 34367935488 >>> #24 0x000000000052fe34 in pcap_process_loop () >>> No symbol table info available. >>> #25 0x0000000801429dcd in pcap_create_interface () from >>> /lib/libpcap.so.8 >>> No symbol table info available. >>> #26 0x000000000053025f in pcap_daq_acquire () >>> No symbol table info available. >>> #27 0x000000000045a1b4 in DAQ_Acquire (max=0, callback=0x434b40 >>> <PacketCallback>, user=0x0) at sfdaq.c:541 >>> err = 0 >>> #28 0x0000000000437616 in PacketLoop () at snort.c:3268 >>> error = 0 >>> pkts_to_read = 0 >>> #29 0x00000000004337c7 in SnortMain (argc=6, argv=0x7fffffffebc0) at >>> snort.c:921 >>> tmp_ptr = 0x0 >>> intf = 0x8024c4540 "mon0" >>> daqInit = 1 >>> #30 0x000000000043364f in main (argc=6, argv=0x7fffffffebc0) at >>> snort.c:817 >>> No locals. >>> rax 0x0 0 >>> rbx 0x7fffffffddec 140737488346604 >>> rcx 0x801fc8fbc 34393067452 >>> rdx 0x0 0 >>> rsi 0x6 6 >>> rdi 0x18817 100375 >>> rbp 0x7fffffffde60 0x7fffffffde60 >>> rsp 0x7fffffffddd8 0x7fffffffddd8 >>> r8 0x0 0 >>> r9 0xfffffe0032ea54a8 -2198169037656 >>> r10 0x59 89 >>> r11 0x202 514 >>> r12 0x80811ce50 34495123024 >>> r13 0x8033a51b8 34413892024 >>> r14 0x82251deaa 34935529130 >>> r15 0x1ba24 113188 >>> rip 0x801f2364c 0x801f2364c <thr_kill+12> >>> eflags 0x206 518 >>> cs 0x43 67 >>> ss 0x3b 59 >>> ds 0x0 0 >>> es 0x0 0 >>> fs 0x0 0 >>> gs 0x0 0 >>> 0x801f2364c <thr_kill+12>: jb 0x801f2364f <thr_kill+15> >>> 0x801f2364e <thr_kill+14>: retq >>> 0x801f2364f <thr_kill+15>: mov 0x2d6bea(%rip),%rcx # >>> 0x8021fa240 <__nsdefaultsrc+5696> >>> 0x801f23656 <thr_kill+22>: jmpq *%rcx >>> 0x801f23658 <thr_kill+24>: nop >>> 0x801f23659 <thr_kill+25>: nop >>> 0x801f2365a <thr_kill+26>: nop >>> 0x801f2365b <thr_kill+27>: nop >>> 0x801f2365c <thr_kill+28>: nop >>> 0x801f2365d <thr_kill+29>: nop >>> 0x801f2365e <thr_kill+30>: nop >>> 0x801f2365f <thr_kill+31>: nop >>> 0x801f23660 <thr_self>: mov $0x1b0,%rax >>> 0x801f23667 <thr_self+7>: mov %rcx,%r10 >>> 0x801f2366a <thr_self+10>: syscall >>> 0x801f2366c <thr_self+12>: jb 0x801f2366f <thr_self+15> >>> >>> Thread 2 (Thread 802407400 (LWP 100375/snort)): >>> #0 0x0000000801f2364c in thr_kill () from /lib/libc.so.7 >>> #1 0x0000000801fc7c4b in abort () from /lib/libc.so.7 >>> #2 0x0000000801fab315 in __assert () from /lib/libc.so.7 >>> #3 0x0000000805068395 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> #4 0x0000000805068781 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> #5 0x000000080506afd0 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> #6 0x000000080506b85b in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> #7 0x000000080506c150 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> #8 0x000000080506cb27 in ?? () from >>> /usr/local/lib/snort_dynamicpreprocessor//libsf_smtp_preproc.so >>> #9 0x00000000004423f7 in DispatchPreprocessors (p=0x8033a3e00, >>> policy_id=0, policy=0x802faa000) at detect.c:136 >>> #10 0x000000000044286d in Preprocess (p=0x8033a3e00) at detect.c:234 >>> #11 0x00000000004e44b8 in _flush_to_seq (tcpssn=0x80811ce50, >>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4, >>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4352 >>> #12 0x00000000004e3e06 in flush_to_seq (tcpssn=0x80811ce50, >>> st=0x80811cfa8, bytes=3644, p=0x8033a4900, sip=0x8033a51a4, >>> dip=0x8033a51b8, sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4493 >>> #13 0x00000000004e4f8e in flush_ackd (tcpssn=0x80811ce50, >>> st=0x80811cfa8, >>> p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8, sp=39946, dp=6400, >>> dir=128) at snort_stream_tcp.c:4559 >>> #14 0x00000000004e3bcb in flush_stream (tcpssn=0x80811ce50, >>> st=0x80811cfa8, p=0x8033a4900, sip=0x8033a51a4, dip=0x8033a51b8, >>> sp=39946, dp=6400, dir=128) at snort_stream_tcp.c:4588 >>> #15 0x00000000004e60b4 in FlushQueuedSegs (scb=0x8a1aad2f0, >>> tcpssn=0x80811ce50) at snort_stream_tcp.c:5074 >>> #16 0x00000000004e61bd in TcpSessionCleanup (scb=0x8a1aad2f0, >>> freeApplicationData=1) at snort_stream_tcp.c:5115 >>> #17 0x00000000004e61ea in TcpSessionCleanupWithFreeApplicationData >>> (scb=0x8a1aad2f0) at snort_stream_tcp.c:5122 >>> #18 0x00000000004e69e1 in StreamProcessTcp (p=0xee2a40, >>> scb=0x8a1aad2f0, >>> s5TcpPolicy=0x812807000, skey=0x7fffffffe6c0) at >>> snort_stream_tcp.c:5648 >>> #19 0x00000000004b5a14 in StreamProcess (p=0xee2a40, context=0x0) at >>> spp_stream6.c:751 >>> #20 0x00000000004423f7 in DispatchPreprocessors (p=0xee2a40, >>> policy_id=0, >>> policy=0x802faa000) at detect.c:136 >>> #21 0x000000000044286d in Preprocess (p=0xee2a40) at detect.c:234 >>> #22 0x00000000004351b3 in ProcessPacket (p=0xee2a40, >>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?, ft=0x0) at >>> snort.c:1873 >>> #23 0x0000000000434ccd in PacketCallback (user=0x0, >>> pkthdr=0x7fffffffe8c0, pkt=0x82251deaa "\220?"\b?) at snort.c:1718 >>> #24 0x000000000052fe34 in pcap_process_loop () >>> #25 0x0000000801429dcd in pcap_create_interface () from >>> /lib/libpcap.so.8 >>> #26 0x000000000053025f in pcap_daq_acquire () >>> #27 0x000000000045a1b4 in DAQ_Acquire (max=0, callback=0x434b40 >>> <PacketCallback>, user=0x0) at sfdaq.c:541 >>> #28 0x0000000000437616 in PacketLoop () at snort.c:3268 >>> #29 0x00000000004337c7 in SnortMain (argc=6, argv=0x7fffffffebc0) at >>> snort.c:921 >>> #30 0x000000000043364f in main (argc=6, argv=0x7fffffffebc0) at >>> snort.c:817 >>> >>> Thread 1 (Thread 815a59400 (LWP 100459/snort)): >>> #0 0x0000000801faa40c in nanosleep () from /lib/libc.so.7 >>> #1 0x0000000801f15a58 in sleep () from /lib/libc.so.7 >>> #2 0x0000000801ca8078 in sleep () from /lib/libthr.so.3 >>> #3 0x000000000043b215 in ReloadConfigThread (data=0x0) at snort.c:5695 >>> #4 0x0000000801ca5dc4 in pthread_getprio () from /lib/libthr.so.3 >>> #5 0x0000000000000000 in ?? () >>> #0 0x0000000801f2364c in thr_kill () from /lib/libc.so.7 >>> The program is running. Quit anyway (and detach it)? (y or n) >>> Detaching >>> from program: /usr/local/bin/snort, process 11057 >>> >>> >>> >>> >>> As gdb detached from snort, I got the signal 6 in my syslog: >>> 2015-06-04 17:51:53 +02:00 foobar kernel: pid 11057 (snort), uid 100: >>> exited on signal 6 >>> >>> >>> >>> >>> >>> So, this time we got a signal 6 but during this sensor's 14 hour uptime >>> we've seen: >>> pid 1199 (snort), uid 100: exited on signal 10 >>> pid 4503 (snort), uid 100: exited on signal 10 >>> pid 5908 (snort), uid 100: exited on signal 10 >>> pid 11057 (snort), uid 100: exited on signal 6 >>> >>> >>> >>> >>> >>> I hope this gdb was helpful. >>> Let me know if it should be run again. >>> >>> >>> >>> >>> >>> This was all performed on a sensor running: >>> >>> ,,_ -*> Snort! <*- >>> o" )~ Version 2.9.7.3 (Build 217) >>> '''' By Martin Roesch & The Snort Team: >>> http://www.snort.org/contact#team >>> Copyright (C) 2014-2015 Cisco and/or its affiliates. All >>> rights reserved. >>> Copyright (C) 1998-2013 Sourcefire, Inc., et al. >>> Using libpcap version 1.4.0 >>> Using PCRE version: 8.37 2015-04-28 >>> Using ZLIB version: 1.2.8 >>> >>> >>> daq-2.0.5 >>> >>> FreeBSD 9.3-RELEASE-p13 >>> >>> >>> /Elof >>> >>> >>> >>> >>> >>> On Thu, 4 Jun 2015, Hui Cao (huica) wrote: >>> >>>> That¹s cool. All looks good to me. No need to do more things... >>>> >>>> Best, >>>> Hui >>>> >>>> On 6/4/15, 11:35 AM, "[email protected]" <[email protected]> wrote: >>>> >>>>> >>>>> Hi Hui. >>>>> >>>>> That much I know. It is the debugging steps I'm curious about. >>>>> >>>>> (I think you forgot one important first command: continue ) >>>>> >>>>> >>>>> Is this a good start: >>>>> >>>>> gdb /path/to/snort 1222 >>>>> (gdb) set logging file gdb-snort.txt >>>>> (gdb) set logging on >>>>> (gdb) continue >>>>> >>>>> <wait for it to crash> >>>>> >>>>> (gdb) backtrace full >>>>> (gdb) info registers >>>>> (gdb) x/16i $pc >>>>> (gdb) thread apply all backtrace >>>>> (gdb) quit >>>>> >>>>> Email the report. >>>>> >>>>> >>>>> Should I prepare more stuff before the 'continue'? >>>>> Like "handle SIG33 pass nostop noprint" or something? >>>>> >>>>> /Elof >>>>> >>>>> >>>>> On Thu, 4 Jun 2015, Hui Cao (huica) wrote: >>>>> >>>>>> Try >>>>>> >>>>>> Assume snort pid is 1222 >>>>>> >>>>>> gdb /path/to/snort 1222 >>>>>> >>>>>> Best, >>>>>> Hui. >>>>>> On 6/4/15, 10:37 AM, "[email protected]" <[email protected]> wrote: >>>>>> >>>>>>> >>>>>>> An update: >>>>>>> >>>>>>> On a sensor where snort crashed with signal 6 three times, I >>>>>>> downgraded >>>>>>> daq to 2.0.4_1 and rebooted the machine to rule out if the problem >>>>>>> seem >>>>>>> to >>>>>>> be in 'snort' or 'daq'. >>>>>>> >>>>>>> With snort 2.9.7.3 and daq 2.0.4_1 I got signal 6 again. >>>>>>> >>>>>>> >>>>>>> This make me believe that there's something wrong in snort 2.9.7.3 >>>>>>> and >>>>>>> not in daq 2.0.5. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On this sensor I have now done the opposite, upgraded daq to 2.0.5 >>>>>>> and >>>>>>> downgraded snort to 2.9.7.2 to see if I get any more signal 6. >>>>>>> >>>>>>> On another sensor, I'm running 2.9.7.3 (compiled with debug) and >>>>>>> daq >>>>>>> 2.0.5 >>>>>>> without chroot and uid/gid change, i.e. running as root, in >>>>>>> order to >>>>>>> create a core file, if the problem happen again. >>>>>>> (if it doesn't happen on this sensor, I guess the problem lies >>>>>>> somewhere >>>>>>> in the chrooting code in snort. I know it has been updated between >>>>>>> 2.9.7.2 >>>>>>> and 2.9.7.3) >>>>>>> >>>>>>> >>>>>>> >>>>>>> Russ C also wrote: >>>>>>>> Elof - since this is happening frequently, you could try attaching >>>>>>>> the >>>>>>>> debugger to one of your Snort processes and wait wait for >>>>>>>> segfault. >>>>>>> >>>>>>> I know too little about debugging. :-/ Can you give me instructions >>>>>>> or >>>>>>> point me to a guide that describes the steps I should take? >>>>>>> >>>>>>> >>>>>>> >>>>>>> /Elof >>>>>>> >>>>>>> >>>>>>> On Thu, 4 Jun 2015, [email protected] wrote: >>>>>>> >>>>>>>> >>>>>>>> Five different sensors have now had bus errors (signal 10), >>>>>>>> segmentation >>>>>>>> faults (signal 11) and even signal 6 (SIGABRT). >>>>>>>> >>>>>>>> My snort config uses both chroot and dropping user privileges, so >>>>>>>> even >>>>>>>> if >>>>>>>> I start out as root with ulimit unlimited, this doesn't seem to be >>>>>>>> in >>>>>>>> effect >>>>>>>> after the chroot/uid-change. >>>>>>>> >>>>>>>> So currently I have no core-file to debug. :-/ >>>>>>>> >>>>>>>> Anyone know how to set the ulimits for a chrooted and >>>>>>>> uid/gid-changed >>>>>>>> process in FreeBSD? >>>>>>>> >>>>>>>> /Elof >>>>>>>> >>>>>>>> >>>>>>>> On Thu, 4 Jun 2015, [email protected] wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> Hi Hui! >>>>>>>>> >>>>>>>>> Yes, the dynamic engine/preproc files are updated as well. >>>>>>>>> >>>>>>>>> Last night the problem reocurred, so this seem to be >>>>>>>>> reproduceable. >>>>>>>>> Good. >>>>>>>>> Then there's a good chance this problem can be sorted out. >>>>>>>>> >>>>>>>>> >>>>>>>>> A few minutes ago a signal 10 happened on another sensor (running >>>>>>>>> FreeBSD 10.1 amd64), so the problem must be in DAQ 2.0.5 or in >>>>>>>>> Snort >>>>>>>>> 2.9.7.3 and not in the hardware nor in FreeBSD. >>>>>>>>> >>>>>>>>> >>>>>>>>> I will compile a debug-snort and try to generate core files. >>>>>>>>> I'll let you know the outcome next week. >>>>>>>>> >>>>>>>>> /Elof >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, 3 Jun 2015, Hui cao wrote: >>>>>>>>> >>>>>>>>>> Hi Elof, >>>>>>>>>> >>>>>>>>>> Are snort and snort dynamic preprocessors are in sync? >>>>>>>>>> >>>>>>>>>> If so, can you help us get a backtrace from the crush? You need >>>>>>>>>> 1) build snort with ./configure --enable-debug >>>>>>>>>> 2) allowing core dump (ulimit -c unlimited) >>>>>>>>>> 3) run the snort >>>>>>>>>> 4) use "gdb snort core_file " and them type "bt" in the gdb >>>>>>>>>> command >>>>>>>>>> line >>>>>>>>>> >>>>>>>>>> Best, >>>>>>>>>> Hui. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On 06/03/2015 05:51 AM, [email protected] wrote: >>>>>>>>>>> Hi all! >>>>>>>>>>> >>>>>>>>>>> This is just a report to inform that after I updated snort and >>>>>>>>>>> DAQ >>>>>>>>>>> to the >>>>>>>>>>> latest versions, one of my sensors started throwing signal 10 >>>>>>>>>>> (bus >>>>>>>>>>> error) >>>>>>>>>>> and signal 11 (segmentation fault). >>>>>>>>>>> >>>>>>>>>>> # uptime >>>>>>>>>>> 11:32AM up 1 day, 9:48, 1 user, load averages: 0.36, 0.37, >>>>>>>>>>> 0.38 >>>>>>>>>>> # dmesg | grep snort >>>>>>>>>>> pid 1183 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 16920 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 17502 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 18862 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 20223 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 20927 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 1193 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 2447 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 3811 (snort), uid 100: exited on signal 10 >>>>>>>>>>> pid 7881 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 9252 (snort), uid 100: exited on signal 10 >>>>>>>>>>> pid 25593 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 26627 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 56658 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 57237 (snort), uid 100: exited on signal 10 >>>>>>>>>>> pid 58595 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 68639 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 70008 (snort), uid 100: exited on signal 11 >>>>>>>>>>> pid 71361 (snort), uid 100: exited on signal 10 >>>>>>>>>>> pid 72725 (snort), uid 100: exited on signal 11 >>>>>>>>>>> >>>>>>>>>>> 20 crashes in a day... >>>>>>>>>>> A reboot didn't help. >>>>>>>>>>> >>>>>>>>>>> This sensor has never behaved like this during its lifetime (1 >>>>>>>>>>> year). >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> FreeBSD 9.3 amd64 >>>>>>>>>>> >>>>>>>>>>> ,,_ -*> Snort! <*- >>>>>>>>>>> o" )~ Version 2.9.7.3 (Build 217) >>>>>>>>>>> '''' By Martin Roesch & The Snort Team: >>>>>>>>>>> http://www.snort.org/contact#team >>>>>>>>>>> Copyright (C) 2014-2015 Cisco and/or its >>>>>>>>>>> affiliates. >>>>>>>>>>> All rights >>>>>>>>>>> reserved. >>>>>>>>>>> Copyright (C) 1998-2013 Sourcefire, Inc., et al. >>>>>>>>>>> Using libpcap version 1.4.0 >>>>>>>>>>> Using PCRE version: 8.37 2015-04-28 >>>>>>>>>>> Using ZLIB version: 1.2.8 >>>>>>>>>>> >>>>>>>>>>> daq-2.0.5 >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Bus errors are quite unusual in general, so I'll keep >>>>>>>>>>> looking at >>>>>>>>>>> this, >>>>>>>>>>> trying to see if it is e.g. paging errors. >>>>>>>>>>> It doesn't look like it though: >>>>>>>>>>> # swapinfo >>>>>>>>>>> Device 1K-blocks Used Avail Capacity >>>>>>>>>>> /dev/mirror/swap 4194300 0 4194300 0% >>>>>>>>>>> >>>>>>>>>>> The machine doesn't seem to be overheated either: >>>>>>>>>>> System Temp: 30 degrees C >>>>>>>>>>> Peripheral Temp: 40 degrees C >>>>>>>>>>> CPU Temp: Low >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> If you need me to do something special to debug this >>>>>>>>>>> further, let >>>>>>>>>>> me >>>>>>>>>>> know. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> PS. It is only one sensor, out of 20, that behaves like >>>>>>>>>>> this. So >>>>>>>>>>> perhaps >>>>>>>>>>> it is something in the mirrored traffic that make DAQ or snort >>>>>>>>>>> point >>>>>>>>>>> at >>>>>>>>>>> illegal memory addresses and crash. >>>>>>>>>>> Or this particular machine is having hardware issues. >>>>>>>>>>> However, it >>>>>>>>>>> is >>>>>>>>>>> strange that those hw-issues should suddenly start right >>>>>>>>>>> after I >>>>>>>>>>> updated >>>>>>>>>>> the software on the machine... >>>>>>>>>>> >>>>>>>>>>> When I write this, the current snort process has been alive >>>>>>>>>>> for 5 >>>>>>>>>>> hours. >>>>>>>>>>> It's going to be interesting to see if the traffic tonight will >>>>>>>>>>> cause it >>>>>>>>>>> to crash many times again. >>>>>>>>>>> >>>>>>>>>>> /Elof >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> ------------------------------------------------------------------ >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> -- >>>>>>>>>>> -------- >>>>>>>>>>> _______________________________________________ >>>>>>>>>>> Snort-devel mailing list >>>>>>>>>>> [email protected] >>>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel >>>>>>>>>>> Archive: >>>>>>>>>>> >>>>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-deve >>>>>>>>>>> >>>>>>>>>>> l >>>>>>>>>>> >>>>>>>>>>> Please visit http://blog.snort.org for the latest news about >>>>>>>>>>> Snort! >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ------------------------------------------------------------------- >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> -- >>>>>>>>>> ------- >>>>>>>>>> _______________________________________________ >>>>>>>>>> Snort-devel mailing list >>>>>>>>>> [email protected] >>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel >>>>>>>>>> Archive: >>>>>>>>>> >>>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Please visit http://blog.snort.org for the latest news about >>>>>>>>>> Snort! >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -------------------------------------------------------------------- >>>>>>>>> >>>>>>>>> -- >>>>>>>>> -- >>>>>>>>> ------ >>>>>>>>> _______________________________________________ >>>>>>>>> Snort-devel mailing list >>>>>>>>> [email protected] >>>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel >>>>>>>>> Archive: >>>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel >>>>>>>>> >>>>>>>>> >>>>>>>>> Please visit http://blog.snort.org for the latest news about >>>>>>>>> Snort! >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> --------------------------------------------------------------------- >>>>>>>> >>>>>>>> -- >>>>>>>> -- >>>>>>>> ----- >>>>>>>> _______________________________________________ >>>>>>>> Snort-devel mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.sourceforge.net/lists/listinfo/snort-devel >>>>>>>> Archive: >>>>>>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel >>>>>>>> >>>>>>>> >>>>>>>> Please visit http://blog.snort.org for the latest news about >>>>>>>> Snort! >>>>>>>> >>>>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------- >>>> >>>> ----- >>>> _______________________________________________ >>>> Snort-devel mailing list >>>> [email protected] >>>> https://lists.sourceforge.net/lists/listinfo/snort-devel >>>> Archive: >>>> http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel >>>> >>>> Please visit http://blog.snort.org for the latest news about Snort! >> >> ------------------------------------------------------------------------------ _______________________________________________ Snort-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/snort-devel Archive: http://sourceforge.net/mailarchive/forum.php?forum_name=snort-devel Please visit http://blog.snort.org for the latest news about Snort!