RE: Tracking Heap Corruption bug

Looks like another domain is triggering this problem now.   Same pattern as before, randomly generated subdomains that don't exist.  Main domain does exist and has spf record.  Username in from field is always "ret".  Following are two back traces and the contents of the spf_request variables.

(gdb) bt
#0  0x4010954b in free () from /lib/libc.so.6
#1  0x401093d3 in free () from /lib/libc.so.6
#2  0x4008a283 in SPF_dns_rr_free (spfrr=0x8505660) at spf_dns_rr.c:115
#3  0x400908d9 in SPF_server_get_record (spf_server=0x839d6a0,
    spf_request=0x83bd718, spf_response=0x8529938, spf_recordp=0xbf5ff980)
    at spf_server.c:351
#4  0x4008f8ac in SPF_request_query_mailfrom (spf_request=0x83bd718,
    spf_responsep=0xbf5ff9b8) at spf_request.c:253
#5  0x08066313 in SPFCheck (MailData=0x84fb170, spf_server=0x839d6a0)
    at milter-greylist.c:510
#6  0x080665a2 in GreyList (ctx=0x8524f50, MailData=0x84fb170)
    at milter-greylist.c:624
#7  0x08064782 in mlfi_envrcpt (ctx=0x8524f50, argv=0x84f5020) at milter.c:140
#8  0x0806b9e7 in st_rcpt ()
#9  0x0806b298 in mi_engine ()
#10 0x08069e09 in mi_handle_session ()
#11 0x08069685 in mi_thread_handle_wrapper ()
#12 0x4004b10c in pthread_detach () from /lib/libpthread.so.0
#13 0x4016c60a in clone () from /lib/libc.so.6
(gdb) frame 4
#4  0x4008f8ac in SPF_request_query_mailfrom (spf_request=0x83bd718,
    spf_responsep=0xbf5ff9b8) at spf_request.c:253
253     spf_request.c: No such file or directory.
        in spf_request.c
(gdb) print *spf_request
$1 = {spf_server = 0x839d6a0, client_ver = 2, ipv4 = {s_addr = 2146635860},
  ipv6 = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0,
        0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}},
  env_from = 0x852bf98 "[email protected]",
  helo_dom = 0x85116c0 "x46677.whoisyourbigcrush.com", rcpt_to_dom = 0x0,
  use_local_policy = 0 '\0', use_helo = 0 '\0', env_from_lp = 0x8526e90 "ret",
  env_from_dp = 0x8511730 "x46677.whoisyourbigcrush.com", client_dom = 0x0,
  cur_dom = 0x8511730 "x46677.whoisyourbigcrush.com", max_var_len = 0}




(gdb) bt
#0  0x40108b3d in malloc () from /lib/libc.so.6
#1  0x40109bdb in realloc () from /lib/libc.so.6
#2  0x401098e4 in realloc () from /lib/libc.so.6
#3  0x40086a96 in SPF_c_ensure_capacity (datap=0x84ebbe0, sizep=0x84ebbe4,
    length=32) at spf_compile.c:107
#4  0x4008796c in SPF_c_mech_add (spf_server=0x839d6a0, spf_record=0x84ebbd8,
    spf_response=0x84ec330, mechtype=0x4009234c, prefix=2,
    mech_value=0xbf5ff900) at spf_compile.c:831
#5  0x40088300 in SPF_record_compile (spf_server=0x839d6a0,
    spf_response=0x84ec330, spf_recordp=0xbf5ff980,
    record=0x84eb650 "v=spf1 a mx a:whosurcrush.com mx:whosurcrush.com ip4:89.149.229.152 ~all") at spf_compile.c:1260
#6  0x400908ce in SPF_server_get_record (spf_server=0x839d6a0,
    spf_request=0x84ebea0, spf_response=0x84ec330, spf_recordp=0xbf5ff980)
    at spf_server.c:348
#7  0x4008f8ac in SPF_request_query_mailfrom (spf_request=0x84ebea0,
    spf_responsep=0xbf5ff9b8) at spf_request.c:253
#8  0x08066313 in SPFCheck (MailData=0x84ee390, spf_server=0x839d6a0)
    at milter-greylist.c:510
#9  0x080665a2 in GreyList (ctx=0x84edc30, MailData=0x84ee390)
    at milter-greylist.c:624
#10 0x08064782 in mlfi_envrcpt (ctx=0x84edc30, argv=0x84ebe78) at milter.c:140
#11 0x0806b9e7 in st_rcpt ()
---Type <return> to continue, or q <return> to quit---
#12 0x0806b298 in mi_engine ()
#13 0x08069e09 in mi_handle_session ()
#14 0x08069685 in mi_thread_handle_wrapper ()
#15 0x4004b10c in pthread_detach () from /lib/libpthread.so.0
#16 0x4016c60a in clone () from /lib/libc.so.6
(gdb) frame 7
#7  0x4008f8ac in SPF_request_query_mailfrom (spf_request=0x84ebea0,
    spf_responsep=0xbf5ff9b8) at spf_request.c:253
253     spf_request.c: No such file or directory.
        in spf_request.c
(gdb) print *spf_request
$1 = {spf_server = 0x839d6a0, client_ver = 2, ipv4 = {s_addr = 2565182809},
  ipv6 = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0,
        0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}},
  env_from = 0x84ebf58 "[email protected]",
  helo_dom = 0x84eab90 "j33261.whosurcrush.com", rcpt_to_dom = 0x0,
  use_local_policy = 0 '\0', use_helo = 0 '\0', env_from_lp = 0x84ebf78 "ret",
  env_from_dp = 0x84ebee8 "j33261.whosurcrush.com", client_dom = 0x0,
  cur_dom = 0x84ebee8 "j33261.whosurcrush.com", max_var_len = 0}



-----Original Message-----
From: David Hinkle [mailto:[email protected]]
Sent: Tue 9/11/2007 6:05 PM
To: [email protected]
Subject: [spf-devel] Tracking Heap Corruption bug
 
Hello guys, First of all, thank you very much for producing the SPF libraries.

I am the Sr. developer for CIPAFilter (cipafilter.com) which, among other things, is an anti-spam appliance that implements greylisting, spamassassin, etc.   A few months ago, I implemented SPF in our product by utilizing your library.

Since then,  we've had crashing with segmentation fault and core dumps on an irregular basis on many customers.  (We have a watchdog that brings the milter back up so the customers don't notice)  I've been trying to trouble shoot this for the last 2 weeks but I'm having alot of trouble tracking it down.  I've thrown gdb, dmalloc, and valgrind at the problem but so far I can't get this one fixed.

All the core dumps back trace something similar to this (But not always the same, sometimes a slightly different trace, but this is the most common):

#0  0x4010954b in free () from /lib/libc.so.6
#1  0x401093d3 in free () from /lib/libc.so.6
#2  0x4008a283 in SPF_dns_rr_free (spfrr=0x851d070) at spf_dns_rr.c:115
#3  0x400908d9 in SPF_server_get_record (spf_server=0x839d6a0,
    spf_request=0x8508b98, spf_response=0x84ef970, spf_recordp=0xbf1ff980)
    at spf_server.c:351
#4  0x4008f8ac in SPF_request_query_mailfrom (spf_request=0x8508b98,
    spf_responsep=0xbf1ff9b8) at spf_request.c:253
#5  0x08066313 in SPFCheck (MailData=0x850ca98, spf_server=0x839d6a0)
    at milter-greylist.c:510
#6  0x080665a2 in GreyList (ctx=0x852b968, MailData=0x850ca98)
    at milter-greylist.c:624
#7  0x08064782 in mlfi_envrcpt (ctx=0x852b968, argv=0x850a558) at milter.c:140
#8  0x0806b9e7 in st_rcpt ()
#9  0x0806b298 in mi_engine ()
#10 0x08069e09 in mi_handle_session ()
#11 0x08069685 in mi_thread_handle_wrapper ()
#12 0x4004b10c in pthread_detach () from /lib/libpthread.so.0
#13 0x4016c60a in clone () from /lib/libc.so.6

First thing I think, is that I have a heap corruption issue, that isn't necessarily in the spf library, so I try and trace it down with dmalloc and valgrind.   I'm still working on that angle but no solid leads yet.   

When I examined the core files in more detail, however, I discovered that all the crashes seem to be associated with a particular domain  (I've examined more than a dozen cores from multiple customer sites and it's always the same story).  They're all associated with from addresses that look like this "[email protected]" and "[email protected]".  (Notice there are 2 domains, willfindsomeone1.com and willfindsomeone.com.) 

There are no SPF records for the s40574 part, but there are for the willfindsomeone.com part... But they don't look corrupt or broken to me. 

For the above backtrace:

(gdb) frame 4
#4  0x4008f8ac in SPF_request_query_mailfrom (spf_request=0x8508b98,
    spf_responsep=0xbf1ff9b8) at spf_request.c:253
253     spf_request.c: No such file or directory.
        in spf_request.c
(gdb) print *spf_request
$2 = {spf_server = 0x839d6a0, client_ver = 2, ipv4 = {s_addr = 4188406856},
  ipv6 = {in6_u = {u6_addr8 = '\0' <repeats 15 times>, u6_addr16 = {0, 0, 0,
        0, 0, 0, 0, 0}, u6_addr32 = {0, 0, 0, 0}}},
  env_from = 0x84f4ea0 "[email protected]",
  helo_dom = 0x84f4e38 "j55767.willfindsome1.com", rcpt_to_dom = 0x0,
  use_local_policy = 0 '\0', use_helo = 0 '\0', env_from_lp = 0x84f4ec8 "ret",
  env_from_dp = 0x84f4e58 "j55767.willfindsome1.com", client_dom = 0x0,
  cur_dom = 0x84f4e58 "j55767.willfindsome1.com", max_var_len = 0}

I'm hoping someone will have some hints.  I'm happy to provide core files, answer whatever I can, etc, etc.   Looks like some spammer has found a way to crash the default implementation!

Dmalloc gives me errors like the following:

1189541998: 229886: ERROR: _dmalloc_chunk_heap_check: failed OVER picket-fence magic-number check (err 27)
1189541998: 229886:   error details: checking user pointer
1189541998: 229886:   pointer '0x40b0ce08' from 'unknown' prev access 'ra=0x4008a395'
1189541998: 229886:   dump of proper fence-top bytes: 'i\336\312\372'
1189541998: 229886:   dump of '0x40b0ce08'+59: '.4.166.251 ~all\000\000\000\000\000'
1189541998: 229886:   next pointer '0x40b0ce80' (size 64) may have run under from 'ra=0x4008f1f2'

Looks like part of this SPF entry is over running it's buffer... But I have no idea why that would happen.

David

-------------------------------------------
-----------------------------------------------------------------------
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?&
Powered by Listbox: http://www.listbox.com

-------------------------------------------
-----------------------------------------------------------------------
To unsubscribe, change your address, or temporarily deactivate your
subscription,
please go to http://v2.listbox.com/member/?member_id=6959932&id_secret=40809153-6d9252
Powered by Listbox: http://www.listbox.com