Re: segfault in mailmbox_append_message_list_no_lock / mmap full mbox

On Sun, Mar 05, 2006 at 01:02:49PM +0100, DINH Viet Hoa wrote:
> Florian Lohoff wrote :
> 
> > Hi,
> > i am getting segfaults in mailmbox_append_message_list_no_lock with a
> > simple application of mine copying folder contents from maildir to an
> > mbox. This does happen irregularly so i compiled libetpan with debugging
> > symbols and let it run. Over the day it copied 66k mails to the mbox 
> > before crashing.
> 
> can you show your code ?

     29 struct mailfolder *folder_open(char *mfolder) {
     30         struct mailstorage      *storage;
     31         struct mailfolder       *folder;
     32
     33         assert(storage=mailstorage_new(NULL));
     34
     35         if (mfolder[strlen(mfolder)-1] == '/')
     36                 assert(maildir_mailstorage_init(storage, mfolder, 0, NULL, NULL) == 0);
     37         else
     38                 assert(mbox_mailstorage_init(storage, mfolder, 0, NULL, NULL) == 0);
     39
     40         assert(folder=mailfolder_new(storage, mfolder, NULL));
     41
     42         assert(mailfolder_connect(folder) == 0);
     43
     44         return(folder);
     45 }

     64         inf=folder_open(inbox);
     65         outf=folder_open(outbox);
     66
     67         if (hdrfile) {
     68                 if (hdrfile[0] == '-')
     69                         hdrfd=STDOUT_FILENO;
     70                 else {
     71                         hdrfd=open(hdrfile, O_RDWR|O_APPEND|O_CREAT, 0644);
     72                         if (hdrfd < 0) {
     73                                 fprintf(stderr, "Unable to open %s as hdrfile\n", hdrfile)        ;
     74                                 return 0;
     75                         }
     76                 }
     77         }
     78 
     79         assert(mailsession_get_messages_list(inf->fld_session, &env_list) == 0);
     80
     81         for(i=0;i<carray_count(env_list->msg_tab);i++) {
     82                 mailmessage *msg;
     83                 msg=carray_get(env_list->msg_tab, i);
     84
     85                 mailmessage_get_flags(msg, &flags);
     86
     87                 if (flags && flags->fl_flags & MAIL_FLAG_DELETED)
     88                         continue;
     89
     90                 if (mailmessage_fetch(msg, &msgbuf, &msglen) != MAIL_NO_ERROR) {
     91                         fprintf(stderr, "Failed to fetch message %d from %s\n", i, outbox)        ;
     92                         break;
     93                 }
     94

This one crashes - As i showed on the gdb backtrace it tries to write to memory
regions which it did not mmaped successfully.

     95                 r=mailfolder_append_message_flags(outf, msgbuf, msglen, msg->msg_flags);
     96                 mailmessage_fetch_result_free(msg, msgbuf);
     97
     98                 if (r != MAIL_NO_ERROR) {
     99                         fprintf(stderr, "Failed to append message %d to %s error %d\n", i,         outbox, r);
    100                         break;
    101                 }
    102
    103                 if (delete) {
    104                         flags->fl_flags |= MAIL_FLAG_DELETED;
    105                         mailmessage_check(msg);
    106                 }
    108                 if (hdrfile) {
    109                         char    xsize[64];
    110                         if (mailmessage_fetch_header(msg, &hdrbuf, &hdrlen) != MAIL_NO_ERR        OR) {
    111                                 fprintf(stderr, "Failed to fetch message hdr - %d %s\n", i        , inbox);
    112                                 break;
    113                         }
    114                         i=sprintf(xsize, "X-MailSize: %lu\n", msglen);
    115                         r=write(hdrfd, &xsize, i);
    116                         if (r != i) {
    117                                 fprintf(stderr, "Error while writing %d bytes to hdrfile\n        ", i+1);
    118                                 break;
    119                         }
    120                         r=write(hdrfd, hdrbuf, hdrlen);
    121                         mailmessage_fetch_result_free(msg, hdrbuf);
    122                         if (r != hdrlen) {
    123                                 fprintf(stderr, "Error while writing %d bytes to hdrfile\n        ", hdrlen);
    124                                 break;
    125                         }
    126                 }
    127
    128
    129                 mailmessage_flush(msg);
    130         }
    131
    132         mailmessage_list_free(env_list);

> > It seems the mbox mmap does not succeed but still continues. Anyway - I
> > guess so far libetpan is the wrong library for me because i am expecting
> > mailfolder much larger than 2GB (for archive reasons) and i see the
> > mailmbox_map mmaps the full file which wont work on 32bit archs anyway.
> 
> If you have some code to read a mbox, a driver could be written to avoid 
> this limitation.

I guess thats needed ...

Flo
-- 
Florian Lohoff                  [email protected]             +49-171-2280134
                        Heisenberg may have been here.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFEFUM/Uaz2rXW+gJcRAnKmAJ9JDJwuhNEbnPS17Rvfaj+Z0lmHQgCgp2xt
qQIdBNZRuHKx05jHxgvlohQ=
=uwZw
-----END PGP SIGNATURE-----