Re: Streaming MIME parser
suresh! <suresh_lakshmanan-/[email protected]>
| Newsgroups | gmane.mail.libetpan.user |
|---|---|
| Message-ID | <[email protected]> |
I'm writing a MIME parser that reads the incoming mail and translates it into the schema of my document store. I was thinking of two ways of parsing. Firstly, you can parse on the fly and push it into the document store. I do not want to do this because of the uncertainity in network connection and having to store the message in memory. Alternatively, I can store the entire message on the disk and then feed the parser. RFC headers does not pose a problem of being in memory. But the body and attachments do. If the MIME parser gives me the position of the start and end of the body and attachments then I can directly goto that position in the file and start reading instead of having to store them in memory. (Ofcourse I would need more events from the MIME Parser as to whether it is an attachment beginning or a multipart beginning etc) Regards, L.Suresh --- Gaël Roualland <[email protected]> wrote: > "suresh!" a écrit : > > > > I don't think you can do that in the current > implementation since it is > based in having the message being parsed in memory. > However, it is using > mmap on temporary files when possible (the so called > mmap_strings), so > it is not really using memory. There's a threshold > on the size of > strings over which it will use a mmaped temporary > file. If you're short > on memory, perhaps you could make sure the threshold > is really low ? > > I can see the need to be able to parse a stream, > however sooner or later > you will probably need the parts you parsed, so > having them handy is > usefull anyway... What are you trying to do ? > > Gaël. > > -- > Gaël Roualland -+- [email protected] > __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click