Re: Adding some extra switches
Jason Rennie <[email protected]> Mon, 18 Nov 2002 18:46:36 -0500
| Newsgroups | gmane.mail.ifile.devel |
|---|---|
| Message-ID | <[email protected]> |
[email protected] said: > However, ifile doesn't seem to like being given large numbers of files > (tens of thousands) to process in one go. The kernel eventually killed > it after trashing VM for a while. I didn't have time to dig deeply > into why. I'll have to come back to it when I have more time. Jeremy Brown ripped out some of the bloat that made ifile not handle lots of messages very well in a patch for 1.1.0, but it looks like there may still be a leak somewhere (I regularly do runs w/ up to 2000 messages and see memory usage increase during the run---which shouldn't happen if ifile is sequentially processing the messages and freeing memory as it goes). One question---how do your patches get around this problem? Seems like ifile would suck up all the memory whether you pass files via the command-line or via stdin... Sorry for all the pestering---just don't want to be adding cmd line options too freely :) Jason D. M. Rennie MIT AI Lab [email protected] (617) 253-5339 http://www.ai.mit.edu/~jrennie/