Re: var too large
Bill Yerazunis <[email protected]>
| Newsgroups | gmane.mail.spam.crm114 |
|---|---|
| Message-ID | <[email protected]> |
From: Thomas Michael Hagen <[email protected]> my program goes through large files. some of them are almost 300Mb. i fix this by setting -w 300000000 on the command line when i call my crm program. however, when i run the program on many files, they somehow accumulate in the program's memory, and after a while exceed the limit. how do i clean the files back out of memory after i have used them? Set the variable that contains their string to empty as in: isolate (:my_big:string:) // that will release and auto-pack the storage. However, realize that the auto-packer might spend some time shuffling things around. The test code in megatest.sh which exercises that memory reclamation (and shuffles something like 80 megabytes a hundred times) spends on the order of ten or twenty seconds doing it. what happens in my program is that i go through several folders, with many files in them: i define an =FCberfolder, and do a syscall ls on that, then i extract the subfoldernames from that list with a match, and syscall ls on each one, then extract the filenames and run the classification program on them Ahhh... you might need to discard the file's texts after you're done using them. That's what the isolate above will do. Or- get on a 64-bit machine and give it a gig or two. :) - Bill Yerazunis ------------------------------------------------------------------------------ Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H