Handling Large Text Files

Steve Chernyak <[email protected]>
Newsgroups gmane.comp.encryption.bouncy-castle.devel
Message-ID <[email protected]>
Hello

I need to handle a large text file and I'm wondering if it's possible to do it without reading the entire thing into memory and also without writing the entire thing out to disk in clear text.

Encrypted, the file is only 4 megs. However, once decrypted it balloons out to a few hundred megs. The file consists of rows of data.

Is it possible to decrypt the file in relatively small chunks and look for the row delimiter character. Once the character is found, process the row and discard the data. Continue to do this until the entire file is processed. Sort of like processing SQL query results with a row handler. 

I see examples have code for dealing with binary data using a byte array as a buffer. Is it safe to convert the buffer to a string or is it possible the buffer will end mid character?

Is there any documentation/examples available for doing what I want? 

Thanks
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.