Handling Large Text Files
Steve Chernyak <[email protected]>
| Newsgroups | gmane.comp.encryption.bouncy-castle.devel |
|---|---|
| Message-ID | <[email protected]> |
Hello I need to handle a large text file and I'm wondering if it's possible to do it without reading the entire thing into memory and also without writing the entire thing out to disk in clear text. Encrypted, the file is only 4 megs. However, once decrypted it balloons out to a few hundred megs. The file consists of rows of data. Is it possible to decrypt the file in relatively small chunks and look for the row delimiter character. Once the character is found, process the row and discard the data. Continue to do this until the entire file is processed. Sort of like processing SQL query results with a row handler. I see examples have code for dealing with binary data using a byte array as a buffer. Is it safe to convert the buffer to a string or is it possible the buffer will end mid character? Is there any documentation/examples available for doing what I want? Thanks