Reading multiple files fails

"Henk-Jan.van.Tuyl" <[email protected]>
Newsgroups gmane.comp.lang.haskell.hugs.bugs
Message-ID <oprwsuema4vhqsjc@localhost>
L.S.,

After reading through 60 or 61 files (about 220 kB), Hugs fails with the message:
     (1426170 reductions, 2976893 cells, 68 garbage collections)
     ERROR - Garbage collection fails to reclaim sufficient space

This is done with the program below (ReadALot.lhs), which reads a series of files (these files are listed in dir.dat).
The program reads the first and the last (non-empty) line of each file and writes them, together with the filename in
the output file result.dat.

The program works fine when compiled with GHC, it then reads 2946 files without a problem.

More info.:
System:       Windows 98
Hugs version: Nov 2002

Best regards,
Henk-Jan van Tuyl


======================================================================
     ReadALot.lhs
     Read a lot of files

     2003-10-06

     History:
     031009  Added dummy parameter to "main"
======================================================================

import IO
import Monad

inputSet   :: String
inputSet   = "dir.dat"

outputFile :: String
outputFile = "result.dat"

======================================================================
main
======================================================================

main :: a -> IO ()
main _ =   do     hInputSet   <- openFile inputSet   ReadMode
      hOutputFile <- openFile outputFile WriteMode
      fileNames   <- hGetContents hInputSet
      allData     <- collectAllData $ lines fileNames     hPutStrLn hOutputFile $ unlines allData       hClose hInputSet
      hClose hOutputFile

======================================================================
collectAllData
Read a list of file names and return file name, first line and last
line of each file in a list
======================================================================

collectAllData :: [String] ->      -- A list of filenames
                    IO [String]      -- A list of results

collectAllData []                     = return []
collectAllData (fileName : fileNames) =   do
      putStrLn $ "Reading: " ++ fileName
      h <- openFile fileName ReadMode

      contents <- hGetContents h

      dataFromFile <-       return           $!         parseFile      $         filter (/= "") $         lines
contents     let (firstLine, lastLine) = dataFromFile

        To force reading of the file before closing, we add a "when" (from
        the Monad library), with a boolean expression that uses lastLine
        and will always be true:

      when (length lastLine >= 0) $
        hClose h

      rest  <- collectAllData fileNames

      return $ fileName : firstLine : lastLine : "" : rest


======================================================================
parseFile
Returns the first and the last string of a list of strings, as a tuple
======================================================================

parseFile :: [String] -> (String, String)
parseFile []       = ("", "")
parseFile [x]      = (x,  "")
parseFile (x : xs) = (x,  last xs)



--
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.