Re: Conduit missing closing file handles

On Fri, May 01, 2026 at 09:56:29PM +0200, Henning Thielemann wrote:
> On Fri, 1 May 2026, Tom Ellis wrote:
> > But getEntrySource was written with the wrong primitives, blocking
> > prompt finalization, so I had to rewrite that.
> 
> Instead of writing and maintaining two new libraries for ZIP and TAR
> handling, would it work to submit improved getEntrySource based on Conduit
> to the 'zip' package project and use Bluefin only in zip2tar? Or do I still
> need Bluefin at all, once getEntrySource is updated?

I'll summarize what I know:

* getEntrySource[1] is buggy because its return Conduit doesn't release
  the file handle it uses internally when consumption is finished.

* The bug can be traced back to `sourceEntry`[2]

* and then to `sourceIOHandle`[3] in combination with `isolate`[4].
  The bug manifests in the Handle created by `sourceIOHandle` not
  being released when the result of `isolate` has been fully consumed.
  That's because `isolate` causes the Conduit returned from
  `sourceIOHandle` not to be fully consumed.  This was demonstrated in
  my original post to this thread.

* The definition of `sourceIOHandle` is

      sourceIOHandle alloc = bracketP alloc IO.hClose sourceHandle

  so it looks like it is upholding its responsibility to manage
  resources.  `isolate` is about as innocuous as can be.
  `sourceEntry` uses `sourceIOHandle ... .| isolate ...` so it
  is also innocuous, modulo the correctness of its components.

* Therefore I hypothesise that this is an unsolvable design flaw of
  Conduit.

So my guess is you can't achieve what you want with conduit. Instead
you could try to reuse some conduit pieces in the context of a safe
setup, such as Bluefin.

A simpler challenge for conduit is whether there can be a conduit that
yields the first n lines of each file it is given as input. I've asked
that on Discourse. Let's see if it can be done.

https://discourse.haskell.org/t/conduit-to-yield-first-lines-from-file-without-leaking-space-or-file-handles/14030

Tom




[1] https://hackage-content.haskell.org/package/zip-2.2.1/docs/src/Codec.Archive.Zip.html#getEntrySource

[2] https://hackage-content.haskell.org/package/zip-2.2.1/docs/src/Codec.Archive.Zip.Internal.html#sourceEntry

[3] https://hackage-content.haskell.org/package/conduit-1.3.6.1/docs/src/Data.Conduit.Combinators.html#sourceIOHandle

[4] https://hackage.haskell.org/package/conduit-extra-1.3.6/docs/src/Data.Conduit.Binary.html#isolate

[5] https://hackage-content.haskell.org/package/conduit-1.3.6.1/docs/Data-Conduit.html#v:.-124-
_______________________________________________
Haskell-Cafe mailing list -- [email protected]
To (un)subscribe, modify options or view archives go to:
Only members subscribed via the mailman list are allowed to post.