Re: apreqXXXXXX temp files remain after processing uploads greater than 256kb. Further large upload fails

On 3/12/07, Joe Schaefer <[email protected]> wrote:
> > Ok, lets presume DeleteFile() is failing, and passing
> > that error along to apr_file_remove().  In apreq_file_cleaup
> > let's then try what cygwin does in its unlink implementation:
> > opening the file again (using the apr API) with the APR_DELONCLOSE
> > flag set, then immediately closing it.
>
> Here's a different suggestion to try:
>
> Index: library/util.c
> ===================================================================
> --- library/util.c      (revision 517052)
> +++ library/util.c      (working copy)
> @@ -816,10 +816,6 @@
>         return rc;
>
>     data = apr_palloc(pool, sizeof *data);
> -    /* cleanups are LIFO, so this one will run just after
> -       the cleanup set by mktemp */
> -    apr_pool_cleanup_register(pool, data,
> -                              apreq_file_cleanup, apreq_file_cleanup);
>
>     /* NO APR_DELONCLOSE! see comment above */
>     flag = APR_CREATE | APR_READ | APR_WRITE | APR_EXCL | APR_BINARY;
> @@ -828,10 +824,11 @@
>      * a grep through the httpd sources seems to indicate
>      * it's only used in sdbm files??
>     */
> -#ifdef WIN32
> -    flag |= APR_FILE_NOCLEANUP | APR_SHARELOCK;
> -#endif
>     rc = apr_file_mktemp(fp, tmpl, flag, pool);
> +    /* cleanups are LIFO, so this one will run just before
> +       the cleanup set by mktemp */
> +    apr_pool_cleanup_register(pool, data,
> +                              apreq_file_cleanup, apreq_file_cleanup);
>
>     if (rc == APR_SUCCESS) {
>         apr_file_name_get(&data->fname, *fp);
>
>
> In other words, delete the file *before* closing it.
>
This shouldn't matter. My version is apreq2-2.0.8 with Apache
2.2.2/2.2.4 build with Visual Studio 2003 running on Windows XP SP2
with all updates.

Here's what I get to see with filemon(from sysinternals) when I run
with my second patch(removing only the #ifdef win32 part):
** first apr_file_mktemp->apr_file_open->CreateFile **
12:33:59.625 AM	httpd.exe:4032	IRP_MJ_CREATE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS	Options: Create
Access: 0012019F

** then after all the reads and writes etc.. finally, apr's
file_cleanup->CloseHandle **
** presense of IRP_MJ_CLEANUP in here indicates that the filehandle
count is zero. **
12:35:29.281 AM	httpd.exe:4032	IRP_MJ_CLEANUP	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS
12:35:29.281 AM	httpd.exe:4032	IRP_MJ_CREATE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS	Options: Open
Access: 00120189
12:35:29.281 AM	httpd.exe:4032	IRP_MJ_SET_INFORMATION
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS	FileBasicInformation
12:35:29.281 AM	System:4032	IRP_MJ_CLEANUP	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS
12:35:29.281 AM	System:4032	IRP_MJ_CLOSE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS
12:35:29.281 AM	httpd.exe:4032	IRP_MJ_CREATE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS	Options: Open
Access: 00000180
12:35:29.281 AM	System:4032	IRP_MJ_CLEANUP	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS
12:35:29.281 AM	System:4032	IRP_MJ_CLOSE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS
12:35:29.281 AM	httpd.exe:4032	IRP_MJ_CLOSE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS


** After that apreq_file_cleanup->apr_file_remove->DeleteFile **
** DeleteFile does this: open the file of the given name, mark it for
deletion, close the file. **
12:38:10.218 AM	httpd.exe:4032	IRP_MJ_CREATE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS	Options: Open
Access: 00010080
12:38:10.218 AM	httpd.exe:4032	IRP_MJ_SET_INFORMATION
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS	Delete
12:38:10.218 AM	httpd.exe:4032	IRP_MJ_CLEANUP	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS
12:38:10.218 AM	httpd.exe:4032	IRP_MJ_CLOSE
	C:\DOCUME~1\pi\LOCALS~1\Temp\apreqq4EH3d	SUCCESS


Notice that interchanging the order of above two cleanup functinons
has no change in the final result. You either close the
file(CloseHandle) and then mark the file for deletion(DeleteFile) or
mark the file for deletion and then close the file, it's all the same.
The file will go if there were no other file handles(say, from inside
perl content handler that ran just before) holding on to it.

I don't know the perl code, but I would assume it would be doing all
it's stuff(open/copy/closing the temp file) before either of the two
cleanup functions are called above.

So, if we apply the patch suggested by Joe or me(second patch) and fix
the perl code which originally triggered all this discussion, all
should be fine.

Thanks,
Vinay Y S