Re: Regarding my old mmap patch

> At 07:07 PM 9/5/2006 -0700, Cory Nelson wrote:
>>The performance boost given by mmapping will be there regardless of file 
>>size.
>
> OK, but this raises the larger question of how does tidy fit into your
> processing flow?  Is it a bottleneck (or even a runner up)?  Are live
> people waiting on the response for large files?  I guess I continue to be
> surprised that Tidy is anyone's performance bottleneck.  I know certain
> bulk web scraping projects (thinking of Mark Weaver) might  have this
> issue, but these are few and far between (or so I thought).
>
> I'm not trying to be a pain.  I just want to justify the complexity.

sure :) Tidy is also a bottleneck on a few applications I have and currently 
I don't ask tidy to parse files, but I rather read them first to memory 
(using mmap()) and then ask tidy to parse them from the memory. I found this 
to be faster.
Using mmap() also enables the OS to share the files between processes. And 
yes, the windows and linux versions of mmap'ing are pretty much the same. 
The code is pretty simple.


Nuno

P.S.: I noticed my mmap() patch misses a fclose(). I'll update the patch if 
you consider including it in the sources. 


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642