Re: File-System optimization

Bill Moseley scribbled on 10/18/06 1:57 PM:
> On Wed, Oct 18, 2006 at 11:03:23AM -0700, brad miele wrote:
>> hi,
>>
>> we currently use fs to index our stuff, this is because we are already 
>> dumping xml for every record in our database.
>>
>> my question is, is it faster to break the directory up into sub 
>> directories? right now, we have one directory called IDX that has all 
>> 900K+ files in it. it takes a very long time for swish-e to move from the 
>> "Checking dir" phase to the actual indexing phase. "a very long time" is 
>> not really quantifiable right now since we generally don't see it 
>> happening and i am just noticing because i am running things manually 
>> today. it has been sitting at this stage for about 1.5 hours so far.
>>
>> so should i try breaking the directory up into sub directories?
> 
> Depends on your file system.  But I'd probably break it up into
> smaller directories.
> 

besides, must be next to impossible to 'ls' in that dir. ;)

check out SWISH::Prog::DBI on cpan too. That lets you index directly from db via 
DBI without the intermediate XML files.

-- 
Peter Karman  .  http://peknet.com/  .  [email protected]