RE: Practicality of using DB_File on a Perl-based book site?

Harrison,

That's useful information. Yes, I'll only be doing lookups, which simplifies
things quite a bit. Given what you said about the Movable Type software, I
assume DB_FILE would be a good way to keep track of website user names,
passwords, cookies, and the like?

- Chris 

-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf
Of Harrison Dekker
Sent: Friday, June 20, 2008 2:19 PM
To: Christopher Morgan
Subject: Re: Practicality of using DB_File on a Perl-based book site?

Chris,

I'm no expert, but it seems to me, that there should be less overhead using
Berkeley DB compared to a relational DB, assuming that all you're doing is
lookups. If you've got a bunch of post processing going on involving
multiple large retrieval sets then you'll probably lose that edge, but
that's only because your perl code would be doing the work that a more
optimized SQL engine could be doing. SQL doesn't give you any improvement,
however, when all you're doing is a key/value type lookup.

Movable Type blog software uses BDB, at least it did in the past, and as far
as I know it's quite reliable/scalable. I use BDB for one web servicey type
application and I do have to throttle my requests if I'm sending them in
batch, but the db isn't the bottleneck, it's apache or the php xml functions
I use.

-Harrison

On Fri, Jun 20, 2008 at 10:42 AM, Christopher Morgan <[email protected]> wrote:
>
> I'm designing a web site that will display MARC authority files 
> onscreen. I use a Perl hash that's tied to a (read-only) Berkeley 
> DB_file, and it works nicely. How practical is this approach if 
> there's going to be moderate traffic on a site?
>
> My DB_FILE is about 200MB, but of course Perl brings only small pieces 
> of the database into memory at any one time. Would the site bog down 
> if people were accessing records at the rate of, say, every few 
> seconds? Should I consider mySQL instead? I'd prefer to stick to 
> DB_FILE, since it's so easy and elegant -- and I can easily create complex
data structures.
>
> What if one of my data files was significantly bigger (say, a GB or 
> two of MARC book records)? I don't have a feel for the pros and cons 
> of the various approaches to accessing large databases using Perl, but 
> tied hashes are pretty fast! In any case, I know I'll have to lock the 
> file during each read, via "flock" or the like. I haven't tried
implementing the latter yet.
>
> Does anyone have any ideas about this? Are there other Perl forums I 
> should investigate regarding this topic?
>
> Many thanks!
>
> - Chris Morgan
>
>



--
Harrison Dekker -- Coordinator of Data Services -- UC Berkeley Libraries
510-642-8095 :: GTalk:vagrantscholar :: AIM:hdekker :: Meebo:ucbdekker
http://sunsite.berkeley.edu/wikis/datalab/
------------------------
Q: Why is this email 5 sentences or less?
A: http://five.sentenc.es