Re: using built-in db

Maksim Lin for technical support mailling lists wrote:
> I think the limits would be more determined by the underlying OS
> filesystem.

I've gotten only vague and conflicting information about this for 
Linux--ext3 in my case (where I'll be running the production version). 
Nothing like just trying it, I suppose.

> created 1000 books in 0.06s scanned 1000 books in 0.05s
> created 5000 books in 0.741s scanned 5000 books in 0.14s
> created 10000 books in 7.231s scanned 10000 books in 0.26s

That's pretty wild how create time gets a bit pathological by 10k, but 
read time is better than linear. This does give me some nice ballpark 
numbers which may be completely non-applicable to my situation but will 
help me feel better anyway :-)

If that's the only obstacle to using the XML DB on large datasets, 
though, I would think it would be easy enough to fix in Helma--just have 
a tiny bit of code to split predictably among n directories.

> BTW Not sure if this is documented anywhere, but one nice feature of
> using the xml data store is you can just delete an apps whole folder
> inside the db folder, restart the app (with the manage app webconsole)
> and hema will recreate a new lcena db folder for the app!

I've done this once already--nice indeed. What does "lcena" mean? 
(Google results suggest it was a typo, but I'm not trying to be 
obtuse--I really don't know what you meant.)

> Also be aware that if you go use the xml-based datastore, you won't be
> able to use the more advanced features of the data mapping layer like
> filters and groupby sorting for collections.

Yeah, I think this is ok for my purposes. I think I'll just need some 
extra grouping objects to take care of my needs here, and I think the 
flexibility of the XML db is still a win.

> res.abort() after adding the objects to the collection, they were left
> in the in-memory cache but not persisted to disk 

Thanks for the heads-up! Shouldn't be much trouble to work around since 
I know it's there.

-Joshua