Re: Feature freeze imminent

On Tue, Feb 27, 2007 at 04:44:52PM +0000, Tuomo Valkonen wrote:
> On 2007-02-27, Sam Mason <[email protected]> wrote:
> > There are some very nice high-level libraries available[1] for Haskell
> > that abstract away even writing the SQL.  Of these I've only played with
> > HaskellDB and was reasonably impressed, but I think a lot has changed
> > since then and it may be even better.
> 
> I actually looked into it today, but: HaskellDB doesn't not seem to
> support creating new tables on the fly, as it demands a static
> definition of the database. And I think the way to implement this,
> would be to have a table for each tag, listing the objects having
> tha tag. Is INTERSECT how efficient usually, btw.?

Tables are never normally created by code.  That would be akin to Python
code generating a source file with a slightly different class definition
in it and telling the interpreter to load it.  You can do it, but it's
not recommended!  I would probably have a schema more like:

  files:
    fileid  INTEGER  NOT NULL  PRIMARY KEY

  tags:
    tag         TEXT  NOT NULL  PRIMARY KEY,
    description TEXT

  filetags:
    tag    TEXT     NOT NULL  REFERENCES tags,
    fileid INTEGER  NOT NULL  REFERENCES files ON DELETE CASCADE,
      PRIMARY KEY (tag, fileid)

In more human terms I would have a table of files, the purpose of which
is to give you the fileid (sort of like the inode number), a table of
tags that define what tags are known, and a table containing the pairs
(tag,fileid) that define what files have what tags.

I would rely on the database to optimise things where necessary---big
databases are fairly amazing at this, Postgres has a good planner and
MySql doesn't.  Tables should have few columns (I tend to think of them
as generalisations of functions) and lots of rows.  I'm working with
a dataset (under Postgres) that has several tables with >100million
rows and everything is fine, rows are not a problem.

If you've ever done any logic programming (Mercury[1] is nice if you're
used to Haskell) then relations (tables) are similar to predicates in
that functions don't just have to return one value.  A bit like lifting
things into Maybe or List Monads in Haskell, except even cleaner than
Haskell.  You know up front if they're going to return either one result
or fail (zero results), or return many (zero or more) results.

Hope that makes sense!

> > I don't think it's implemented anywhere at the moment, but have you
> > seen the ICFP2005 paper by Iavor Diatchi et. al.[2]?  
> 
> Nope. It would be nice to have something that provides nice and simple
> bit-level access, though... Or just some low-level language that isn't
> as painful as C.

Yes it would!


  Sam

 [1] http://www.cs.mu.oz.au/research/mercury/