Re: Filling a hash view

Marcin Krol wrote:

> I'm "testdriving" metakit 4 py now, and I have a little trouble with hash=
ed view: when the number of keys in view starts exceeding 2000, filling up =
a hash view becomes extremely slow.
>=20
> Is there some way to fill a view that is hashed to begin with, so the db =
knows its way around db and adding rows becomes faster?

This is the normal way to open a hash view:

	db =3D mk.storage("demo2.db",1)
	vw =3D db.getas('words[idxs1:S,idxs2:S,idxi:I,val:S]')
	map =3D db.getas('map[_H:I,_R:I]')
	vwh =3D vw.hash(map)

After this point you must make all changes to vwh to keep everything in syn=
c. Changes to the (non-persistent / virtual) vwh view will internally get t=
ranslated to the proper changes to the persistent vw and map view.

To fill it more quickly:
- make changes directly to the underlying "vw", ignoring map and vwh
- empty the map (that's what "del map[:]" does)
- reconstruct a new vwh with "vwh =3D vw.hash(map)"

This means you reconstruct the hash after filling the underlying view, inst=
ead up updating it with each change. It's similar to reconstructing vs. mai=
ntaining an index in a relational database.

> And why vw.hash(map) has to be repeated n times? Sadly, the docs do not e=
xplain what's this all about.

 t0 =3D time();
 for i in xrange(n):
   del map[:]
   vwh =3D vw.hash(map)
 print ' hash_ini %d times, size %d: %g sec' % (n, len(map), time() - t0)

That's a benchmark, it reports the number of seconds needed to perform n ha=
sh constructions.

-jcw

PS. Pat's comment about "blocked" views is indeed another optimization to c=
onsider. Blocked and hashed views were designed to work together in all thr=
ee combinations (vw blocked, map blocked, or both blocked). The optimal cho=
ice depends greatly on the application usage patterns.

--=20
You received this message because you are subscribed to the "metakit" group=
.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to metakit-unsubscribe@googlegro=
ups.com
For more options, visit this group at http://groups.google.com/group/metaki=
t?hl=3Den