Re: python is blocked by 13500 entries in a bib library

Frederic Gobry <[email protected]> Mon, 12 Dec 2005 14:16:30 +0100
Newsgroups gmane.comp.gnome.apps.pybliographer
Message-ID <[email protected]>
> >From using export functions of Endnote, I have a bib library with more
> than 13000 entries. Opening the library needs some minutes but finally
> the library is open. However, this blocks almost all other activity on
> my computer (debian linux, athlon 2000 ).

Indeed, pyblio with several thousand records is not efficient at all. We
did not optimize it along the lines of memory usage as we hoped to have
a database-backed version much sooner. This work is still under way, but
if you need a visual editor for a database of that size _today_, pyblio is
not the way to go, unfortunately.

As I did not investigate on the memory usage seriously, there might be a
simple solution improving the situation easily (I made some parts of the
parser 10x faster with very few changes at the beginning :-)).

If someone wants to give it a look, I'm interested in the outcome.

> I will be able to send the file, however, this would take some time,
> since am I only connected via dial-in, and I would have to go to some
> place with true ethernet connection. You would also need to have a large
> incoming email directory at your provider's server.

It should be quite manageable once bzipped. If someone wishes to have a
look at it, I'm willing to do the middle man, opening a ftp repository
for you to upload the file.

Frédéric
signature.asc (application/pgp-signature, 189 B)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)

iD8DBQFDnXguFjQHpltE9KURAiDHAJ94qLWAOKioIQpxNyiF0gb0O7NpSACgkmDu
y+4XhonPxfwDuqqth3mWwHI=
=joFX
-----END PGP SIGNATURE-----