Python bindings / Hello from new Gentoo maintainer
Chris Kerr <[email protected]> Thu, 23 May 2013 23:46:01 +0100
| Newsgroups | gmane.comp.db.mdb-tools.devel |
|---|---|
| Message-ID | <[email protected]> |
--===============7281805852406816389==
Content-Type: multipart/signed; boundary="nextPart12687631.br5vs60k7O";
micalg="pgp-sha1"; protocol="application/pgp-signature"
--nextPart12687631.br5vs60k7O
Content-Type: multipart/alternative; boundary="nextPart3163655.Yf1LuhdSyi"
Content-Transfer-Encoding: 7Bit
This is a multi-part message in MIME format.
--nextPart3163655.Yf1LuhdSyi
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
Hi there,
I'm a PhD student studying battery chemistry at the University of
Cambridge, and I use mdbtools to extract data from the JetDB-based log
format produced by Arbin Instruments battery test stations. (As is often
the case with research hardware, the closed-source vendor-provided tool
is buggy and runs only on Windows.)
Since there is currently no maintainer for mdbtools on the Gentoo Linux
distribution, I wrote my own ebuild for mdbtools-0.7 and have volunteered
to be the mdbtools maintainer from now on. Hopefully mdbtools-0.7 should
be in the main Portage tree in the next few days. I've signed up to this
mailing list and am following mdbtools on GitHub - please let me know if
there is any other source of info I should be following.
I do almost all my data processing in Python; currently to extract the data I
run mdb-export via subprocess.Popen and pass the INSERT statements
directly into a temporary sqlite3 database, then query the sqlite3
database for the data I need. Recently I've been having a quick look at the
source code and I think I could knock together a Python binding that would
support the very limited subset of operations I need relatively quickly.
Searching online for Python MDB bindings seems only to give bindings for
the Windows Jet DB API, but I thought I'd ask if anyone knows of anything
I've missed, or has any suggestions on where to start. (I've only ever used
F2Py to make bindings to Fortran code before, but I don't think it's the
right solution for libmdb because of the heavy use of pointers.)
The format of the databases I'm using is basically a dozen or so relatively
complicated tables with only a few rows each holding metadata and so on,
plus one table holding the actual data (all columns are integers or floats).
What I want to be able to do is read out the metadata tables into
something similar to the sqlite3.Row object, and read the data table into a
NumPy array.
Thanks for all the work you've done developing this code; it's been very
helpful!
Chris
--nextPart3163655.Yf1LuhdSyi
Content-Transfer-Encoding: 7Bit
Content-Type: text/html; charset="us-ascii"
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0//EN" "http://www.w3.org/TR/REC-html40/strict.dtd">
<html><head><meta name="qrichtext" content="1" /><style type="text/css">
p, li { white-space: pre-wrap; }
</style></head><body style=" font-family:'Sans Serif'; font-size:10pt; font-weight:400; font-style:normal;">
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Hi there,</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I'm a PhD student studying battery chemistry at the University of Cambridge, and I use mdbtools to extract data from the JetDB-based log format produced by Arbin Instruments battery test stations. (As is often the case with research hardware, the closed-source vendor-provided tool is buggy and runs only on Windows.)</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Since there is currently no maintainer for mdbtools on the Gentoo Linux distribution, I wrote my own ebuild for mdbtools-0.7 and have volunteered to be the mdbtools maintainer from now on. Hopefully mdbtools-0.7 should be in the main Portage tree in the next few days. I've signed up to this mailing list and am following mdbtools on GitHub - please let me know if there is any other source of info I should be following.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">I do almost all my data processing in Python; currently to extract the data I run mdb-export via subprocess.Popen and pass the INSERT statements directly into a temporary sqlite3 database, then query the sqlite3 database for the data I need. Recently I've been having a quick look at the source code and I think I could knock together a Python binding that would support the very limited subset of operations I need relatively quickly. </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Searching online for Python MDB bindings seems only to give bindings for the Windows Jet DB API, but I thought I'd ask if anyone knows of anything I've missed, or has any suggestions on where to start. (I've only ever used F2Py to make bindings to Fortran code before, but I don't think it's the right solution for libmdb because of the heavy use of pointers.)</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">The format of the databases I'm using is basically a dozen or so relatively complicated tables with only a few rows each holding metadata and so on, plus one table holding the actual data (all columns are integers or floats). What I want to be able to do is read out the metadata tables into something similar to the sqlite3.Row object, and read the data table into a NumPy array.</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Thanks for all the work you've done developing this code; it's been very helpful!</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p>
<p style=" margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; -qt-user-state:0;">Chris</p>
<p style="-qt-paragraph-type:empty; margin-top:0px; margin-bottom:0px; margin-left:0px; margin-right:0px; -qt-block-indent:0; text-indent:0px; "> </p></body></html>
--nextPart3163655.Yf1LuhdSyi--
--nextPart12687631.br5vs60k7O
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part.
Content-Transfer-Encoding: 7Bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)
iQEcBAABAgAGBQJRnpwyAAoJEBUILloQf8Sg9acH/3+xrrUGODiuz1DsOmdLk+P2
hC8N2o2xEXXW9evK+e06/AAel9+0rlX3KU3p+WTLG9YYl2KR/hj/d37fDN2cF6T4
3kaZEFb4Hvl6Lm8mDEOezgySLlzNR4Rv16KeA74TKkA2dLDCXKK0uUrEeuhy6IDM
1ALcd9QFDuSUywyycNaMwoc4vyxBlm7LMJrtOv2csMXMwPUPD4vkpNqzkEEsuVSa
EXzfE5nW5ATipa/McaWTjpT+ki4NkkGkU7LnFPIEOPrpeT1ElKoet06B85NLGaA+
5GQWVzZpRQIjwmsRUK54Acmf+Au8hH7v9D95qkn01EKlj5gZqkcwOBG7PAZjkeo=
=NAFP
-----END PGP SIGNATURE-----
--nextPart12687631.br5vs60k7O--
--===============7281805852406816389==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
------------------------------------------------------------------------------
Try New Relic Now & We'll Send You this Cool Shirt
New Relic is the only SaaS-based application performance monitoring service
that delivers powerful full stack analytics. Optimize and monitor your
browser, app, & servers with just a few lines of code. Try New Relic
and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_may
--===============7281805852406816389==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
mdbtools-dev mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/mdbtools-dev
--===============7281805852406816389==--