CLSQL:MAP-QUERY more memory hungry than CLSQL:QUERY
Jason H <[email protected]> Thu, 8 Jul 2010 22:30:52 -0700
| Newsgroups | gmane.lisp.clsql.general |
|---|---|
| Message-ID | <[email protected]> |
Hi,
I'm using the ODBC back-end of clsql-5.0.2 to load data from an MDB file.
The data consists of 1839364 rows and each row has 7 integers. At first,
I used CLSQL:QUERY to get a list of 1839364 7-element tuples and processed
this long list into something else. This worked but generated lots of garbage
because of the transient long list returned by CLSQL:QUERY. To reduce the
amount of garbage, I switched to CLSQL:MAP-QUERY but was quite surprised
to find out CLSQL:MAP-QUERY used even more memory to do the same thing.
To make what I just described more concrete for this email, I created
the following 2 test functions:
(defun test-1 ()
(clsql:map-query nil
#'identity
(clsql:query "select c1, c2, c3, c4, c5, c6, c7 from Cells")))
(defun test-2 ()
(clsql:query "select c1, c2, c3, c4, c5, c6, c7 from Cells") )
The 2 test functions were run in the following way:
(progn
(test-1)
(null (test-2)))
while being profiled.
This is the profile report:
seconds | consed | calls | sec/call | name
---------------------------------------------------------
52.395 | 147,196,856 | 1 | 52.395000 | TEST-2
44.894 | 161,911,208 | 1 | 44.894000 | TEST-1
---------------------------------------------------------
97.289 | 309,108,064 | 2 | | Total
estimated total profiling overhead: .000 seconds
overhead estimation parameters:
4.e-8s/call, 6.028e-6s total profiling, 1.48e-6s internal profiling
This result is contrary to what I expected based on what I saw in
the source code of CLSQL-SYS::MAP-QUERY-FOR-EFFECT: the same ROW was
passed to CLSQL-SYS::DATABASE-STORE-NEXT-ROW repeatedly. I, a CLSQL idiot,
must be missing something big time. Could somebody enlighten me?
Best wishes,
Jason
ps. i'm using win32 version of sbcl 1.0.37.12.