[spr27863] ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT

dispatched to handler
No acknowledgement sent

@@id: spr27863
@@reference: 
@@summary: ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT
@@product: acl
@@version: 
@@component: allegroserve
@@customer-name: Andrew Philpot
@@customer-company: isi
@@customer-phone: 
@@customer-email-address: [email protected]
@@email-cc: [email protected], opensource
@@hardware: 
@@operating-system-and-version: 
@@implementation-time: 
@@state: logged 7/29/2003 by handler
@@state: dispatched 7/29/2003 to handler by handler
@@endstate
-------------------------------------------------------------------------------
@@resolution: 

-------------------------------------------------------------------------------
@@discussion:

NOTE:  This is an old question sent to opensource which jkf has long
       since answered.

       I'm creating the spr with John's answer just for the record.

lois/handler

Return-Path: owner-opensource
Delivery-Date: Tue Jul 15 10:16:51 2003
Received: from turtlemail.franz.com (turtle [192.132.95.23])
	by quadra.franz.com (8.11.6/8.9.0) with ESMTP id h6FHGo728563
	for <handler@quadra>; Tue, 15 Jul 2003 10:16:50 -0700
Received: (from majordomo@localhost)
	by turtlemail.franz.com (8.11.6/8.9.0) id h6FHFiM08592
	for opensource-list; Tue, 15 Jul 2003 10:15:44 -0700
X-Authentication-Warning: turtle.franz.com: majordomo set sender to [email protected] using -f
Received: from spider.franz.com (spider [10.1.0.2])
	by turtlemail.franz.com (8.11.6/8.9.0) with ESMTP id h6FHFhP08589
	for <[email protected]>; Tue, 15 Jul 2003 10:15:43 -0700
Received: (from root@localhost)
	by spider.franz.com (8.11.6/8.11.6) id h6FHFhZ14186
	for [email protected]; Tue, 15 Jul 2003 10:15:43 -0700
Received: from nitro.isi.edu (nitro.isi.edu [128.9.208.207])
	by spider.franz.com (8.11.6/8.11.6) with ESMTP id h6FHFgB14161
	for <[email protected]>; Tue, 15 Jul 2003 10:15:42 -0700
X-Fi-Border: spider
Received: from altamira.isi.edu (altamira.isi.edu [128.9.208.165])
	by nitro.isi.edu (8.11.6p2/8.11.2) with ESMTP id h6FHFbd05062;
	Tue, 15 Jul 2003 10:15:37 -0700 (PDT)
Received: (from philpot@localhost)
	by altamira.isi.edu (8.11.6/8.8.6) id h6FHFi525856;
	Tue, 15 Jul 2003 10:15:44 -0700
Date: Tue, 15 Jul 2003 10:15:44 -0700
Message-Id: <[email protected]>
From: Andrew Philpot <[email protected]>
To: opensource
CC: [email protected]
Subject: ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT
X-scanner: scanned by Inflex 1.0.10 - (http://pldaniels.com/inflex/)
Sender: owner-opensource
Precedence: bulk


As well as the server functionality of ASERVE, I also find that
NET.ASERVE.CLIENT::DO-HTTP-REQUEST, i.e., the client side, is very
useful to me.

I have run into a problem and am not sure how to proceed.

Since DO-HTTP-REQUEST returns a string containing the contents, I can
run afoul of ARRAY-TOTAL-SIZE-LIMIT.  This limits me to documents no
larger that 16MB (in my ACL 6.2 Linux).  I'm now dealing with database
extracts obtained via HTTP in XML format, and they can easily exceed
this limit.

My first inclination is to (locally) patch this routine to return a
list (or vector) of strings each adhering to the limit.  It seems
pretty ugly to me.

Has anyone encountered this problem with ASERVE, or a similar problem
in another set of circumstances and care to offer a workaround?

Thanks,
Andrew

CL-USER(36): (net.aserve.client::do-http-request "http://altamira.isi.edu/large.html")
Error: The size of this array 17001732 exceeds array-total-size-limit,
       16777216 or is negative
  [condition type: SIMPLE-ERROR]

Restart actions (select using :continue):
 0: Return to Top Level (an "abort" restart).
 1: Abort entirely from this process.
[1] CL-USER(37): array-total-size-limit
16777216
[1] CL-USER(38): (excl::shell "ls -lga /var/www/html/large.html")
-rw-r--rw-    1 philpot  simsgrp  17001732 Jul 15 10:00 /var/www/html/large.html
0
[1] CL-USER(41): net.aserve::*aserve-version*
(1 2 27)

To: Andrew Philpot <[email protected]>
cc: opensource
Subject: Re: ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT 
In-Reply-To: Message from Andrew Philpot <[email protected]> 
   of "Tue, 15 Jul 2003 10:15:44 PDT." <[email protected]> 
Date: Tue, 15 Jul 2003 11:19:34 -0700
From: jkf
Message-Id: <[email protected]>
Sender: owner-opensource
Precedence: bulk


 You can build your own do-http-request that can retrieve any
size response by using
 
    make-http-client-request
    read-client-response-headers
    client-request-read-sequence  (repeat until eof)
    client-request-close


    
for details see:

    http://opensource.franz.com/aserve/aserve-dist/doc/aserve.html#f-make-http-client-request


Return-Path: owner-opensource
Delivery-Date: Wed Jul 16 04:03:08 2003
Received: from turtlemail.franz.com (turtle [192.132.95.23])
	by quadra.franz.com (8.11.6/8.9.0) with ESMTP id h6GB38702564
	for <handler@quadra>; Wed, 16 Jul 2003 04:03:08 -0700
Received: (from majordomo@localhost)
	by turtlemail.franz.com (8.11.6/8.9.0) id h6GB2e910370
	for opensource-list; Wed, 16 Jul 2003 04:02:40 -0700
X-Authentication-Warning: turtle.franz.com: majordomo set sender to [email protected] using -f
Received: from duck.franz.com (duck [10.2.0.2])
	by turtlemail.franz.com (8.11.6/8.9.0) with ESMTP id h6GB2XP10367
	for <[email protected]>; Wed, 16 Jul 2003 04:02:34 -0700
Received: (from root@localhost)
	by duck.franz.com (8.11.6/8.11.6) id h6GB2Vu22867
	for [email protected]; Wed, 16 Jul 2003 04:02:31 -0700
Received: from c06620.glos.ac.uk (mailgw.glos.ac.uk [194.81.184.203])
	by duck.franz.com (8.11.6/8.11.6) with ESMTP id h6GB2TK22842
	for <[email protected]>; Wed, 16 Jul 2003 04:02:30 -0700
X-Fi-Border: duck
Received: from exchange.chelt.local (unverified) by c06620.glos.ac.uk
 (Content Technologies SMTPRS 4.2.5) with ESMTP id <[email protected]>;
 Wed, 16 Jul 2003 12:02:11 +0100
Received: by exchange.glos.ac.uk with Internet Mail Service (5.5.2653.19)
	id <39783D7S>; Wed, 16 Jul 2003 12:02:11 +0100
Message-ID: <[email protected]>
From: "MARTINS, Maria Cleci" <[email protected]>
To: "'Andrew Philpot '" <[email protected]>,
   "'[email protected] '"
	 <opensource>
Subject: RE: ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT
Date: Wed, 16 Jul 2003 12:02:09 +0100
MIME-Version: 1.0
X-Mailer: Internet Mail Service (5.5.2653.19)
Content-Type: text/plain;
	charset="iso-8859-1"
X-scanner: scanned by Inflex 1.0.10 - (http://pldaniels.com/inflex/)
Sender: owner-opensource
Precedence: bulk

Hi everyone!

I am new to Lisp and I would like some help on how to produce webpages using
lisp server. I read the tutorial on dynamic HTML, but I am still confused. 
Please let me know of any references that gives one basic information on how
to build a web project using lisp and Allegro server.

Thanks,

Maria 

....

Date: Wed, 16 Jul 2003 15:52:45 +0200
Subject: AServe examples (was Re: ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT)
Content-Type: text/plain; charset=US-ASCII; format=flowed
Mime-Version: 1.0 (Apple Message framework v552)
Cc: "'[email protected] '" <opensource>
To: "MARTINS, Maria Cleci" <[email protected]>
From: Rudi Schlatte <[email protected]>
In-Reply-To: <[email protected]>
Message-Id: <[email protected]>
Content-Transfer-Encoding: 7bit
X-Pgp-Agent: GPGMail 0.5.4 (v22 Jaguar)
X-Mailer: Apple Mail (2.552)
X-scanner: scanned by Inflex 1.0.10 - (http://pldaniels.com/inflex/)
Sender: owner-opensource
Precedence: bulk

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On Mittwoch, Juli 16, 2003, at 01:02  Uhr, MARTINS, Maria Cleci wrote:

> Hi everyone!
>
> I am new to Lisp and I would like some help on how to produce webpages 
...

Take a look at the examples/ subdirectory in the AllegroServe source; 
it shows how to do a lot of things with AllegroServe.

Rudi

Date: Wed, 16 Jul 2003 11:53:42 -0400
Subject: Re: ASERVE client and pages larger than ARRAY-TOTAL-SIZE-LIMIT
Content-Type: text/plain; charset=US-ASCII; format=flowed
Mime-Version: 1.0 (Apple Message framework v552)
Cc: "'[email protected] '" <opensource>
To: "MARTINS, Maria Cleci" <[email protected]>
From: Brendan Burns <[email protected]>
In-Reply-To: <[email protected]>
Message-Id: <[email protected]>
Content-Transfer-Encoding: 7bit
X-Mailer: Apple Mail (2.552)
X-scanner: scanned by Inflex 1.0.10 - (http://pldaniels.com/inflex/)
Sender: owner-opensource
Precedence: bulk

Hey there,
I've found aserve's htmlgen functionality is a little annoying to use, 
so I developed the code to allow you to inline Lisp in HTML  eg:

<body>
1 + 2 = <% (+ 1 2) %>
</body>

If you (or anyone else) is interested in the code, let me know.

--brendan