AW: AW: Proper upload streaming? Not possible with WO!

Hi Will,

I think you misinterprete the log. I always thought one WorkerThread handles one request completely. If so, WorkerThread4 is the thread of interest. WOApplication.createRequest, then awake and the dispatchRequest is called without any delay.
Everything is fine :-)


Ciao Frank

 
-----Ursprüngliche Nachricht-----
Von: Will Scheidegger [mailto:[email protected]] 
Gesendet: Dienstag, 25. April 2006 18:12
An: Frank Caputo
Cc: [email protected]
Betreff: Re: AW: Proper upload streaming? Not possible with WO!

Hi Frank

Thanks for your reply. This might be true, as Karl mentioned, but still the question remains, why it takes 17 seconds until
createRequest() gets called and then less than a second to upload > 500KB over a 400kbits/sec upstream connection.

The only explanation for me is still that the whole request gets uploaded first to the adapter an is then handed in one piece to WO.

- Will

P.S.: Some of the _private classes really should not be _private but rather well documented. The only reference to the class you've mentioned was in a post from David Teran claiming that what I want to do is very simple and could be done with less then 100 lines of code.  
Unfortunately we cannot ask David anymore what he would think about the "upload delay" I'm experiencing.

On 25.04.2006, at 16:14, Frank Caputo wrote:

> Hi Will,
>
> I guess the NSData parameter in the createRequest is not a 
> com.webobjects.foundation.NSData, but a 
> com.webobjects.appserver._private.WOInputStreamData, which wraps an 
> inputstream. Streaming should work fine unless you access the 
> formvalues.
>
>
> Regards Frank
>
>
> -----Ursprüngliche Nachricht-----
> Von: [email protected] [mailto:webobjects-dev- 
> [email protected]] Im Auftrag von Will Scheidegger
> Gesendet: Dienstag, 25. April 2006 14:06
> An: webobjects-dev Forms
> Betreff: Re: Proper upload streaming? Not possible with WO!
>
> After some more debugging and playing with the developer examples I 
> came to the conclusion that truly streaming a request to WO is _not 
> possible_!
>
> Here's the output of a modified wis request, where the whole http 
> request gets "streamed" directly to the disk - at least according to 
> Apple docs:
>
> [2006-04-25 11:15:21 CEST] <WorkerThread3> Application.createRequest
> (): /cgi-bin/WebObjects/FileUpload.woa/womp/instanceRequest
> [2006-04-25 11:15:21 CEST] <WorkerThread3>
> Application.dispatchRequest()
> [2006-04-25 11:15:21 CEST] <WorkerThread3> Application.awake()
> [2006-04-25 11:15:38 CEST] <WorkerThread4> Application.createRequest
> (): /cgi-bin/WebObjects/FileUpload.woa/1/wis/ProcessRequestContent
> [2006-04-25 11:15:38 CEST] <WorkerThread4> Data size: 571602
> [2006-04-25 11:15:38 CEST] <WorkerThread4>
> Application.dispatchRequest()
> [2006-04-25 11:15:38 CEST] <WorkerThread4> Application.awake() 
> ===============================================
> Direct Action Request Content Streaming Upload:
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 84585 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 142985 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 217445 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 253945 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 297745 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 332785 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 416005 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 472945 bytes uploaded
> [2006-04-25 11:15:38 CEST] <WorkerThread4> 571602 bytes uploaded Wrote 
> file to '/var/tmp/DARCSUReq56899.tmp'
>
> The first three lines are from wotaskd which checked to see if the app 
> was alive. They don't have anything to do with my problem...
> except that I triggered the upload exactly at this time. Then, 17 
> seconds later, my request finally arrived at the application. Now if 
> the request would really be streamed directly to the disk, then I 
> would be one happy owner of a very fast internet connection since the 
> rest (uploading, streaming to disk of > 500kb) would have happened in 
> less than a second - which would result in a 4Mbits/sec upload stream!
>
> Now when you look at the method that gets called first by the 
> woadaptor - createRequest() - you'll notice that this method already 
> has an NSData object as parameter. According to the docs this 
> parameter contains "the HTML content of the receiver", which i don't 
> fully understand. But since it's size is exactly the total size of the 
> whole request, I guess we know what this means. Looking at the time 
> span between the moment I submitted the form and the moment the app 
> got called by the adaptor (17 seconds) makes me believe that too that 
> the upload actually happens _before_ the request is forwarded to the 
> WO app which would definitely _not_ correspond to what is said in the 
> docs and the samples:
>
> "Because the request content is not displayed or read into memory, 
> this page can be used to upload an arbitrarily large file."
>
> Hm... if that NSData object containing the whole request data is not 
> in memory, where is it then?
>
> If I'm missing something please do tell me so. Otherwise I'm going to 
> waste a Tech Support incident on this.
>
> Thanks!
>
> -Will
>
> On 24.04.2006, at 16:15, Kevin Windham wrote:
>
>>
>> On Apr 24, 2006, at 4:41 AM, Will Scheidegger wrote:
>>
>>> I'm looking for a solution for proper upload streaming in WO. I want 
>>> the user to be able to upload a lot of data (file upload) to my WO 
>>> app and want Result with all 3 attempts:
>>> - A long period with no visible action at all
>>> - Then the upload appears within a second
>>
>> I suspect what you are seeing here is the browser preparing the post 
>> response. The format that browsers upload with requires a boundary to 
>> be assigned in the headers preceding the data. This boundary has to 
>> be checked to make sure it doesn't exist in the file being uploaded. 
>> If it did, it would mess up the parsing of the post data. The bigger 
>> the file the longer this check takes. I'm not sure but there may be 
>> some implementations that use probability to predetermine a boundary 
>> that doesn't exist in the file, but I've noticed the slow start 
>> behavior on all the browsers I tested with when writing my own parser 
>> for doing uploads a while back.
>>
>> HTH,
>> Kevin
>> _______________________________________________
>> WebObjects-dev mailing list
>> [email protected]
>> http://www.omnigroup.com/mailman/listinfo/webobjects-dev
>
> _______________________________________________
> WebObjects-dev mailing list
> [email protected]
> http://www.omnigroup.com/mailman/listinfo/webobjects-dev