Re: Processing Streamed Data

You wouldn't process records one by one, they would be buffered and
flushed as the buffer became full. There are many different
possibilities, different solutions for different needs. I don't know if
sending a zip file over is all that great; it would be a bit slow.
Better to have a stream open between the two processes and compress the
stream on the sending side. But that's just me. Btw, you probably
wouldn't want to process records one by one in real time if you had a
system that required intermittent connections. You might prefer a
scheduled replication.

R.

________________________________

From: Raymond Isom [mailto:[email protected]] 
Sent: Thursday, May 12, 2005 2:40 PM
To: Robert DiFalco; [email protected]
Subject: Re: Processing Streamed Data



Hi, I don't have an answer for you (yes, I'm a newbie
<http://graphics.hotmail.com/i.p.emcrook.gif> ), but I was just
wondering why would you want the server to be held up by processing
records 1 by 1?  Why not have the client invoke a sendFile method on the
server.  This method would read all records into a file, zip it and then
send it to the client to do whatever is needed.  You could even have it
set up where a file of the records already exists and the server appends
new records to it or updates records that have been changed.  This way
server resources would be free to handle new client request instead of
concurrently writing 400k records sequentially.  I guess it would depend
on the possible number of clients able to connect.   

Tell me what you think.  Anyone else care to join in on the problem feel
free.  I love to learn.





>From: Robert DiFalco <[email protected]>
>Reply-To: Robert DiFalco <[email protected]>
>To: [email protected]
>Subject: Re: Processing Streamed Data
>Date: Thu, 12 May 2005 10:53:57 -0700
>
>FWIW, my current guess would be to make an Externalizable class with no
>data, and do all my database queries in #writeExternal flushing the
data
>as appropriate. The client side logic for what to do with the streamed
>data would be in #readExternal. The client wouldn't really do anything
>with the returned object. Just something like this:
>
>         Object o = server.readStreamedData();
>
>The server would just implement the remote method like so:
>
>         public Object readStreamedData() ...
>         {
>             return new MyExternalizable();
>         }
>
>Seems like cheating, but given that I can't find any preferred ways to
>do this with RMI, it seems like this would work. Thoughts?
>
>R.
>
>-----Original Message-----
>From: Robert DiFalco
>Sent: Thursday, May 12, 2005 10:40 AM
>To: [email protected]
>Subject: Processing Streamed Data
>
>I have a large set of data to process, consider say 400,000 records
read
>from a database that I want to pass to another machine to build a local
>copy of the data.
>
>I would like the client to make a request for the data and then setup
an
>ObjectInputStream to receive the streamed data from the server. The
>server would just create an ObjectOutputStream and write the objects
one
>by one, with the client processing them with its ObjectInputStream
until
>EOF is hit.
>
>What approach should I take for doing this with RMI, just using sockets
>it would be pretty simple. Is there already an established protocol
>(besides marshalling) that I could use to do this with RMI?
>
>TIA,
>
>R.
>
>=======================================================================
=
>===
>To unsubscribe, send email to [email protected] and include in the
>body of the message "signoff RMI-USERS".  For general help, send email
>to [email protected] and include in the body of the message "help".
>
>For a list of frequently asked RMI questions please refer to:
>http://java.sun.com/j2se/1.3/docs/guide/rmi/faq.html
>
>To view past RMI-USERS postings, please see:
>http://archives.java.sun.com/archives/rmi-users.html
>
>=======================================================================
====
>To unsubscribe, send email to [email protected] and include in the
body
>of the message "signoff RMI-USERS".  For general help, send email to
>[email protected] and include in the body of the message "help".
>
>For a list of frequently asked RMI questions please refer to:
>http://java.sun.com/j2se/1.3/docs/guide/rmi/faq.html
>
>To view past RMI-USERS postings, please see:
>http://archives.java.sun.com/archives/rmi-users.html

===========================================================================
To unsubscribe, send email to [email protected] and include in the body
of the message "signoff RMI-USERS".  For general help, send email to
[email protected] and include in the body of the message "help".

For a list of frequently asked RMI questions please refer to:
http://java.sun.com/j2se/1.3/docs/guide/rmi/faq.html

To view past RMI-USERS postings, please see:
http://archives.java.sun.com/archives/rmi-users.html