Re: Architecture advice requested

chris fedde wrote:
> My second reaction is to wonder if you can do better in perl than a
> demand dial PPP link would provide.
>   

?  You need both.  At $1.50/min you want this dialup link only up when 
you need it

Right now since most users are on windows we just show them the normal 
windows dialup feature and they dial when they need to do stuf

> My third reaction is to wonder if it is not better to use something
> like UUCP and batch load across the link.
>   

Hardly.  Doing the algorithm described you are pretty close to a solid 
block of data moving in both directions simultaneously, including 
compression and management of tcp packet size.  So the efficiency 
approaches that of UUCP + intensive compression, but both ends talk a 
standard protocol

> Interactive protocols like POP, IMAP, and HTTP are going to be very
> hard to use over such a slow high latency link.
>   

?? That's why this project exists in the first place? 

Just to be clear: this exists already, I'm doing a structural 
enhancement.  Current code is really only a few hundred lines at it's 
core, runs to a couple of thousand once you handle all the protocol 
parsing, gui and other special cases

Download a trial from http://www.mailasail.com - the client is packed as 
an EXE, but I doubt that will stop an enterprising soul who wants to see 
how it works...

> If your goal is just email then I'd recommend setting up a local MTA
> and configure it to use ETRN to collect mail queued at an MTA on an
> internet connected host. 

OK, so that's only about 5x slower than my proposed solution.  No 
compression and large latency due to each SMTP command being sent one by 
one.  Don't forget that tcp slow start will also hurt you.

I think my suggestion is much better...

>   You'll also want very good spam filtering
> on the connected side to avoid paying for too much spam.
>   

Sure - done.

Also we do a huge amount more like killing redundant email headers, 
eliminating extra HTML sections and optionally blackberry like 
attachment mangling into text. 



What I am really looking for are suggestions on how to structure a small 
proxy application which will really stretch the POE architecture because 
the outer layers are quite integrated with the inner layers

The inner side of the app is presumably just going to be several wheels 
proxying to each other, eg pop server is the in side, talking to a pop 
client wheel which then goes back out to the network and the real 
server.  However, given that I want to compress this output and 
multiplex it, is it best to call that an output filter?  How might I 
signal to the output filter that I want the compression buffered at this 
point in the stream?  How might I indicate that this stream should get 
priority over this other stream?)

What about the receiving end which gets a compressed stream packing 
several protocols.  There will need to be a bit of logic to handle 
corruption, invalid input, etc.  Then it needs to unpack this into 
several protocol streams and feed them to several protocol parsers.  
Does that sound like an input filter or is it a wheel because there 
might be some state information (several packets to join, etc)

Any thoughts?  Any demo apps which show parts of this functionality?

Thanks

Ed W