Accessing File-like Structures in a String World

On Wed,  6 May 2009 11:25:36 -0400 (EDT)
[email protected] (Bill Yerazunis) wrote:

> So- Chris- can you send a pointer to this magic file that 
> can't be read properly?

Thanks, Bill, but Paolo got this one...

It was an ordinary PNM bitmap graphic. The main issue was the per call
input limit on "syscall <keep>". I was trying to duplicate an image
manipulation process that is normally performed in a pipeline like
"pselect | gs | pnmflip | pnmcrop | pnmtogif" or the like. The file
system I/O for CRM is generally faster than native OS calls so doing
something *like* this is often as fast as a pipe and enables the script
to reuse the streams:

	syscall (:ps_doc:) (:ps_page:) /psselect/
	syscall (:ps_page:) (:pnm_raw:) /gs -sDEVICE=pnmraw/
	syscall (:pnm_raw:) (:pnm_oriented:) /pnmflip/
	syscall (:pnm_oriented:) (:pnm_cropped:) /pnmcrop/
	syscall <async> (:pnm_oriented:) /pnmtogif full.gif/
	syscall <async> (:pnm_cropped:) /pnmtogif cropped.gif/

With Paolo's help, I got the original concept working in a way that
unfortunately negates the advantage of the  idea. Something like this
might work and have some advantage over a physical file:

	syscall (:ps_page:) /gs -sDEVICE=pnmraw >3/	
	input (:pnm_raw:) [\/dev\/fd\/3]

I just don't have a clue on what to do to make file descriptors work
within a script and a sneaky suspicion that syscall will block waiting
for the pipe to be open for a read. If that's the case, then calling gs
asyncronously won't solve the problem... or at least it didn't when I
was using named pipes to communicate with a database accessor:

	syscall <async> /mkfifo  :*:fifo:/
	syscall <async> /:sql-call:/
	## Initialize database
	input (:query:) [initialize.sql]
	output [:*:fifo:] /:*:query:/
	## (fork processes that read and analyse data)
	exit

	:sql-call:
	syscall <keep> (::) (::) (:status:) /:*:sql: <:*:fifo:/
	exit

The idea here is that multiple forked processes can use CRM I/O to
communicate with a single instance of the SQL client. This is an
example of a script trying to populate a database as rapidly as
possible. The SQL client stays alive and syscall blocks until the FIFO
is closed. If we wanted to get data from the process then the process
would redirect stdout to a pipe and the script would use "input
<byline> ..." to read it. A good application of this is a forked
instance of nc or socat. If you use async and stdio then you have to
poll the process. With a pipe, "syscall <keep>" and "input <byline>"
your handler can block waiting on input and handle each line of input
in a network dialog. Without input <byline>, syscall <keep> blocks
until EOF... or a fixed amount of data in the input buffer.

I've been playing with the architecture of this application a little
and started to get a handle on version control for the files. I'd like
to use this knowledge for evil. I mean I want to set up a predictive
cache. I'm not going to use PNM files for this because 2.6 MB is too
big; It would still be helpful to know whether I have all the data when
syscall <keep> returns. If there is some way to control the size of the
input buffer at run time (or at least predict it) I would use it here.

Second choice would be file descriptors, which I will have to figure
out sooner or later to filter streams on chat gateways with socat.

Thanks,
Chris

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image 
processing features enabled. http://p.sf.net/sfu/kodak-com

_______________________________________________
Crm114-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crm114-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iQEcBAEBAgAGBQJKAfm3AAoJEASgqNsqZfCHpKAH/3YxmmjKMXcNCr2d44+02O5u
mv85IlDNJ7ZLkOU9E2S+r71NROvm1OvC30OZ62zbUYSnYv2IBTbwiUpYzBG6W5sj
rpMRSsad45IHZ2opAbvmgKn+9kmJmoAPEXDozwvRhwqWx5pEAxmqzp2Vzh8+UDdQ
Fp1I7w2WqwV1IwvGPyrIVgK96Lun24OTohOXOWiVvRnKwTIfFxRjhDBy3Wny7AyY
Cqel+/yUHN6v7EzuxlzHbblrW5ECQvKz2IFK0qG419AMHzNUKahfRgGoH2iu9bEV
h00riKOc3AKwh7wPQkbKv0mnXHfLao6jbpLwU7bPnZreqVhaqWvIr07m4vScfvI=
=3hp3
-----END PGP SIGNATURE-----