Writing my own FastCGI Server

I'm thinking about writing my own FastCGI Server...

I'll post some reasons below but, I'd appreciate any opinion about it
from you. I'm ready for any kind of criticism.

Now, the story in details. I'll be prolix here, so , read only if
you're really interested on the issue...

We're running WebSite PRO, (WS PRO) from OReilly. We handle
approximatelly 300 K transactions per day. All of them are dynamic
content and httpS. Things would be OK if this server had not been
discontinued. Also, when we have a very large simultaneous demand, the
server gets crowded : it creates too many threads and will not close
them after. Then, from time to time, we need to restart the http server.

Why did we choose THIS server ? Because the story started back in 1998,
when there were few servers out there. And, this server was very easy
because it had the CGI-WIN protocol. The CGI-WIN protocol creates a INI
file and launches the CGI process. Then, it waits for a file to be
ready on a path specified inside the INI file (outfile). So, the CGI
must create that output file before it exits.

At this time, we created a tiny CGI exe that would only "fake" the http
server - instead of creating the out file from it, we had an external
process that would do the job.

Also, we created those files (ini / out file) on a RAMDRIVE folder.

The external process would keep an opened connection (proprietary
protocol) to a Unisys mainframe. Then, requests would be processed
very, very fastly. As a matter of fact , we had to choose to do it like
this because, if we had to open the connection within the CGI itself,
that is, inside each request, it would take about 30 seconds to
process. Unfeasible. So, we choose the external process. Not because we
were smart but, because there was no other way !

But things developed way more than that.

We turned that CGI-WIN into a DLL that runs inside the http server
itself, using it's proprietary API, which is called WSAPI.

This DLL also does a lot more things : it dumps the request into a file
and also commands or controls that external process, creating "n"
clones of it. We call it "tasks". So, it has usually a minimum tasks #
(currently 8) and a maximum tasks # (now 60). When demand increases, it
spawns new "tasks". When demand decreases, it closes them too, of
course. Also keeps a queue of requests to control all that. Those
external tasks are kernel EXEs and, the commercial apps are built into
dynamic DLLs, that is, DLLs that are loaded/unloaded as the requests
come in. The API also has some sort of session affinity so that it try
to dispatch transactions to a task that already has that commercial app
DLL loaded.

It's a multi-tasked environment. On the other hand, the commercial or
daily developers work on these tasks as if they were on a single
threaded environment. They're insulated from these complexities.
Currently we have about 4.000 scripts or different page templates on
top of it. It looks like ASP but we built the language compiler
ourselves...

That WSAPI DLL however is highly or totally multi-threaded.

Anyway, I need to keep this task spawning mechanism. For doing it, I
need some global data structures, like this queue and the state of the
external tasks.

If I pick single FastCGI or even a ready FastCGI server, I'll have to
write some sort of driver on my own. I mean, I'll have to make this
control mechanism outside the server. It will be one more step for the
request to be processed.

Then, I decided to write my own FastCGI server cause I could embed it
all in there.

As for the skills for writing a MT tcp/ip server, I think I already
have them. For instance, I have build one called "TCK", which is
"Telnet Connection Keeper". What is it ? On one side, it receives
transactions from the Web, via http/html. On the other side, it
translates it into Telnet for a Unisys mainframe. IOW, it translates
primitive black and green terminal emulator screens into HTML and
vice-versa. For the mainframe, it's still a terminal. Final users
however access those legacy apps via HTML (it's not a java applet).
And, we can mix or enhance those screens with more things cause, after
translation, it's simply a html string ! For instance, we can spice it
with JavaScript , images or any html tags.

So, I think I have the skills...

Language ? XBase++ (www.alaska-software.com ), which is a Clipper
successor.

The plans are to run it under Windows only. I shall make it public on
the XBase++ NewsGroups but, I dont have the goal to have it running it
everywhere. Just need to solve my particular issue, which is to replace
the http server + the DLL API.

Now, some opinions ? I'll start some of them :

- Be aware of the ....

- You may want to see the source code on ....

- Forget it - too complicated.

- Why dont you do it another way ...

- If you do that, you'll have to ask permission to Mr....

Again, thank you so much for your cooperation and sorry for being so
over-talkative !

--
Obrigado,
------------------------------------------------------------------------------
Jose Adriano Baltieri - Analista de Sistemas
DTI - CENTRO - UNIMEP - Universidade Metodista de Piracicaba
PIRACICABA - SP - Brasil - Fone : (19) 3124-1858
------------------------------------------------------------------------------

___________________________________
fastcgi-developers mailing list
http://fastcgi.com/fastcgi-developers/