[Patch] Improve handling of interrupted system calls
Rainer Jung <[email protected]>
| Newsgroups | gmane.comp.web.fastcgi.devel |
|---|---|
| Message-ID | <[email protected]> |
Hello, during stress tests we realized, that mod_fastcgi aborts requests in some cases, where the root cause is an interrupted system call. After adding a retry functionality, we noticed no more of these errors and the responses were delivered with the correct content. So I attach the patch we used. Some comments: There are two situations, that are handled by the patch: A) After select() in socket_io() indicates, that the response can be read, but the following readv() in fcgi_buf_socket_recv() returns with EAGAIN. Normally this should not happen, but it isn't forbidden and at least on AIX 5.2 we noticed exactly this behaviour. It only showed up sporadically, but when it happens, because mod_fastcgi aborts the request we get status codes 500. B) select() in socket_io() can return with EINTR. We implemented a retry loop, which sleep 1 second and loops max 10 times in order to prevent a non terminating loop. We also noticed this behaviour on AIX 5.2 under stress (sporadically). The loop never ran more than twice. The one second sleep might not be ideal, but since the problem only happens very rarely, it looked safer, than a shorter sleep. Thanks for producing mod_fastcgi! Regards, Rainer _______________________________________________ FastCGI-developers mailing list FastCGI-developers-xGejAJT2w6xVgU18Zptdi0EOCMrvLtNR@public.gmane.org http://mailman.pins.net/mailman/listinfo.cgi/fastcgi-developers
mod_fastcgi.c.patch
(text/x-patch, 1.7 KB)
--- mod_fastcgi.c.orig 2008-01-07 11:58:32.000000000 +0100
+++ mod_fastcgi.c 2008-06-23 13:35:24.000000000 +0200
@@ -1972,6 +1972,7 @@
static int socket_io(fcgi_request * const fr)
{
+ int select_count;
enum
{
STATE_SOCKET_NONE,
@@ -2193,7 +2194,18 @@
}
/* wait on the socket */
- select_status = ap_select(nfds, &read_set, &write_set, NULL, &timeout);
+ select_count = 0;
+ do {
+ if (select_count) {
+ sleep(1);
+ ap_log_rerror(FCGI_LOG_INFO_NOERRNO, r,
+ "FastCGI: select returned %d, retry number %d",
+ select_status, select_count);
+ }
+ select_status = ap_select(nfds, &read_set, &write_set, NULL, &timeout);
+ select_count++;
+ } while (select_status < 0 && select_count <= FCGI_MAX_SELECT_RETRIES &&
+ errno == EINTR);
if (select_status < 0)
{
@@ -2264,9 +2276,15 @@
if (rv < 0)
{
- ap_log_rerror(FCGI_LOG_ERR, r, "FastCGI: comm with server "
- "\"%s\" aborted: read failed", fr->fs_path);
- state = STATE_ERROR;
+ if (errno == EAGAIN) {
+ ap_log_rerror(FCGI_LOG_INFO_NOERRNO, r, "FastCGI: comm with server "
+ "\"%s\" interrupted: read will be retried in 1 second", fr->fs_path);
+ sleep(1);
+ } else {
+ ap_log_rerror(FCGI_LOG_ERR, r, "FastCGI: comm with server "
+ "\"%s\" aborted: read failed", fr->fs_path);
+ state = STATE_ERROR;
+ }
break;
}
mod_fastcgi.h.patch
(text/x-patch, 591 B)
--- mod_fastcgi.h.orig 2008-01-07 11:58:32.000000000 +0100
+++ mod_fastcgi.h 2008-06-23 13:11:18.000000000 +0200
@@ -54,6 +54,7 @@
#define FCGI_MIN_EXEC_RETRY_DELAY 10 /* minimum number of seconds to
wait before restarting */
#define MAX_INIT_ENV_VARS 64 /* max # of -initial-env options */
+#define FCGI_MAX_SELECT_RETRIES 10 /* max # of retries after select returns EINTR */
/* max number of chars in a line of stderr we can handle from a FastCGI Server */
#define FCGI_SERVER_MAX_STDERR_LINE_LEN 1023