Re: listen backlog patch
Stephan Leemburg <[email protected]>
| Newsgroups | gmane.comp.audio.icecast.devel |
|---|---|
| Organization | NPO |
| Message-ID | <[email protected]> |
Hi Thomas, I don't know if you like top or bottom quoting. That seems to be a big-little endian thing ;-) So, I will top quote and inline quote. Please see my comments inline, below. Kind regards, Stephan On 02/19/2015 04:18 PM, "Thomas B. Rücker" wrote: > Hi, > > On 02/19/2015 03:07 PM, Stephan Leemburg wrote: >> Hello Icecast-dev, >> >> I am new to this list. > > Welcome! Thank you. > >> I am working for the NPO, the Dutch Public Broadcasting agency. >> We do a lot of icecast streaming. We run at least 20 icecast server >> instances on our media streaming cluster. > > That's very nice to hear. > >> We ran into an issue that clients which where connecting to our streams >> seemed to be 'hanging' on the connection setup frequently. The client >> 'thinks' it is connected, but no data. >> >> People suggested that it probably had to do with the poll() call. So, I >> looked into that. >> >> I found that the issue was actually caused by the very low listen >> backlog (5). >> >> On our clusters, we typically set this to 8192. Yes it is high, but we >> do a _lot_ of streaming and host very high volume websites. > > I'm not very familiar with socket programming, so will let Philipp > comment on this. Interesting enough this issue hasn't come up so far as > far as I can tell and there are some pretty high load deployments out there. We often have 'bursts' of new connections. Due to something said on a website, radio, tv, some top-2000 end of year event, etc. And when we get a lot of simultanious connection requests, this becomes an issue. I wrote a small server to simulate it. And the linux backlog acts a little bit different then expected. > > If you can share that info, what sort of concurrent listener load are we > talking about? Sure I can share, we are a national broadcasting agency funded by tax money. So no secrets here ;-) I just asked the media streaming guys.. 70k icecast connections on a regular day and 150k around special broadcasts (like top 2000 around new year). > > >> Currently we are using icecast 2.3. We are migrating to 2.4. >> So, I have written patches for 2.3 and 2.4, but also for the current 2.5 >> git tree. >> >> Unfortunately, I am a newby when it comes to git (sorry). But I do have >> unified diff patch files for the 2.3, 2.4 and 2.5 source trees. >> >> The patched 2.4 icecast was tested by our media streaming team and they >> confirmed that their issue was solved by it. >> >> Can I submit them (and how)? > > Just send them as attachments to this list, or open a ticket over at > https://trac.xiph.org. > If trac is naughty and thinks you're a spammer, please let me know. > > If there are differences between them, then please the 2.5 and 2.4 > patches. Otherwise 2.5 will do just fine. > I have attached the 2.4 and 2.5 patches. > > Thanks a lot for taking the time to reach out to us! Thank you for your Open Source contributions and efforts! And as it is Open Source, we can fix problems ourself and share. > > > Cheers > > Thomas > > _______________________________________________ > Icecast-dev mailing list > [email protected] > http://lists.xiph.org/mailman/listinfo/icecast-dev > -- Met vriendelijke groet, Stephan Leemburg, internetbeheerder NPO ICT Internet Services Bart de Graaffweg 2, 1217 ZL Hilversum [email protected], 035-6773555 _______________________________________________ Icecast-dev mailing list [email protected] http://lists.xiph.org/mailman/listinfo/icecast-dev
icecast-2.4.1-somaxconn.patch
(text/x-patch, 4.4 KB)
diff -Naur ../icecast-2.4.1-org/src/cfgfile.c ./src/cfgfile.c
--- ../icecast-2.4.1-org/src/cfgfile.c 2014-11-17 22:37:44.000000000 +0100
+++ ./src/cfgfile.c 2015-02-06 10:51:37.300458056 +0100
@@ -32,12 +32,17 @@
#include "client.h"
#include "logging.h"
+#ifndef SOMAXCONN
+#define SOMAXCONN 128
+#endif
+
#define CATMODULE "CONFIG"
#define CONFIG_DEFAULT_LOCATION "Earth"
#define CONFIG_DEFAULT_ADMIN "icemaster@localhost"
#define CONFIG_DEFAULT_CLIENT_LIMIT 256
#define CONFIG_DEFAULT_SOURCE_LIMIT 16
#define CONFIG_DEFAULT_QUEUE_SIZE_LIMIT (500*1024)
+#define CONFIG_DEFAULT_BACKLOG_SIZE_LIMIT SOMAXCONN
#define CONFIG_DEFAULT_BURST_SIZE (64*1024)
#define CONFIG_DEFAULT_THREADPOOL_SIZE 4
#define CONFIG_DEFAULT_CLIENT_TIMEOUT 30
@@ -412,6 +417,7 @@
configuration->client_limit = CONFIG_DEFAULT_CLIENT_LIMIT;
configuration->source_limit = CONFIG_DEFAULT_SOURCE_LIMIT;
configuration->queue_size_limit = CONFIG_DEFAULT_QUEUE_SIZE_LIMIT;
+ configuration->backlog_size_limit = CONFIG_DEFAULT_BACKLOG_SIZE_LIMIT;
configuration->threadpool_size = CONFIG_DEFAULT_THREADPOOL_SIZE;
configuration->client_timeout = CONFIG_DEFAULT_CLIENT_TIMEOUT;
configuration->header_timeout = CONFIG_DEFAULT_HEADER_TIMEOUT;
@@ -606,6 +612,10 @@
tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
configuration->queue_size_limit = atoi(tmp);
if (tmp) xmlFree(tmp);
+ } else if (xmlStrcmp (node->name, XMLSTR("backlog-size")) == 0) {
+ tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
+ configuration->backlog_size_limit = atoi(tmp);
+ if (tmp) xmlFree(tmp);
} else if (xmlStrcmp (node->name, XMLSTR("threadpool")) == 0) {
tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
configuration->threadpool_size = atoi(tmp);
diff -Naur ../icecast-2.4.1-org/src/cfgfile.h ./src/cfgfile.h
--- ../icecast-2.4.1-org/src/cfgfile.h 2014-11-17 22:37:44.000000000 +0100
+++ ./src/cfgfile.h 2015-02-06 10:51:37.302458052 +0100
@@ -147,6 +147,7 @@
int client_limit;
int source_limit;
unsigned int queue_size_limit;
+ unsigned int backlog_size_limit;
int threadpool_size;
unsigned int burst_size;
int client_timeout;
diff -Naur ../icecast-2.4.1-org/src/connection.c ./src/connection.c
--- ../icecast-2.4.1-org/src/connection.c 2014-11-08 15:38:05.000000000 +0100
+++ ./src/connection.c 2015-02-06 10:51:37.303458050 +0100
@@ -1445,7 +1445,7 @@
sock_t sock = sock_get_server_socket (listener->port, listener->bind_address);
if (sock == SOCK_ERROR)
break;
- if (sock_listen (sock, ICECAST_LISTEN_QUEUE) == SOCK_ERROR)
+ if (sock_listen (sock, config->backlog_size_limit) == SOCK_ERROR)
{
sock_close (sock);
break;
diff -Naur ../icecast-2.4.1-org/src/global.h ./src/global.h
--- ../icecast-2.4.1-org/src/global.h 2014-11-06 12:57:01.000000000 +0100
+++ ./src/global.h 2015-02-06 10:51:37.303458050 +0100
@@ -13,8 +13,6 @@
#ifndef __GLOBAL_H__
#define __GLOBAL_H__
-#define ICECAST_LISTEN_QUEUE 5
-
#define ICECAST_RUNNING 1
#define ICECAST_HALTING 2
diff -Naur ../icecast-2.4.1-org/src/main.c ./src/main.c
--- ../icecast-2.4.1-org/src/main.c 2014-11-08 15:25:28.000000000 +0100
+++ ./src/main.c 2015-02-06 11:11:49.820120360 +0100
@@ -294,8 +294,10 @@
static int _start_listening(void)
{
int i;
+ ice_config_t *config = config_get_config_unlocked();
+
for(i=0; i < global.server_sockets; i++) {
- if (sock_listen(global.serversock[i], ICECAST_LISTEN_QUEUE) == SOCK_ERROR)
+ if (sock_listen(global.serversock[i], config->backlog_size_limit) == SOCK_ERROR)
return 0;
sock_set_blocking(global.serversock[i], 0);
diff -Naur ../icecast-2.4.1-org/src/net/sock.c ./src/net/sock.c
--- ../icecast-2.4.1-org/src/net/sock.c 2013-01-06 11:29:12.000000000 +0100
+++ ./src/net/sock.c 2015-02-19 16:36:11.453150373 +0100
@@ -69,6 +69,10 @@
# define AI_ADDRCONFIG 0
#endif
+#ifndef SOMAXCONN
+#define SOMAXCONN 128
+#endif
+
/* sock_initialize
**
** initializes the socket library. you must call this
@@ -913,8 +917,9 @@
if (!sock_valid_socket(serversock))
return 0;
- if (backlog <= 0)
- backlog = 10;
+ if (backlog <= 0)
+ backlog = SOMAXCONN;
+
return (listen(serversock, backlog) == 0);
}
icecast-2.5-somaxconn.patch
(text/x-patch, 7.3 KB)
diff -Naur icecast-server-vanilla/conf/icecast_urlauth.xml.in icecast-server-patched/conf/icecast_urlauth.xml.in
--- icecast-server-vanilla/conf/icecast_urlauth.xml.in 2015-02-04 13:20:14.241981200 +0100
+++ icecast-server-patched/conf/icecast_urlauth.xml.in 2015-02-19 15:22:20.451086458 +0100
@@ -4,6 +4,7 @@
<clients>100</clients>
<sources>2</sources>
<queue-size>524288</queue-size>
+ <backlog-size>8192</backlog-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
diff -Naur icecast-server-vanilla/conf/icecast.xml.in icecast-server-patched/conf/icecast.xml.in
--- icecast-server-vanilla/conf/icecast.xml.in 2015-02-18 16:45:55.800129862 +0100
+++ icecast-server-patched/conf/icecast.xml.in 2015-02-19 15:22:39.652009936 +0100
@@ -16,6 +16,7 @@
<clients>100</clients>
<sources>2</sources>
<queue-size>524288</queue-size>
+ <backlog-size>8192</backlog-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
diff -Naur icecast-server-vanilla/doc/config-file.html icecast-server-patched/doc/config-file.html
--- icecast-server-vanilla/doc/config-file.html 2015-02-04 13:20:14.254981187 +0100
+++ icecast-server-patched/doc/config-file.html 2015-02-19 15:21:04.645388567 +0100
@@ -65,6 +65,7 @@
<span class="nt"><clients></span>100<span class="nt"></clients></span>
<span class="nt"><sources></span>2<span class="nt"></sources></span>
<span class="nt"><queue-size></span>102400<span class="nt"></queue-size></span>
+ <span class="nt"><backlog-size></span>102400<span class="nt"></backlog-size></span>
<span class="nt"><client-timeout></span>30<span class="nt"></client-timeout></span>
<span class="nt"><header-timeout></span>15<span class="nt"></header-timeout></span>
<span class="nt"><source-timeout></span>10<span class="nt"></source-timeout></span>
@@ -89,6 +90,10 @@
found will be removed from the stream. This will be the default setting for the streams which is
512k unless overridden here. You can override this in the individual mount settings which can be
useful if you have a mixture of high bandwidth video and low bitrate audio streams.</dd>
+ <dt>backlog-size</dt>
+ <dd>This defines the size of the listen backlog queue. On Linux the default is 128, but it can be raised by
+ issueing a sysctl command. The actual backlog queue size is the minimum of this setting and the kernel setting.
+ </dd>
<dt>client-timeout</dt>
<dd>This does not seem to be used.</dd>
<dt>header-timeout</dt>
diff -Naur icecast-server-vanilla/src/cfgfile.c icecast-server-patched/src/cfgfile.c
--- icecast-server-vanilla/src/cfgfile.c 2015-02-04 13:20:14.272981169 +0100
+++ icecast-server-patched/src/cfgfile.c 2015-02-19 15:10:37.959944608 +0100
@@ -40,12 +40,17 @@
#include "fserve.h"
#include "stats.h"
+#ifndef SOMAXCONN
+#define SOMAXCONN 128
+#endif
+
#define CATMODULE "CONFIG"
#define CONFIG_DEFAULT_LOCATION "Earth"
#define CONFIG_DEFAULT_ADMIN "icemaster@localhost"
#define CONFIG_DEFAULT_CLIENT_LIMIT 256
#define CONFIG_DEFAULT_SOURCE_LIMIT 16
#define CONFIG_DEFAULT_QUEUE_SIZE_LIMIT (500*1024)
+#define CONFIG_DEFAULT_BACKLOG_SIZE_LIMIT SOMAXCONN
#define CONFIG_DEFAULT_BURST_SIZE (64*1024)
#define CONFIG_DEFAULT_THREADPOOL_SIZE 4
#define CONFIG_DEFAULT_CLIENT_TIMEOUT 30
@@ -707,6 +712,8 @@
configuration
->queue_size_limit = CONFIG_DEFAULT_QUEUE_SIZE_LIMIT;
configuration
+ ->backlog_size_limit = CONFIG_DEFAULT_BACKLOG_SIZE_LIMIT;
+ configuration
->client_timeout = CONFIG_DEFAULT_CLIENT_TIMEOUT;
configuration
->header_timeout = CONFIG_DEFAULT_HEADER_TIMEOUT;
@@ -1035,6 +1042,11 @@
configuration->queue_size_limit = atoi(tmp);
if (tmp)
xmlFree(tmp);
+ } else if (xmlStrcmp (node->name, XMLSTR("backlog-size")) == 0) {
+ tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
+ configuration->backlog_size_limit = atoi(tmp);
+ if (tmp)
+ xmlFree(tmp);
} else if (xmlStrcmp (node->name, XMLSTR("threadpool")) == 0) {
ICECAST_LOG_WARN("<threadpool> deprecated and will be removed in version 2.5.");
} else if (xmlStrcmp (node->name, XMLSTR("client-timeout")) == 0) {
diff -Naur icecast-server-vanilla/src/cfgfile.h icecast-server-patched/src/cfgfile.h
--- icecast-server-vanilla/src/cfgfile.h 2015-02-04 13:20:14.272981169 +0100
+++ icecast-server-patched/src/cfgfile.h 2015-02-19 15:11:25.977741850 +0100
@@ -108,6 +108,8 @@
*/
int burst_size;
unsigned int queue_size_limit;
+ /* listen backlog */
+ unsigned int backlog_size_limit;
/* Do we list this on the xsl pages */
int hidden;
/* source timeout in seconds */
diff -Naur icecast-server-vanilla/src/common/net/sock.c icecast-server-patched/src/common/net/sock.c
--- icecast-server-vanilla/src/common/net/sock.c 2015-02-04 13:20:27.382968417 +0100
+++ icecast-server-patched/src/common/net/sock.c 2015-02-19 15:15:07.755810882 +0100
@@ -66,6 +66,10 @@
# define AI_ADDRCONFIG 0
#endif
+#ifndef SOMAXCONN
+#define SOMAXCONN 128
+#endif
+
/* sock_initialize
**
** initializes the socket library. you must call this
@@ -922,8 +926,9 @@
if (!sock_valid_socket(serversock))
return 0;
- if (backlog <= 0)
- backlog = 10;
+ if (backlog <= 0)
+ backlog = SOMAXCONN;
+
return (listen(serversock, backlog) == 0);
}
diff -Naur icecast-server-vanilla/src/connection.c icecast-server-patched/src/connection.c
--- icecast-server-vanilla/src/connection.c 2015-02-13 15:59:49.347310224 +0100
+++ icecast-server-patched/src/connection.c 2015-02-19 15:12:14.386537440 +0100
@@ -1557,7 +1557,7 @@
sock_t sock = sock_get_server_socket (listener->port, listener->bind_address);
if (sock == SOCK_ERROR)
break;
- if (sock_listen (sock, ICECAST_LISTEN_QUEUE) == SOCK_ERROR) {
+ if (sock_listen (sock, config->backlog_size_limit) == SOCK_ERROR) {
sock_close (sock);
break;
}
diff -Naur icecast-server-vanilla/src/global.h icecast-server-patched/src/global.h
--- icecast-server-vanilla/src/global.h 2015-02-04 13:20:14.285981157 +0100
+++ icecast-server-patched/src/global.h 2015-02-19 15:02:39.432965217 +0100
@@ -13,8 +13,6 @@
#ifndef __GLOBAL_H__
#define __GLOBAL_H__
-#define ICECAST_LISTEN_QUEUE 5
-
#define ICECAST_RUNNING 1
#define ICECAST_HALTING 2
diff -Naur icecast-server-vanilla/src/main.c icecast-server-patched/src/main.c
--- icecast-server-vanilla/src/main.c 2015-02-04 13:20:14.285981157 +0100
+++ icecast-server-patched/src/main.c 2015-02-19 15:02:39.432965217 +0100
@@ -304,8 +304,10 @@
static int _start_listening(void)
{
int i;
+ ice_config_t *config = config_get_config_unlocked();
+
for(i=0; i < global.server_sockets; i++) {
- if (sock_listen(global.serversock[i], ICECAST_LISTEN_QUEUE) == SOCK_ERROR)
+ if (sock_listen(global.serversock[i], config->backlog_size_limit) == SOCK_ERROR)
return 0;
sock_set_blocking(global.serversock[i], 0);
icecast-2.3.3-somaxconn.patch
(text/x-patch, 3.9 KB)
--- ./src/cfgfile.h.org 2015-02-05 15:48:24.339178719 +0100
+++ ./src/cfgfile.h 2015-02-05 15:49:48.482977902 +0100
@@ -119,6 +119,7 @@
int client_limit;
int source_limit;
unsigned int queue_size_limit;
+ unsigned int backlog_size_limit;
int threadpool_size;
unsigned int burst_size;
int client_timeout;
--- ./src/main.c.org 2015-02-05 15:54:26.254365035 +0100
+++ ./src/main.c 2015-02-05 15:55:23.140239523 +0100
@@ -278,8 +278,10 @@
static int _start_listening(void)
{
int i;
+ ice_config_t *config = config_get_config_unlocked();
+
for(i=0; i < global.server_sockets; i++) {
- if (sock_listen(global.serversock[i], ICE_LISTEN_QUEUE) == SOCK_ERROR)
+ if (sock_listen(global.serversock[i], config->backlog_size_limit) == SOCK_ERROR)
return 0;
sock_set_blocking(global.serversock[i], 0);
--- ./src/connection.c.org 2015-02-05 15:52:24.133634479 +0100
+++ ./src/connection.c 2015-02-05 15:53:35.288477485 +0100
@@ -1414,7 +1414,7 @@
sock_t sock = sock_get_server_socket (listener->port, listener->bind_address);
if (sock == SOCK_ERROR)
break;
- if (sock_listen (sock, ICE_LISTEN_QUEUE) == SOCK_ERROR)
+ if (sock_listen (sock, config->backlog_size_limit) == SOCK_ERROR)
{
sock_close (sock);
break;
--- ./src/cfgfile.c.org 2015-02-05 15:44:45.084784201 +0100
+++ ./src/cfgfile.c 2015-02-05 15:57:06.616011217 +0100
@@ -29,12 +29,17 @@
#include "client.h"
#include "logging.h"
+#ifndef SOMAXCONN
+#define SOMAXCONN 128
+#endif
+
#define CATMODULE "CONFIG"
#define CONFIG_DEFAULT_LOCATION "Earth"
#define CONFIG_DEFAULT_ADMIN "icemaster@localhost"
#define CONFIG_DEFAULT_CLIENT_LIMIT 256
#define CONFIG_DEFAULT_SOURCE_LIMIT 16
#define CONFIG_DEFAULT_QUEUE_SIZE_LIMIT (500*1024)
+#define CONFIG_DEFAULT_BACKLOG_SIZE_LIMIT SOMAXCONN
#define CONFIG_DEFAULT_BURST_SIZE (64*1024)
#define CONFIG_DEFAULT_THREADPOOL_SIZE 4
#define CONFIG_DEFAULT_CLIENT_TIMEOUT 30
@@ -347,6 +352,7 @@
configuration->client_limit = CONFIG_DEFAULT_CLIENT_LIMIT;
configuration->source_limit = CONFIG_DEFAULT_SOURCE_LIMIT;
configuration->queue_size_limit = CONFIG_DEFAULT_QUEUE_SIZE_LIMIT;
+ configuration->backlog_size_limit = CONFIG_DEFAULT_BACKLOG_SIZE_LIMIT;
configuration->threadpool_size = CONFIG_DEFAULT_THREADPOOL_SIZE;
configuration->client_timeout = CONFIG_DEFAULT_CLIENT_TIMEOUT;
configuration->header_timeout = CONFIG_DEFAULT_HEADER_TIMEOUT;
@@ -518,6 +524,10 @@
tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
configuration->queue_size_limit = atoi(tmp);
if (tmp) xmlFree(tmp);
+ } else if (xmlStrcmp (node->name, XMLSTR("backlog-size")) == 0) {
+ tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
+ configuration->backlog_size_limit = atoi(tmp);
+ if (tmp) xmlFree(tmp);
} else if (xmlStrcmp (node->name, XMLSTR("threadpool")) == 0) {
tmp = (char *)xmlNodeListGetString(doc, node->xmlChildrenNode, 1);
configuration->threadpool_size = atoi(tmp);
--- ./src/global.h.org 2015-02-05 13:35:31.546686092 +0100
+++ ./src/global.h 2015-02-05 15:56:08.271139948 +0100
@@ -13,8 +13,6 @@
#ifndef __GLOBAL_H__
#define __GLOBAL_H__
-#define ICE_LISTEN_QUEUE 5
-
#define ICE_RUNNING 1
#define ICE_HALTING 2
--- ./src/net/sock.c.org 2015-02-05 13:38:01.933228126 +0100
+++ ./src/net/sock.c 2015-02-05 15:59:47.562656109 +0100
@@ -69,6 +69,10 @@
# define AI_ADDRCONFIG 0
#endif
+#ifndef SOMAXCONN
+#define SOMAXCONN 128
+#endif
+
/* sock_initialize
**
** initializes the socket library. you must call this
@@ -913,8 +917,9 @@
if (!sock_valid_socket(serversock))
return 0;
- if (backlog <= 0)
- backlog = 10;
+ if (backlog <= 0)
+ backlog = SOMAXCONN;
+
return (listen(serversock, backlog) == 0);
}