Re: Core dump with net-snmp-5.8

Hi,

The issue also occurs with the following patches.

    NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3

 0001-agentx-logging-to-late-responses.patch
 0002-agentx-do-not-shut-down-all-sessions-when-one-sessio.patch


The issue occurs with the following patch (2914) too but I found
the cause of this issue.

 https://sourceforge.net/p/net-snmp/bugs/2914/
 0001-BUG2914-Agent-master-needs-to-treat-resend-as-normal.patch


With the patch 2914, netsnmp_free_delegated_cache is called
several times for the same object as follows:

 1. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_RESEND,
 2. agentx_got_response's NETSNMP_CALLBACK_OP_RESEND handler do nothing
 3. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_SEND_FAILED, 
 4. agentx_got_response's NETSNMP_CALLBACK_OP_SEND_FAILED handler calls netsnmp_free_delegated_cache,
 5. snmp_sess_close calls agentx_got_response with NETSNMP_CALLBACK_OP_TIMED_OUT,
 6. agentx_got_response's NETSNMP_CALLBACK_OP_TIMED_OUT handler calls netsnmp_free_delegated_cache
    (double free)

gdb
--------------------------------------------------------------------------
Breakpoint 2, snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5eb160, incr_retries=1) at snmp_api.c:6747
6747                rp->callback(NETSNMP_CALLBACK_OP_RESEND, sp,
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f2ec0) at agent_handler.c:929
#1  0x00007fab254d5363 in agentx_got_response (operation=<optimized out>, session=0x564eec4ad560, reqid=2, pdu=0x564eec5e3050, magic=<optimized out>)
    at mibgroup/agentx/master.c:262
#2  0x00007fab24c6b58f in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000) at snmp_api.c:6813
#3  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
#4  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
#5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 1, snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
6735                rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
#1  0x00007fab254d541a in agentx_got_response (operation=3, session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
    at mibgroup/agentx/master.c:223
#2  0x00007fab24c69325 in snmp_resend_request (slp=slp@entry=0x564eec5df000, rp=rp@entry=0x564eec5f3e50, incr_retries=1) at snmp_api.c:6735
#3  0x00007fab24c6b5db in snmp_sess_timeout (sessp=sessp@entry=0x564eec5df000) at snmp_api.c:6826
#4  0x00007fab24c6b710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000564eeb4c0f58 in receive () at snmpd.c:1347
#6  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 3, netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x564eec5f3730) at agent_handler.c:929
#1  0x00007fab254d541a in agentx_got_response (operation=2, session=0x564eec4ad560, reqid=4, pdu=0x564eec5e54a0, magic=0x564eec5f3730)
    at mibgroup/agentx/master.c:223
#2  0x00007fab24c69586 in snmp_sess_close (sessp=0x564eec5df000) at snmp_api.c:1975
#3  0x00007fab24c6afea in snmp_sess_select_info2_flags (sessp=0x0, numfds=0x7fff68db3694, fdset=0x7fff68db36b0, timeout=0x7fff68db36a0,
    block=0x7fff68db369c, flags=0) at snmp_api.c:6556
#4  0x0000564eeb4c0e95 in receive () at snmpd.c:1263
#5  0x0000564eeb4c066e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Program received signal SIGABRT, Aborted.
0x00007fab2335f93f in raise () from /lib64/libc.so.6
--------------------------------------------------------------------------


On the other hand, without the patch 2914 netsnmp_free_delegated_cache is called
several times for the same object as follows:

 1. snmp_resend_request calls agentx_got_response with NETSNMP_CALLBACK_OP_RESEND,
 2. agentx_got_response's "Unknown operation" handler calls netsnmp_free_delegated_cache,
 3. (retry) snmp_resend_request calls agentx_got_response AGAIN with NETSNMP_CALLBACK_OP_RESEND,
 4. agentx_got_response's "Unknown operation" handler calls netsnmp_free_delegated_cache
    (double free)

gdb
------------------------------------------------------
(gdb) b netsnmp_free_delegated_cache
Breakpoint 1 at 0x7f03bd437250: file agent_handler.c, line 929.
(gdb) c
Continuing.

Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
#1  0x00007f03bd44a2df in agentx_got_response (operation=6, session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=<optimized out>)
    at mibgroup/agentx/master.c:292
#2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90, rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
#3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90) at snmp_api.c:6826
#4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000558980c5df58 in receive () at snmpd.c:1347
#6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Breakpoint 1, netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
929     {
(gdb) bt
#0  netsnmp_free_delegated_cache (dcache=0x558981be9110) at agent_handler.c:929
#1  0x00007f03bd44a5ce in agentx_got_response (operation=6, session=0x558981aa1540, reqid=12, pdu=0x558981be9370, magic=0x558981be9110)
    at mibgroup/agentx/master.c:223
#2  0x00007f03bcbde0e6 in snmp_resend_request (slp=slp@entry=0x558981bd2e90, rp=rp@entry=0x558981be67a0, incr_retries=1) at snmp_api.c:6747
#3  0x00007f03bcbe05db in snmp_sess_timeout (sessp=sessp@entry=0x558981bd2e90) at snmp_api.c:6826
#4  0x00007f03bcbe0710 in snmp_timeout () at snmp_api.c:6660
#5  0x0000558980c5df58 in receive () at snmpd.c:1347
#6  0x0000558980c5d66e in main (argc=<optimized out>, argv=<optimized out>) at snmpd.c:1126
(gdb) c
Continuing.

Program received signal SIGABRT, Aborted.
------------------------------------------------------


I am not sure it's acceptable but attach a patch to fix the issue.

Regards,

Shogo Matsumoto
Fujitsu Ltd.


> The introduction of that code fixes another issue;
> "commit 56c30b11f3616ea4f0c38a21e08e78f050096020
> Author: Bill Fenner <fenner@...>
> Date:   Wed Dec 20 21:52:10 2017 +0000
>
>     NEWS: snmplib: PATCH: 1349: Fix perl/other crash against bad SNMPv3
> agent
>
>     With the patch in 1214, the snmp_api code assumed that if magic was
>     set, it was the "struct synch-state" from snmp_client.  Of course,
>     magic belongs to the caller, and the perl library uses it differently,
>     so reaching into it is verboten.  Introduce a new callback (that
>     was already introduced in 5.8) to report this "retries exceeded"
>     state, and use it in snmp_client."
>
> I think the problem is really about shutting down the agentx connection
> when one(1) response is to late. I have
> done 2 patches (one that only write a better log message and one that
> removes the "bad" code.
> With these patches I don't get any crash. I think that 5.7.3 has this issue
> as well, but it can not be crashed with the agentofdead code
>
> Can you please try this?
>
> Regards
> Anders Wallin

_______________________________________________
Net-snmp-coders mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/net-snmp-coders

diff --git a/agent/mibgroup/agentx/master.c b/agent/mibgroup/agentx/master.c
index 99c4123ed..d0f24a5e6 100644
--- a/agent/mibgroup/agentx/master.c
+++ b/agent/mibgroup/agentx/master.c
@@ -286,6 +286,9 @@ agentx_got_response(int operation,
          */
         CLEAR_SNMP_STRIKE_FLAGS(session->flags);
         break;
+    case NETSNMP_CALLBACK_OP_RESEND:
+        DEBUGMSGTL(("agentx/master", "callback resend\n"));
+        return 0;
     default:
         snmp_log(LOG_ERR, "Unknown operation %d in agentx_got_response\n",
                  operation);
diff --git a/snmplib/snmp_api.c b/snmplib/snmp_api.c
index 554767a83..087ba2e56 100644
--- a/snmplib/snmp_api.c
+++ b/snmplib/snmp_api.c
@@ -6731,9 +6731,6 @@ snmp_resend_request(struct session_list *slp, netsnmp_request_list *rp,
         sp->s_snmp_errno = SNMPERR_BAD_SENDTO;
         sp->s_errno = errno;
         snmp_set_detail(strerror(errno));
-        if (rp->callback)
-            rp->callback(NETSNMP_CALLBACK_OP_SEND_FAILED, sp,
-                         rp->pdu->reqid, rp->pdu, rp->cb_data);
         return -1;
     } else {
         netsnmp_get_monotonic_clock(&now);