Seems that I found bugs related to stash_cashe in 5.7...

Hi all,

 

            I'll be very brief. I'm a newbie in NET-SNMP, so I'm
informing you about problematic issues I found in the package that it
may be bugs):

injectHandler stash_cache TABLE table_iterator

 

            doesn't work at all in 5.7. After digging in code, I found
the following:

1)       That netsnmp_init_stash_cache_helper  function needed for
snmpd.conf parser is not called at all because
NETSNMP_FEATURE_HAS_STASH_CACHE  is not defined anywhere (in
all_helpers.c line 34):

#ifdef NETSNMP_FEATURE_HAS_STASH_CACHE

            netsnmp_init_stash_cache_helper();

   #endif /* NETSNMP_FEATURE_HAS_STASH_CACHE */

On other hands other places that stash_handler code was used it is
defined as #ifndef NETSNMP_FEATURE_REMOVE_STASH_CACHE

This causes that stash code is defined partially ( maybe it is
intentional?)

2)       After fixing issue 1 above, I found that only cache_handler was
called ("stash_cache" was not), and as my understanding was caused
because 

_clone_handler is wronly called in agent_handler.c lines 1176/1185,
(_clone_handler does not create a deep cloning, including next handlers

netsnmp_inject_handler_before(tptr->reginfo, _clone_handler(handler)
/*<=wrong?*/  ,

                                          before_what);

I substituted by: netsnmp_inject_handler_before(tptr->reginfo,
netsnmp_handler_dup(handler),

                                          before_what);

3)       It still did not worked,  I found that cache were being
collected, but not used, so I found in stash_cache line 140:

cache = netsnmp_cache_reqinfo_extract( reqinfo, reginfo->handlerName );
But the handler name is not the cache name, so I substituted with code

I found in other place:

    char addrstr[32];

    snprintf(addrstr,sizeof(addrstr), "%ld", (long
int)handler->prev->myvoid);

cache = netsnmp_cache_reqinfo_extract( reqinfo, addrstr );

4)       After all these fixes, now I see stash cache working partially
( I mean for some SNMP collections it works, but for others it crashes
with:

*** glibc detected *** double free or corruption (out): 0x0fbfc180 ***

Thread [1] (Suspended: Signal 'SIGABRT' received. Description: Aborted.)


            25 raise()  0x0fae9144    

            24 abort()  0x0faea8bc    

            23 __libc_message()  0x0fb20630            

            22 malloc_printerr()  0x0fb2be88  

            21 _int_free()  0x0fb2a964           

            20 free()  0x0fb292f8       

            19 snmp_free_var_internals() snmp_api.c:5049 0x0fc8271c


            18 snmp_free_var() snmp_api.c:5065 0x0fc8281c  

            17 snmp_free_varbind() snmp_api.c:5075 0x0fc82870        

            16 table_data_free_func() table.c:1100 0x0ff96288 

            15 netsnmp_free_list_data() data_list.c:33 0x0fcb2cc0      

            14 netsnmp_free_all_list_data() data_list.c:45 0x0fcb2d28 

            13 netsnmp_free_request_data_sets() agent_handler.c:1034
0x0ffa6744     

            12 free_agent_snmp_session() snmp_agent.c:1441 0x0ffb4ca4


            11 netsnmp_remove_and_free_agent_snmp_session()
snmp_agent.c:1860 0x0ffb6104         

            10 netsnmp_wrap_up_request() snmp_agent.c:1818 0x0ffb599c


            9 netsnmp_handle_request() snmp_agent.c:3307 0x0ffb9728


            8 handle_snmp_packet() snmp_agent.c:1987 0x0ffb6504   

            7 _sess_process_packet() snmp_api.c:5407 0x0fc835e4   

            6 _sess_read() snmp_api.c:5836 0x0fc84aa4       

            5 snmp_sess_read2() snmp_api.c:5868 0x0fc84bc0          

            4 snmp_read2() snmp_api.c:5470 0x0fc837e8      

            3 snmp_read() snmp_api.c:5460 0x0fc8377c        

            2 agent_check_and_process() snmp_agent.c:675 0x0ffb2f40


            1 main() subagentmain.c:311 0x10002e94            

 

For this I have no solution yet...  Right now I left stash_cache ...

 

Ivo

 

 

 

 

 

________________________________

From: Ivo Faldini 
Sent: Monday, July 18, 2011 1:20 PM
To: '[email protected]'
Subject: Looking for help: I'm having problems with performance.
Stash_cashe is not working for me...

 

Hi all,

            I have the following scenario:

- For a standalone snmp MASTER agent (Note: in order to troubleshoot the
problem explained below, I'm NOT using snmpd at all - no agentX neither
proxy)  written in with net-snmp (ver 5.7 cross-compiled for PPC
processor) I have some snmp tables containing several lines (dynamic
sized: using rowStatus). The data to fill these tables is acquired
through IPC (for example: UDP socket msg sent from SNMP agent to an
server app that returns back data to agent). In other words: when the
agent receive a snmp get request, the request is forwarded to the server
app that returns back the data that is line based to the agent).

- MIB code was implemented upon
mib2c.iterate_access.conf/mib2c.access_functions.conf mib2c methodology
(exceptionally mib2c used to compile MIBs was from net-snmp 5.4.2.1, but
I don't think that it makes any significant difference).

 

            Right now I realized that I have a very hard performance
problem:

- For example when one of the tables contains 130 lines for example, it
takes a very long time to collect data. This is caused due to the fact
that for every get-next, the whole table is collected from server app
(so to collect 130 lines, I'm having 130*130 IPC calls). If I'll have
1000 lines it will be almost impossible to collect data for this table!!

I tried to find a solution for this problem within net-snmp framework
itself without a whole code refactoring nor re-compiling the MIBs using
another methodology (for example MFD).

If I understood correctly the net-snmp docs I think that
caching/stash_cache could be a good solution in order to alleviate these
performance problems (as stated in doc, stashed cache may be used with
table iterators). However when trying to applying it: OR I did not
understood how to implement/apply this solution NEITHER it didn't work.

I'll explain what I tried to do:

(First of all: I understood that stash cache may cache OID results (for
a limited period of time, like 30 secs) in such a way that table
collection is made only once (for 130 lines: it will cache data results,
decreasing from 130 * 130 IPC calls to only 130) .

This cache is automatically built from data collected by net-snmp stash
cache engine with no needs to add anything to the code like
*_load.../*_free... functions used in other methods like the one in
netsnmp_get_cache_handler. stash cache is good only for 30 secs. Please
correct me if I misunderstand it.)

1)       I found in documentation that stash cache may be added in
configuration file, so I've added the following line to .conf file:

injectHandler stash_cache onuCfgTable table_iterator

(where onuCfgTable is the name I gave in
netsnmp_create_handler_registration("onuCfgTable", ...) )

The table collection still work but no performance increase noted ( I
added printing withing onuCfgTable_get_first_data_point/
onuCfgTable_get_next_data_point, and I saw that still access was 130 *
130 times and even when trying to get the table again these functions
still were called 130 * 130 times).

2)       I found another way to make the same thing, but in code:

netsnmp_inject_handler( my_handler, netsnmp_get_stash_cache_handler());

This method also not only did not worked, but it was worse: now no lines
were returned at all to MIB browser (it returned empty table for first
time or it returned snmp generic error for second try).

 

(I'm attaching a code snippet in order to clarify)

////////////////////////////////////////////////////////////////////////
/////////////////////////////////////

    netsnmp_table_registration_info *table_info;

    netsnmp_handler_registration *my_handler;

    netsnmp_iterator_info *iinfo;

 

    /** create the table registration information structures */

    table_info = SNMP_MALLOC_TYPEDEF(netsnmp_table_registration_info);

    iinfo = SNMP_MALLOC_TYPEDEF(netsnmp_iterator_info);

 

    my_handler = netsnmp_create_handler_registration("onuCfgTable",

 
onuCfgTable_handler,

                                                     onuCfgTable_oid,

                                                     OID_LENGTH

                                                     (onuCfgTable_oid),

 
HANDLER_CAN_RWRITE);

 

    if (!my_handler || !table_info || !iinfo) {

        snmp_log(LOG_ERR, "malloc failed in
initialize_table_onuCfgTable");

        return; /** Serious error. */

    }

 

    /***************************************************

     * Setting up the table's definition

     */

    netsnmp_table_helper_add_indexes(table_info, ASN_UNSIGNED,

                                               /** index: XXXX */

                                     ASN_UNSIGNED,

                                               /** index: XXXX */

                                     ASN_UNSIGNED,

                                               /** index: XXXX */

                                     0);

 

    /** Define the minimum and maximum accessible columns.  This

        optimizes retrival. */

    table_info->min_column = 4;

    table_info->max_column = 61;

 

    /** iterator access routines */

    iinfo->get_first_data_point = onuCfgTable_get_first_data_point;

    iinfo->get_next_data_point = onuCfgTable_get_next_data_point;

 

    /** tie the two structures together */

    iinfo->table_reginfo = table_info;

    //iinfo->flags = NETSNMP_ITERATOR_FLAG_SORTED;

 

    /***************************************************

     * registering the table with the master agent

     */

    DEBUGMSGTL(("initialize_table_onuCfgTable",

                "Registering table onuCfgTable as a table iterator\n"));

    netsnmp_register_table_iterator(my_handler, iinfo);

    netsnmp_inject_handler( my_handler,
netsnmp_get_stash_cache_handler());

////////////////////////////////////////////////////////////////////////
/////////////////////////////////////

3)       I found  also that replacing netsnmp_inject_handler(
my_handler, netsnmp_get_stash_cache_handler()); with the following code
should also work:

      netsnmp_inject_handler( my_handler,

netsnmp_get_timed_stash_cache_handler(30, onuCfgTable_oid,
OID_LENGTH(onuCfgTable_oid)));

            But it still behaved exactly as the previous solution (empty
table + error).

 

            I would be glad if someone could help me with an advice what
on some kind of solution that may be applied in this case in order to
increase performance without a big effort nor massive code re-writing).

 

Ivo

 

 

 

 

 


NOTICE: This email and any files transmitted with it are Enablence and Teledata-Networks confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This message contains Enablence and Teledata-Networks confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

------------------------------------------------------------------------------
5 Ways to Improve & Secure Unified Communications
Unified Communications promises greater efficiencies for business. UC can 
improve internal communications as well as offer faster, more efficient ways
to interact with customers and streamline customer service. Learn more!
http://www.accelacomm.com/jaw/sfnl/114/51426253/

_______________________________________________
Net-snmp-users mailing list
[email protected]
Please see the following page to unsubscribe or change other options:
https://lists.sourceforge.net/lists/listinfo/net-snmp-users