Re: [Myrinet] gm_alarm /send failure

In my stuff whenever I do any communication I have a loop where I call 
gm_receive, grab the event (interpreting GM_NO_RECV_EVENT as a null event) 
until I'm happy with the result.  If I did a send I wait for 
GM_SENT_EVENT, say.  One can obviously put timing in such a loop.  In fact 
I keep track of the number of times I see GM_NO_RECV_EVENT.

On Wed, 25 Sep 2002, Gabor Forgacs wrote:

> Hi All
> 
> I would like to figure out some robust/fast error handling for send
> failures.
> All it should do to ignore the erronous sending and try to send the next
> buffer.
> The basic method i understand is to handle the errors in the send callback
> and call the gm_drop_sends in the callback. It works fine but the send
> timeout is 30sec or like
> it is too long. I would like  only 1/2 sec delays. Can I somehow set this
> value up?
> 
> The other way i see is to setup an alarm and from the callback or in the
> event loop to call the
> gm_drop_sends. It just doesn't seem to work for me,the message i get:
> 
> //////////////////////////////////////////////////
> [send] Opened board 0 port 2
> [send] 0-th Sending Buffer
> [send] Alarm Callback called
> [send] Alarm Event
> [send] Drop Callback called
> [send] 1-th Sending Buffer
> Program passed bad resend command to GM.
> Port 2 disabled.
> PANIC: libgm/gm_unknown.c:301:???():userland
> 
> WARNING:
> Program aborted.
> //////////////////////////////////////////////
> 
> I suppouse , that is my fault but just don't know how to solve.
> Is the second method is possible at all ?
> 
> Here the code:
> 
> // TestMyrinet.cpp : Defines the entry point for the console application.
> //
> #include "stdafx.h"
> #define GM_PORT_NUM 2
> #define GM_BUFFER_LENGTH (12*1024*1024)
> #define GM_RECEIVER_NODE_ID 6
> #define GM_SEND_TIMEOUT 1000000
> 
> void my_drop_callback (struct gm_port *port, void *context, gm_status_t
> the_status)
> {
>     (*(int *)context)--;
>     gm_printf ("[send] Drop Callback called\n");
> }
> 
> void my_alarm_callback (void *context)
> {
>     // (*(int *)context)--;
>     gm_printf ("[send] Alarm Callback called\n");
> }
> 
> void my_send_callback (struct gm_port *port, void *context, gm_status_t
> the_status)
> {
>     if (the_status != GM_SUCCESS)
>     {
>         if (the_status != GM_SEND_DROPPED)
>         {
>             gm_perror ("send completed with error", the_status);
> 
> gm_drop_sends(port,GM_LOW_PRIORITY,GM_RECEIVER_NODE_ID,GM_PORT_NUM,my_drop_c
> allback,context);
>         } else {
>             gm_printf ("[send] GM_SEND_DROPPED status in send callback \n");
>         }
>     } else {
>         (*(int *)context)--;
>     }
> }
> 
> int main(int argc, char* argv[])
> {
> int i;
> int my_board_num = 0;
> void *out_buffer = 0;
> gm_alarm_t *gm_alarm;
> gm_recv_event_t *event;
> int expected_callbacks = 0;
> gm_status_t main_status;
> static struct gm_port *my_port = NULL;
> 
>     main_status = gm_open (&my_port, my_board_num, GM_PORT_NUM,
>     "test_send",(enum gm_api_version) GM_API_VERSION_1_1);
>     if (main_status == GM_SUCCESS)
>     {
>         gm_printf ("[send] Opened board %d port %d\n",my_board_num,
> GM_PORT_NUM);
>     }
>     else
>     {
>         gm_perror ("[send] Couldn't open GM port", main_status);
>         goto abort_with_nothing;
>     }
> 
>     out_buffer = gm_dma_calloc (my_port, 1,GM_BUFFER_LENGTH);
>     if (out_buffer == 0)
>     {
>         gm_printf ("[send] Couldn't allocate out_buffer\n");
>         goto abort_with_open_port;
>     }
> 
>     gm_alarm = new gm_alarm_t;
>     gm_initialize_alarm (gm_alarm);
> 
>     // Send Loop
>     for(i=0;i<100;i++)
>     {
>         gm_printf ("[send] %d-th Sending Buffer\n",i);
> 
> gm_set_alarm(my_port,gm_alarm,GM_SEND_TIMEOUT,my_alarm_callback,&expected_ca
> llbacks);
> 
> gm_send_to_peer_with_callback(my_port,out_buffer,gm_min_size_for_length(GM_B
> UFFER_LENGTH),
> 
> GM_BUFFER_LENGTH,GM_LOW_PRIORITY,GM_RECEIVER_NODE_ID,my_send_callback,&expec
> ted_callbacks);
>         expected_callbacks++;
> 
>         /* Now we wait for the callbacks for the sends we did above */
>         while (expected_callbacks)
>         {
>             event = gm_receive (my_port);
>             switch (gm_ntoh_u8 (event->recv.type))
>             {
>             case GM_RECV_EVENT:
>             case GM_PEER_RECV_EVENT:
>             case GM_FAST_PEER_RECV_EVENT:
>                 gm_printf ("[send] Receive Event (UNEXPECTED)\n");
>                 break;
>             case GM_NO_RECV_EVENT:
>                 break;
>             case GM_SEND_DROPPED:
>                 gm_printf ("[send] Drop Sends Event\n");
>                 break;
>             case GM_ALARM_EVENT:
>                     gm_unknown (my_port, event);
>                     gm_printf ("[send] Alarm Event\n");
> 
> gm_drop_sends(my_port,GM_LOW_PRIORITY,GM_RECEIVER_NODE_ID,GM_PORT_NUM,my_dro
> p_callback,&expected_callbacks);
>                     break;
>             default:
>                 gm_unknown (my_port, event); /* gm_unknown calls the
> callback */
>         }
>        }
>     }
> 
>     gm_dma_free (my_port, out_buffer);
>     gm_close (my_port);
>     gm_exit (GM_SUCCESS);
> 
>     abort_with_out_buffer:
>     gm_dma_free (my_port, out_buffer);
> 
>     abort_with_open_port:
>     gm_close (my_port);
> 
>     delete gm_alarm;
>     gm_alarm=NULL;
> 
>     abort_with_nothing:
>     gm_finalize();
>     gm_exit (main_status);
> }
> 
> It should be a test to send a 100 times to a non responsive host  without
> hanging and delays.
> Thanx in advance
> 
> 
> _______________________________________________
> Myrinet mailing list
> [email protected]
> http://email.osc.edu/mailman/listinfo/myrinet
> 

-- 
William K. Allard
Professor of Mathematics
Duke University 
Box 90320
Durham, NC 27708-0320
(919) 660-2861 Fax:(919) 660-2821