Re: Shared Memory, semaphores and a new "box"

On 03.03.2008, at 01:45, Alejandro Guerrieri wrote:

> Dear List,
>
> I'm trying to develop a new "box" that I'm planning to contribute it  
> GPL'ed to the project when it's ready. My goal is a box that reads  
> MT messages from a shared memory segment and injects them into  
> kannel. This way, a number of interfaces could be easily done that  
> write to that shared memory segment to send messages.

Please note that kannel is not under GPL but under its own license  
which allows to do more than GPL. So integrating it into the core  
product would not work.

Also whats the purpose of this? It won't work under unixes and most  
probably not under windows 2000 upwards neither as memory is protected  
between processes or in other words, they don't share common memory.  
Otherwise you would have to do it in the kernel which is pretty much  
hard core. Much easier to use a socket in such a case as we are not  
transporting gigabytes of memory. And HTTP is one such socket. I have  
not found an performance bottlenecks there yet.

> I'm pretty close to make it work, and I even developed a PHP module  
> that implements the function "kannel_send_sms()" on PHP. I've called  
> the box "phpbox", but I'll probably rename it to "membox", since it  
> can be used with any language, as long as it writes messages to the  
> shm.

Well... I think your problem is somewhere else.

> The performance improvement over the HTTP interface and even over  
> sqlbox are impressive (though it's only for MT, of course). It makes  
> sense, since all negotiation is done on memory, so it could be a  
> very fast method to inject MT messages.


How many 10'000 messages per second do you need to transport that you  
have bottlenecks like that?

> I've done most of the job, I've borrowed the structure from sqlbox- 
> standalone (Renee and Martin's ork, credited accordingly) and I've  
> mostly succeeded in developing the box and php interface, but after  
> I've started testing it with thousands of messages, I've discovered  
> a race condition, probably having to do with semaphores.
>
> I'm copying relevant code at the bottom. It's two pieces of code:  
> the server (aka phpbox, runs as a box) and the client (a test  
> program written in C that enqueues 100K messages).
>
> Everything works fine so far, but when I try running two instances  
> of the client program (and sometimes with a single instance), I'm  
> starting getting the same message (in fact, it's the last message  
> that replaces all the previous).
>
> To be clear, the client program does:
>
>     for(c=1;c<=100000;c++) {
>       asprintf(&ms, "Hello %d", c);
>       send_sms(ms);
>     }
>
> So I should receive:
>
> Hello 1
> Hello 2
> ...
> Hello 100000
>
> But instead, I receive:
>
> Hello 1
> Hello 2
> ...
> Hello 23456
> Hello 100000
> Hello 100000
> ...
> Hello 100000
> Hello 100000
>
> No messages are lost, but somewhat the shared memory area is being  
> overwritten before it's being read, I suppose.

Well thats why there is locking...

>
>
> I'm a total newby at semaphores, and I've borrowed the semaphore  
> code from a few examples floating on the net, so I suppose I'm doing  
> something wrong there.
>

Semaphores are to sync inside a process in multiple threads. They dont  
work between multiple processes. You would need a file locking  
mechanism or the like.


> Could someone take a look at the code and tell me if there's  
> something wrong, or how to rewrite it to use gwlib's semaphores? If  
> you need the full (flawed) source code I'll gladly send it.

I would say the whole design is wrong. Shared memory is different in  
most operating systems and frankly I don't think your real problem is  
there. Maybe you share the real issue you are facing?






Andreas Fink

Fink Consulting GmbH
Global Networks Schweiz AG
BebbiCell AG

---------------------------------------------------------------
Tel: +41-61-6666330 Fax: +41-61-6666331  Mobile: +41-79-2457333
Address: Clarastrasse 3, 4058 Basel, Switzerland
E-Mail:  [email protected]
www.finkconsulting.com www.global-networks.ch www.bebbicell.ch
---------------------------------------------------------------
ICQ: 8239353 MSN: [email protected] AIM: smsrelay Skype: andreasfink
Yahoo: finkconsulting SMS: +41792457333

Say NO to Power Line Communications: http://www.youtube.com/watch?v=pdcY0Eetvsw