Re: access log format

Dear Al,

I believe I have a proposal for both issues.

1) If this is taken by maintree kannel, no such index will be necessary 
(read point 2). Maintenance of such code would be taken with core kannel, 
and any broken compatibilities would be handled as bugs.

2) Assuming that access log info is interesting enough to people, to be 
indexed in a DB, and giving the ability to do anything with it, from 
statistics to data mining, the proper architecture to follow would not be an 
external parser. Instead we should give DB capability to bb_alog.c. As each 
entry is recorded to the log file, it could also be inserted in the DB. DLRs 
could be matched to be inserted in the same row as the original SMS.

I propose 2 additional configuration variables in core group:

access-db
access-db-fields

and 1 single group:

access-db

access-db would be boolean, default false. access-db-fields would be a comma 
seperated list taking the usual access log variables (%A,%l, %P, etc.), and 
access-db group would match database, table and columns (to be decided). It 
would use the usual connection groups. If no connection group for the db 
specified in access-db group exists, db logging is ignored. Same if 
access-db variable is set and no access-db group exists.

It should be on a seperate thread, with rest of logging, so as not to delay 
main processing.

Any thoughts?

PS: I could work on it.

BR,
Nikos

----- Original Message ----- 
From: "Alvaro Cornejo" <[email protected]>
To: "Alok Vaidya" <[email protected]>
Cc: "Kannel Devel" <[email protected]>
Sent: Monday, September 21, 2009 4:28 PM
Subject: Re: access log format


Hi List

Wouldn't it be possible to use an "index"  for this messages that will
remain along updates/modifications, the text can change/be updated in
the future, but the "index" or message number will remain. ie message
"Message Sent" can have id 1 and if in the future, the message is
changed to "MT Sent" it will still be id=1.  The text can be something
like "(1)-Message Sent"  and after modification "(1) MT Sent"
therefore, if the message is changed or a new message is added,  the
parser will recongnize it. The list of message Id asigned can be kept
hardcoded into bb comments, no need to add aditional file/code

Also what Nikos said about having a log parser officialy in code would
be great, but then, what to do with extratrated info will be the
problem: .. count events, insert into X,Y, or Z database, fwd to a
script.... options are endless. A basic parser that will allow people
to do basic parsing and a start point for own devs would be great.

Regards

Alvaro

|-----------------------------------------------------------------------------------------------------------------|
EnvΞ½e y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier
celular y Nextel
en el PerΟ, MΞΉxico y en mas de 180 paises. Use aplicaciones 2 vias via
SMS y GPRS online
              Visitenos en www.perusms.NET www.smsglobal.com.mx y
www.pravcom.com



On Mon, Sep 21, 2009 at 1:16 AM, Alok Vaidya <[email protected]> wrote:
> Hi Alejandro,
>
> No, I guess you misunderstood me, I not asking to change any formats. I 
> just
> wanted to know the changed format more closely so I can modify our parsing
> scripts accordingly, I guess Nikos got it and he has provided the format.
>
> Alejandro Guerrieri wrote:
>
> What's the issue? Sorry after so many garbage on the thread I still not 
> see
> what's the point. Logs are fine and any changes made on the format are
> documented on ChangeLog.
> If your software croaks with the new format, either define a custom log
> format or change your software to use the new one. You don't expect that
> everybody else should accommodate to your particular requirements right?
> Regards,
> --
> Alejandro Guerrieri
> [email protected]
>
>
> On 19/09/2009, at 15:19, Alok Vaidya wrote:
>
> Hi,
>
> Just that we concentrate towards the main issue now as all other things 
> have
> been settled.
>
> Alejandro Guerrieri wrote:
>
> What do you mean with "please solve the format issue"?
> --
> Alejandro Guerrieri
> [email protected]
>
>
> On 19/09/2009, at 15:02, Alok Vaidya wrote:
>
> Hi Nii,
>
> I now get the exact meaning of what Nikos was trying to say, but it looked 
> a
> bit sarcastic then, now after your detailed explanation I can see why it 
> is
> not. Thanks for clearing the haze. Thank you very much.
>
> Also since I can now see the things clear, let me sincerely apologize 
> Nikos,
> for taking him wrong. Sorry Nikos, but it then so cleanly felt just the
> opposite. Anyways I got it now. Sorry for whatever wrong I might have said
> to you.
>
> And also as suggested please solve the format issue.
>
> Nii Ako Ampa-Sowa wrote:
>
> Alok,
>
> Nikos' comment was far from sarcastic. Earlier in the thread, Guillaume
> Cottenceau suggested that you make use of Kannel's DLR mechanism for your
> needs instead of parsing DLR logs.
>
> I think Nikos response was more in reference to that suggestion; he simply
> explained why it might be necessary for someone to parse log files instead
> of relying on Kannel's DLR mechanism. I happen to agree with his opinion
> that parsing the log files is superior for some scenarios. I actually 
> parse
> bearerbox access logs myself on a daily basis for reporting. Others may 
> not
> need this, but I find it more appropriate for my requirements.
>
> His final statement finally just states that if someone has committed an
> update which somehow changed the format, it must not have been deliberate.
>
> There's no taunting going on here. He was really just trying to help :)
>
> Nii.
>
> On Sat, Sep 19, 2009 at 9:04 AM, Alok Vaidya <[email protected]> wrote:
>>
>> Hi Donald,
>>
>> I respect your opinion here and the explanation that you offer. As 
>> someone
>> who follows the mailing list I do know Nikos as being someone who replies
>> frequently to the list, but can you explain me the comment he offered 
>> there.
>> Did you think that was helpful ? Even remotely ? That was sarcastic 
>> wasn't
>> it ? Now I am game for some light jokes as I am also for any opinion or
>> discussion one has to offer once I put my query in the public domain by
>> mailing it to a list, and yes I am certainly not here to prove my 
>> knowledge
>> or to prove any else for that matter. When anyone asks on a mailing list
>> that is because he is looking for some information beyond his knowledge 
>> and
>> for that I am open to any back and forth discussions with anyone on the 
>> list
>> which as you might have observed I was doing till (and even after) Nikos
>> intervened. But if someone hurls unnecessary, unrequired and uncalled for
>> taunts, then I am certainly not going to take that. And I beg to differ 
>> and
>> I am sorry for my strong language here but I didn't at all think that
>> through his comments Nikos was offering any help worth even a dime. Even
>> after that I tried my best explaining my situation without insulting him
>> anywhere. Also contrary to what you think I never disrespected Nikos or 
>> even
>> intended to rather it was quite the opposite. But in the mail after that 
>> he
>> came up with something that to me was a blatant lie, mentioning he never
>> ever mentioned me, which if you go through his remarks, you can very well
>> see for yourself.
>>
>> I would just want to close this topic down here as I do not want to blast
>> the mailing list with off topic posts, but my original question still
>> remains unanswered.
>>
>> Donald Jackson wrote:
>>
>> Hi Alok,
>> I think there has been a miscommunication or misunderstanding in the
>> dialog here. Alok, I don't think Nikos was trying to insult you.
>> Nikos is one of the most active and helpful members of the Kannel mailing
>> lists and should be treated with some respect, as he was after all trying 
>> to
>> help you.
>> If you want to continue this debate, please do so off the mailing list
>> (mail Nikos directly) as it has no relevance here.
>> Thanks,
>> Donald
>>
>> 2009/9/19 Alok Vaidya <[email protected]>
>>>
>>> Hi Nikos,
>>>
>>> I guess whom you are addressing then when you say "Alok's way is
>>> superoir....". I always get the facts straight and never comment on 
>>> anything
>>> before making sure, unlike you.
>>>
>>> Nikos Balkanas wrote:
>>>
>>> Hi,
>>>
>>> You are mistaken. I am not taunting you, not even addressing you. Please
>>> get your facts straight before you mail insults.
>>>
>>> Nikos
>>>
>>> ----- Original Message -----
>>> From: Alok Vaidya
>>> To: Nikos Balkanas
>>> Cc: Guillaume Cottenceau ; [email protected]
>>> Sent: Friday, September 18, 2009 3:47 PM
>>> Subject: Re: access log format
>>> Hi Nikos,
>>>
>>> I ain't sure where you are taking this, but I guess I know, so if you 
>>> are
>>> hurling taunts about anything I suggest you find someone else, as you 
>>> have
>>> completely misunderstood my requirement and my question here in the 
>>> first
>>> place.
>>> I ain't interfering with Kannel's DLR handling at all we have been using
>>> kannel for 3 years now and I guess that much time is enough to know 
>>> about
>>> any software what it does good or bad. I never said we were talking 
>>> about
>>> DLR's god knows from where you got that notion.
>>> We do not use the smsbox but a proprietary smppbox (licensed from Stipe
>>> Tolj) and we parse the log records whether MT or MO to get the 
>>> information
>>> about messages and their delivery reports from log files to db.
>>>
>>> Yes I am aware about the dlr-url feature of smsbox to get dlr HTTP 
>>> Posted
>>> back to me, but as mentioned above we do it other way.
>>>
>>>
>>> Nikos Balkanas wrote:
>>>
>>> Hi,
>>>
>>> Alok's way is superior to kannel's DLR handling. It inserts to the DB
>>> much more information about the original message than can be obtained
>>> through the DLRs alone and passed on to the dlr-url.
>>>
>>> If anyone has reduced the resolution of the logs, I am sure it was
>>> unintentional.
>>>
>>> BR,
>>> Nikos
>>> ----- Original Message ----- From: "Guillaume Cottenceau" <[email protected]>
>>> To: "Alok Vaidya" <[email protected]>
>>> Cc: <[email protected]>
>>> Sent: Friday, September 18, 2009 2:59 PM
>>> Subject: Re: access log format
>>>
>>>
>>> Alok Vaidya <alok 'at' routesms.com> writes:
>>>
>>> Hi,
>>>
>>> Sorry I didn't get you.
>>>
>>> Are you programmatically looking for sent/failed SMS by looking
>>> at Kannel logs?
>>>
>>> Guillaume Cottenceau wrote:
>>>
>>> Alok Vaidya <alok 'at' routesms.com> writes:
>>>
>>>
>>>
>>> Hi,
>>>
>>> Thanks for replying. Yes I checked the documentation we can
>>> certainly have a
>>> custom format, but it is about the events that I talking here
>>> like "Sent SMS"/
>>> "FAILED SMS"/"Discarded SMS" etc., I see some changes there.
>>> Mostly I have
>>> figured them out but, I fail to understand some other entries
>>> which I have
>>> mentioned below as also I want to know more such changes.
>>>
>>> Let me elaborate why this causes problems. Our parser uses
>>> patterns such as
>>> "Failed SMS" to search for current (14.2 box) entries, now this
>>> event logging
>>> has changed to "FAILED SEND SMS" naturally this will not be
>>> picked up and hence
>>> we loose on such entries. That's my point here.
>>>
>>>
>>> I might be a little off topic, but why not using the mechanism
>>> made for that kind of things, e.g. receiving delivery reports
>>> from Kannel? It is supported, lets you ask what information you
>>> need, and is of course "supported" accross versions.
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Guillaume Cottenceau
>>>
>>>
>>>
>>>
>>
>>
>>
>> --
>> Donald Jackson
>> http://www.thearchitech.com
>> donald(a)thearchitech.com
>
> <alok.vcf>
>
> <alok.vcf>
>