Re: removeheaders

MrC wrote:
> Tom Metro wrote:
>> Mike Cappella wrote:
--
> 
> Still, I think it makes sense to focus on large performance areas..  I 
> know how to shave off minutes!  [ :-) Sorry Mike, I think 2 minutes is 
> too long!  The impact on moderate systems can be noticeable [eg. when 
> the large IO xfers cause SA timeouts, etc]

:) Funny thing is I just started getting this problem on a machine 
[P3-733mhz] after the upgrade to SA 3.2.x. [Not due to Logwatch of 
course but understand the point.] So I'm all for any improvements and 
I've got several large Mail-Gateways with big fat log files that can be 
used to test on.

>>
>> Of course now that we've optimized this, I'd like to voice my 
>> agreement with a comment made recently that dates and other "header" 
>> information should probably not get stripped off before handing the 
>> data to the service filter. As I noted in another message, having a 
>> common intermediate log format would permit building an infrastructure 
>> for doing common things, like splitting a line into a standard set of 
>> components (date, host, service, PID, message), making the job of the 
>> service filter just as easy, wile still giving it access to the 
>> greater detail if it needs it.
> 
> Great!  Let's see what we can come up with.  I'd like to see:
> 
>  1) standard date (syslog logs don't include year, vsftpd does, and 
> clamd host no time stamps at all.)
>  2) hostname
>  3) service w/PID (amavis has some non-standard entries, essentially 2 
> ids').
>  4) log line remainder
> 
>

Obviously anything under "syslog" standard is easy, its all the other 
stuff which has become on of Logwatch's strengths that is going to be a 
problem.

Random Thoughts:
Probably the easiest way to start would be to knock of the syslog 
standard and add a new config variable to the logfile group conf that 
tells it to validate using "syslog" then add other validations as they 
get worked out.

Check out the Cron applydate to remind us of the horrors lurking in the 
system. ;) [Solaris cron logging does not go through syslog and the 
format is somehwat odd.]

Samba and Apache [http] are pretty big services that are also not Syslog 
formatted. These should provide you with pretty examples of the what it 
is going to take.

-Mike