Re: LogFormat in analog.cfg broken

Quoting Aengus <[email protected]>:

> Ulf Hofemeier <ulf-vqempv/qbUL2fBVCVOL8/[email protected]> wrote:
>
>> I have to admit that the analog configuration is getting quite
>> confusing to me. Unfortunately the APACHELOGFORMAT line doesn't solve
>> my problem, so please allow me to provide you with a little more
>> information regarding the purpose of the updated analog.cfg, as well
>> as what I'm doing before the problem occurs.
>>
>> 1. Copy the previous month Apache log to a temporary location
>> 2. Run a script to extract page visitor data from the general Apache
>> log file and store it in a separate file
>> 3. Run a bash 'for i' loop on the new log files and store the data in
>> page visitor sub directores
>>
>> Unfortunately I decided that Apache has to write more information to
>> its access_log log file, which is finally the reason why there is
>> issues with analog now. According to the analog documentation there is
>> a way to set up a hierarchy so that it will understand a log file
>> syntax even if it changes from old to new over time, but I haven't
>> been able to figure out how to make it work.
>
> If you have multiple LOGFORMAT statements, Analog will try them each  
> in turn until it finds one that matches the entries in each of your  
> logiles. That means that if you have multiple logfiles, and they  
> aren't all the same format, Analog can still create a single report  
> from these different logfiles. Obviously the report may understate  
> this items that weren't recorded in some of the logfiles - for  
> example, you might have a million requests, but only only 200,000  
> Browser strings if you only added that field in leater log files.
>
> LOGFORMAT commands apply to LOGFILEs that are specified after the  
> LOGFORMAT in the .cfg file. DEFAULTLOGFORMAT commands apply to  
> logfiles that are specified on the command line.
>
> It's not clear from your description whether your script calls  
> Analog and passes it the name of the logfile as a paramter, or  
> whether Analog picks up the logfile from the LOGFILE log-????-??.gz  
> statement in your .cfg file.
>
> If you're speciying the LOGFILES in the .cfg file, then these lines  
> should do the job:
> APACHELOGFORMAT (%h %l \"%u\" %t \"%r\" %>s %b)
> APACHELOGFORMAT (%h %l \"%u\" %t \"%r\" %>s %b \"%{Referer}i\"  
> \"%{User-Agent}i\")

My script calls analog like this:

cd $datadir/$domain

# Determine the range of months from the list of log files that were  
not empty.
first=`ls log*|sort|head -1|cut -b5-`
first=`echo $first |rev |cut -b4- |rev` ;          # YEAR-MO
last=`ls access_log*|sort|tail -1|cut -b12-`
last=`echo $last |rev |cut -b4- |rev` ;            # YEAR-MO
range="$first--$last";                             # YEAR-MO--YEAR-MO

# Collect summary information from all the log files.
/data/stats/analog/analog access_log.????-??.gz >  
$analogdir/$domain/$range.html

# Collect information by month in seperate files.
for i in access_log.????-??.gz ;
do
     file=`echo $i |cut -b12-`                       # YEAR-MO.gz
     file=`echo $file |rev |cut -b4- |rev` ;        # YEAR-MO
     /data/stats/analog/analog $i > $analogdir/$domain/$file.html
done

So I pass the log file to analog as a parameter on the command line  
rather than using analog.cfg.

> If you're calling Analog with the logfiles specified on the command  
> line, then these lines should work:
> DEFAULTAPACHELOGFORMAT (%h %l \"%u\" %t \"%r\" %>s %b)
> DEFAULTAPACHELOGFORMAT (%h %l \"%u\" %t \"%r\" %>s %b  
> \"%{Referer}i\" \"%{User-Agent}i\")
>

I will give these two lines a try in my analog.cfg.

Thank you.

+------------------------------------------------------------------------
|  TO UNSUBSCRIBE from this list:
|    http://lists.meer.net/mailman/listinfo/analog-help
|
|  Analog Documentation: http://analog.cx/docs/Readme.html
|  List archives:  http://www.analog.cx/docs/mailing.html#listarchives
|  Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------