Re: LogFormat in analog.cfg broken

Quoting Aengus <[email protected]>:

> Ulf Hofemeier <ulf-vqempv/qbUL2fBVCVOL8/[email protected]> wrote:
>> Quoting Aengus <[email protected]>:
>>
>>> Ulf Hofemeier <ulf-vqempv/qbUL2fBVCVOL8/[email protected]> wrote:
>>>> Hi folks:
>>>>
>>>> I could use some help figuring out the correct syntax for a new
>>>> LogFormat line in my analog.cfg file after changing the LogFormat
>>>> settings for Apache a few weeks ago.
>>>>
>>>> In my httpd.conf I have the following line:
>>>>
>>>> LogFormat "%h %l \"%u\" %t \"%r\" %>s %b \"%{Referer}i\"
>>>> \"%{User-Agent}i\"" combined
>>>>
>>>
>>> Have you tried just using your Apache command directly?
>>>
>>> APACHELOGFORMAT ("%h %l \"%u\" %t \"%r\" %>s %b \"%{Referer}i\"
>>> \"%{User-Agent}i\"")
>>>
>>> Aengus
>>
>>
>> Hi Aengus:
>>
>> I tried APACHELOGFORMAT in my analog.cfg without success (still
>> corrupt logfile lines).
>>
>> My Apache access_log looks like this:
>>
>> 1.2.3.4 - "-" [02/Oct/2008:09:31:33 -0600] "GET /images/black.gif
>> HTTP/1.1" 200 43 "http://www.unm.edu/" "Mozilla/5.0 (Windows; U;
>> Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3"
>> 1.2.3.4 - "-" [02/Oct/2008:09:31:33 -0600] "GET /images/white.gif
>> HTTP/1.1" 200 43 "http://www.unm.edu/" "Mozilla/5.0 (Windows; U;
>> Windows NT 5.1; en-US; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3"
>
> My mistake - I forgot to strip the double-quotes from the start and  
> end of your line from the Apache entry.
>
> APACHELOGFORMAT (%h %l \"%u\" %t \"%r\" %>s %b \"%{Referer}i\"  
> \"%{User-Agent}i\")
>
> This works for the sample lines that you posted.
>
> Aengus

I have to admit that the analog configuration is getting quite  
confusing to me. Unfortunately the APACHELOGFORMAT line doesn't solve  
my problem, so please allow me to provide you with a little more  
information regarding the purpose of the updated analog.cfg, as well  
as what I'm doing before the problem occurs.

1. Copy the previous month Apache log to a temporary location
2. Run a script to extract page visitor data from the general Apache  
log file and store it in a separate file
3. Run a bash 'for i' loop on the new log files and store the data in  
page visitor sub directores

Unfortunately I decided that Apache has to write more information to  
its access_log log file, which is finally the reason why there is  
issues with analog now. According to the analog documentation there is  
a way to set up a hierarchy so that it will understand a log file  
syntax even if it changes from old to new over time, but I haven't  
been able to figure out how to make it work.

The question is how do I let analog know that the log file syntax for  
the reports has changed over time. If it's not possible, I can  
probably incorporate an if statement in the scripts that checks for  
the date the change occurs and feeds analog a new configuration file  
that will work, but I would love to avoid that.

My old Apache log file lines look like this:

1.2.3.4 - "-" [19/Jun/2008:23:36:03 +0000] "GET /images/black.gif  
HTTP/1.1" 200 43
1.2.3.4 - "-" [19/Jun/2008:23:36:03 +0000] "GET /images/white.gif  
HTTP/1.1" 200 43
1.2.3.4 - "-" [19/Jun/2008:23:36:03 +0000] "GET /images/headlines.gif  
HTTP/1.1" 200 4352
1.2.3.4 - "-" [19/Jun/2008:23:36:03 +0000] "GET /images/gray.gif  
HTTP/1.1" 200 43
1.2.3.4 - "-" [19/Jun/2008:23:36:03 +0000] "GET /images/red.gif HTTP/1.1" 200
45

My new Apache log file line look like this:

1.2.3.4 - "-" [10/Sep/2008:15:12:11 -0600] "GET /images/header.jpg  
HTTP/1.1" 200 38750 "http://ladb.unm.edu/" "Mozilla/5.0 (Macintosh; U;  
PPC Mac OS
X Mach-O; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16"
1.2.3.4 - "-" [10/Sep/2008:15:12:12 -0600] "GET /images/black.gif  
HTTP/1.1" 200 43 "http://ladb.unm.edu/" "Mozilla/5.0 (Macintosh; U;  
PPC Mac OS X Ma
ch-O; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16"
1.2.3.4 - "-" [10/Sep/2008:15:12:12 -0600] "GET /images/white.gif  
HTTP/1.1" 200 43 "http://ladb.unm.edu/" "Mozilla/5.0 (Macintosh; U;  
PPC Mac OS X Ma
ch-O; en-US; rv:1.8.1.16) Gecko/20080702 Firefox/2.0.0.16"

My analog.cfg looks like this:

APACHELOGFORMAT (%S %l \"%u\" %t \"%r\" %>s %b \"%{Referer}i\"  
\"%{User-Agent}i\")

DEFAULTLOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%j %r %j" %c %b)
DEFAULTLOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%j %r %j" %c %b)
DEFAULTLOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%j %r" %c %b)
DEFAULTLOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%r" %c %b)

LOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%j %r %j" %c %b)
LOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%j %r" %c %b)
LOGFORMAT (%S %j "%u" [%d/%M/%Y:%h:%n:%j] "%r" %c %b)
LOGFORMAT (%S %l \"%u\" %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\")

DEBUG ON

LOGFILE log-????-??.gz

# OUTFILE Report.html

REQINCLUDE pages


Thank you.

+------------------------------------------------------------------------
|  TO UNSUBSCRIBE from this list:
|    http://lists.meer.net/mailman/listinfo/analog-help
|
|  Analog Documentation: http://analog.cx/docs/Readme.html
|  List archives:  http://www.analog.cx/docs/mailing.html#listarchives
|  Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------