RE: Corrupt log entries

Jeremy Wadsack <[email protected]>
Newsgroups gmane.comp.web.analog.general
Message-ID <[email protected]> 
I don't know if it got modified in the email, but it looks like there are extra spaces around the IP address in the corrupt line. In your LOGFORMAT you might try using %w instead of a space as a separator.

--

Jeremy Wadsack
Seven Simple Machines

________________________________
From: [email protected] [mailto:[email protected]] On Behalf Of Brian Clanton
Sent: Friday, July 25, 2008 12:29 PM
To: [email protected]
Cc: [email protected]
Subject: [analog-help] Corrupt log entries


I have submitted this question to the analog forum as well as going through the documentation for about a week now and I am no closer to finding a resolution.  Any help you may be able to give would be much appreciated.

I am running an Ezrproxy server (Fedora 8) and I want Analog (Same Fedora 8 Server) to provide stats based on the Ezproxy logs.

The designated Ezproxy log format is:
LogSPU  spu.log %{ezproxy-spuaccess}i %h  %l %u %t "%r" %s %b

Here are a few logs in the "spu.log" file:
./analog: analog version 6.0/Unix
F: Closing configuration file /usr/local/analog-6.0.3/analog.cfg
F: Opening /usr/local/analog-6.0.3/lang/uk.lng as language file
F: Closing language file /usr/local/analog-6.0.3/lang/uk.lng
F: Opening /usr/local/analog-6.0.3/lang/ukdom.tab as domains file
F: Closing domains file /usr/local/analog-6.0.3/lang/ukdom.tab
F: Opening /usr/local/analog-6.0.3/lang/ukdesc.txt as report descriptions file
F: Closing report descriptions file /usr/local/analog-6.0.3/lang/ukdesc.txt
F: Opening /usr/local/analog-6.0.3/logfile.log as logfile
C: proxy 205.188.116.15 - gNDs4AexHJMk3B9 [15/Jul/2008:21:04:33 -0600] "GET http://proxy-ob.dupagels.lib.il.us:18457/login?url=http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:                      *
C: proxy 205.188.116.205 - gNDs4AexHJMk3B9 [15/Jul/2008:21:04:33 -0600] "GET http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:                       *
C: proxy  192.168.1.130  - E37HRQKiv0dANot [18/Jul/2008:14:53:48 -0600] "GET http://infoweb.newsbank.com?db=OBIT HTTP/1.1" 200 0

C:        *
C: proxy  66.158.45.2  - N5IpSRYjj6PC0q4 [18/Jul/2008:14:54:16 -0600] "GET http://search.ebscohost.com/login.aspx?authtype=ip,uid&profile=novelist HTTP/1.1" 200 0

C:        *
C: proxy  66.158.45.2  - N5IpSRYjj6PC0q4 [18/Jul/2008:14:59:29 -0600] "GET http://search.ebscohost.com/login.aspx?authtype=ip,uid&profile=novelist HTTP/1.1" 200 0

C:        *
C: proxy  66.158.45.2  - 6sjKkVq74O8W7qZ [18/Jul/2008:15:21:22 -0600] "GET http://search.ebscohost.com/login.aspx?authtype=ip,uid&profile=novelistk8 HTTP/1.1" 200 0

C:        *
C: proxy  66.158.45.2  - deRkpGPltRsoYFP [18/Jul/2008:15:48:30 -0600] "GET http://proquest.umi.com/login HTTP/1.1" 200 0

C:        *
C: proxy  205.188.116.196  - RmvHR9lpYXIJyDT [19/Jul/2008:09:03:05 -0600] "GET http://proquest.umi.com/pqdweb?SQ=%22Nick+Nitti%22&DBId=15108&onDate=&beforeDate=&fromDate=&toDate=&FT=1&AT=any&author=&sortby=CHRON&RQT=305&querySyntax=PQ&searchInterface=1&moreOptState=CLOSED&TS=1215778558&h_pubtitle=&h_pmid=&clientId=20908&JSEnabled=1 HTTP/1.1" 200 0

C:        *
C: proxy  205.188.117.15  - RmvHR9lpYXIJyDT [19/Jul/2008:09:05:57 -0600] "POST http://proquest.umi.com/pqdweb?SQ=%22Nick+Nitti%22&DBId=15108&onDate=&beforeDate=&fromDate=&toDate=&FT=1&AT=any&author=&sortby=CHRON&RQT=305&querySyntax=PQ&searchInterface=1&moreOptState=CLOSED&TS=1215778558&h_pubtitle=&h_pmid=&clientId=20908&JSEnabled=1 HTTP/1.1" 200 0

C:        *
C: proxy  205.188.117.139  - RmvHR9lpYXIJyDT [19/Jul/2008:09:06:25 -0600] "GET http://proxy-ob.dupagels.lib.il.us:18457/login?url=http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:        *
C: proxy  205.188.116.205  - RmvHR9lpYXIJyDT [19/Jul/2008:09:06:25 -0600] "GET http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:        *
C: proxy  205.188.116.13  - Cjm6To7u638jzSX [19/Jul/2008:17:28:44 -0600] "GET http://proquest.umi.com/pqdweb?SQ=Yonkee&DBId=15108&onDate=&beforeDate=&fromDate=&toDate=&FT=1&AT=any&author=&sortby=CHRON&RQT=305&querySyntax=PQ&searchInterface=1&moreOptState=CLOSED&TS=1215778558&h_pubtitle=&h_pmid=&clientId=20908&JSEnabled=1 HTTP/1.1" 200 0

C:        *
C: proxy  205.188.117.139  - Cjm6To7u638jzSX [19/Jul/2008:17:29:44 -0600] "GET http://proxy-ob.dupagels.lib.il.us:18457/login?url=http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:        *
C: proxy  205.188.116.205  - Cjm6To7u638jzSX [19/Jul/2008:17:29:45 -0600] "GET http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:        *
C: proxy  205.188.117.134  - SIB7ja516kjANPa [21/Jul/2008:11:21:21 -0600] "GET http://proxy-ob.dupagels.lib.il.us:18457/login?url=http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:        *
C: proxy  205.188.116.205  - SIB7ja516kjANPa [21/Jul/2008:11:21:21 -0600] "GET http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

C:        *
F: Closing logfile /usr/local/analog-6.0.3/logfile.log
S: Successful requests: 52
S: Redirected requests: 21
S: Failed requests: 0
S: Requests returning informational status code: 0
S: Status code not given: 0
S: Unwanted lines: 0
S: Corrupt lines: 16
./analog: Warning L: Large number of corrupt lines in logfile
  /usr/local/analog-6.0.3/logfile.log: turn debugging on or try different
  LOGFORMAT
  (For help on all errors and warnings, see docs/errors.html)
    Current logfile format:
      %j %S  %j %j [%d/%M/%Y:%h:%n:%j] "%j%w%r%wHTTP%j" %c %b\n
S: Earliest entry in logfile: 16/Jul/08:1147
S: Latest entry in logfile: 23/Jul/08:0919
F: Opening /usr/local/apache2/htdocs/ob_proxyStats.html as output file
F: Opening /usr/local/apache2/htdocs/org.png as pie chart file
F: Closing /usr/local/apache2/htdocs/org.png
./analog: Warning R: In Request Report, turning off pie chart with no wedges
F: Closing /usr/local/apache2/htdocs/ob_proxyStats.html

To me, it looks like the logfiles marked as "corrupt" are in the same format as the logfiles that are not corrupt:

Marked as corrupted:
C: proxy  192.168.1.130  - E37HRQKiv0dANot [18/Jul/2008:14:53:48 -0600] "GET http://infoweb.newsbank.com?db=OBIT HTTP/1.1" 200 0

Not marked as corrupted:
proxy 64.12.117.9  - CGATZNblYc4mtcZ [17/Jul/2008:00:09:34 -0600] "GET http://infoweb.newsbank.com?db=CTRB HTTP/1.1" 200 0

I've attached the test logfile that I have been using to generate these errors.
<<...>>

Again, any help whatsoever would be appreciated.

+------------------------------------------------------------------------
|  TO UNSUBSCRIBE from this list:
|    http://lists.meer.net/mailman/listinfo/analog-help
|
|  Analog Documentation: http://analog.cx/docs/Readme.html
|  List archives:  http://www.analog.cx/docs/mailing.html#listarchives
|  Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.