Re: RE: 1and1 logfile format
Jeremy Wadsack <[email protected]> Mon, 2 Nov 2009 08:56:15 -0800
| Newsgroups | gmane.comp.web.analog.general |
|---|---|
| Message-ID | <[email protected]> |
--===============1164933513== Content-Type: multipart/alternative; boundary=0016364eee547c41d004776641aa --0016364eee547c41d004776641aa Content-Type: text/plain; charset=ISO-8859-1 LOGFORMAT (or APACHELOGFORMAT) must precede LOGFILE. It defines the log file format used for subsequest LOGFILE commands. See http://analog.cx/docs/logfmt.html. You can use %v in Analog's LOGFORMAT command to define the virtual host that the log line refers to. Then you can use the Virtual Host report to view cross-host comparisons or VHOSTEXCLUDE to constrain a report to a single virtual host. The errors.txt file is meant for human consumption and provides no additional details (from an analytics standpoint) than are already listed in the access.log. You can skip the file completely. See http://analog.cx/docs/faq.html#faq173 -- Jeremy Wadsack On Mon, Nov 2, 2009 at 7:53 AM, Iain Hunneybell <iain-Vvd6Z7Soz6DPlRtfX+RAZ1pr/1R2p/[email protected]>wrote: > Well I've cure one issue but found another... > > By placing my LOGFORMAT ahead of the LOGFILE directive I've produced a > report, but the issues I have are: > > 1. This odd multi-host combined logfile with it would seem multiple domains > owned by a single 1and1 account in the one logfile, and > > 2. The search engine queries are lost and <errors.txt> reports: > > analog: Warning R: Turning off empty Search Query Report > analog: Warning R: Turning off empty Search Word Report > > But...I can find the various Google and other referrals with queries so > don't yet understand why the search word reports are empty and so switched > off. > > So if anyone has experience of 1and1 logfile format I'd be very grateful > :-) > > Many thanks.../Iain > > _____________________________________________ > *From: * Iain Hunneybell [*mailto:iain-Vvd6Z7Soz6DPlRtfX+RAZ1pr/1R2p/[email protected]*<iain-Vvd6Z7Soz6DPlRtfX+RAZ1pr/1R2p/[email protected]>] > > *Sent: * 02 November 2009 15:17 > *To: * 'Support for analog web log analyzer' > *Subject: * 1and1 logfile format > > Has anyone any experience of processing 1and1 logfiles through analog? They > seem to use an unusual format that analog doesn't recognise. The record > layout seems to be: > > Requestor IP > Username > Date/time > "GET requested URL inc HTTP version" > HTTP response > Bytes > Domain to which the request refers <-- This seems to be something > particular to 1and1 and is unquoted > "Referrer" > "User agent" > "-" <-- No idea what this is meant to be! > > So for example: > > 124.115.0.145 - - [07/Sep/2009:00:24:08 +0200] "GET / HTTP/1.1" 200 17616 > *www.mydomain.com* "*http://www.mydomain.com/* <http://www.mydomain.com/>" > "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)" "-" > > I'm presuming a LOGFORMAT would fix it but no luck. I tried: > > LOGFORMAT (%s - %u [%d/%M/%Y:%h:%n:%j %j] "%j %r %j" %c %b %j %f %B %j) > > Which when matched up with the log record wold read as: > > %s - Requestor IP > - > %u - Username > [%d/%M/%Y:%h:%n:%j %j] - Date/time > %j - GET > %r - requested URL > %j - HTTP version > %c - HTTP response > %b - Bytes > %j - Domain to which the request refers > %f - Referrer > %B - Browser/user agent > %j - "-" > > Strangely, despite the LOGFORMAT command <errors.txt> reports: > > analog: Warning F: Can't auto-detect format of logfile > Logs\20090907.log: ignoring it > > Any ideas? > > Many thanks.../Iain > > +------------------------------------------------------------------------ > | TO UNSUBSCRIBE from this list: > | http://lists.meer.net/mailman/listinfo/analog-help > | > | Analog Documentation: http://analog.cx/docs/Readme.html > | List archives: http://www.analog.cx/docs/mailing.html#listarchives > | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general > +------------------------------------------------------------------------ > > --0016364eee547c41d004776641aa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable LOGFORMAT (or APACHELOGFORMAT) must precede LOGFILE. It defines the log fil= e format used for subsequest LOGFILE commands. See <a href=3D"http://analog= .cx/docs/logfmt.html">http://analog.cx/docs/logfmt.html</a>.<br><br>You can= use %v in Analog's LOGFORMAT command to define the virtual host that t= he log line refers to. Then you can use the Virtual Host report to view cro= ss-host comparisons or VHOSTEXCLUDE to constrain a report to a single virtu= al host.<br> <br>The errors.txt file is meant for human consumption and provides no addi= tional details (from an analytics standpoint) than are already listed in th= e access.log. You can skip the file completely. See <a href=3D"http://analo= g.cx/docs/faq.html#faq173">http://analog.cx/docs/faq.html#faq173</a> <br> <br>--<br clear=3D"all">Jeremy Wadsack<br> <br><br><div class=3D"gmail_quote">On Mon, Nov 2, 2009 at 7:53 AM, Iain Hun= neybell <span dir=3D"ltr"><<a href=3D"mailto:iain-Vvd6Z7Soz6DPlRtfX+RAZ1pr/1R2p/[email protected]">iai= [email protected]</a>></span> wrote:<br><blockquote class=3D"gmail_quo= te" style=3D"border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt= 0.8ex; padding-left: 1ex;"> <div> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">Well I've cure one= issue but found another...</font> </p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">By placing my LOGFORMA= T ahead of the LOGFILE directive I've produced a report, but the issues= I have are:</font> </p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">1. This odd multi-host= combined logfile with it would seem multiple domains owned by a single 1an= d1 account in the one logfile, and </font></p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">2. The search engine q= ueries are lost and <errors.txt> reports:</font> </p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">analog: Warning R: Tur= ning off empty Search Query Report</font> <br><font face=3D"Arial" size=3D"2" color=3D"#0000ff">analog: Warning R: Tu= rning off empty Search Word Report</font> </p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">But...I can find the v= arious Google and other referrals with queries so don't yet understand = why the search word reports are empty and so switched off.</font></p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">So if anyone has exper= ience of 1and1 logfile format I'd be very grateful :-)</font> </p> <p><font face=3D"Arial" size=3D"2" color=3D"#0000ff">Many thanks.../Iain</f= ont> </p> <br> <p><font face=3D"Tahoma" size=3D"1">_______________________________________= ______ </font> <br><b><font face=3D"Tahoma" size=3D"1">From: =A0</font></b> <font face=3D"= Tahoma" size=3D"1">Iain Hunneybell [</font><a href=3D"mailto:iain@ipmarketi= ng.co.uk" target=3D"_blank"><u><font face=3D"Tahoma" size=3D"1" color=3D"#0= 000ff">mailto:iain-Vvd6Z7Soz6DPlRtfX+RAZ1pr/1R2p/[email protected]</font></u></a><font face=3D"Tahoma" si= ze=3D"1">] </font> <br><b><font face=3D"Tahoma" size=3D"1">Sent:=A0=A0</font></b> <font face= =3D"Tahoma" size=3D"1">02 November 2009 15:17</font> <br><b><font face=3D"Tahoma" size=3D"1">To:=A0=A0=A0=A0</font></b> <font fa= ce=3D"Tahoma" size=3D"1">'Support for analog web log analyzer'</fon= t> <br><b><font face=3D"Tahoma" size=3D"1">Subject:=A0=A0=A0=A0=A0=A0=A0</font= ></b> <font face=3D"Tahoma" size=3D"1">1and1 logfile format</font> </p><div><div></div><div class=3D"h5"> <p><font face=3D"Courier New" size=3D"2">Has anyone any experience of proce= ssing 1and1 logfiles through analog? They seem to use an unusual format tha= t analog doesn't recognise. The record layout seems to be:</font></p> <p><font face=3D"Courier New" size=3D"2">Requestor IP</font> <br><font face=3D"Courier New" size=3D"2">Username</font> <br><font face=3D"Courier New" size=3D"2">Date/time</font> <br><font face=3D"Courier New" size=3D"2">"GET requested URL inc HTTP = version"</font> <br><font face=3D"Courier New" size=3D"2">HTTP response</font> <br><font face=3D"Courier New" size=3D"2">Bytes</font> <br><font face=3D"Courier New" size=3D"2">Domain to which the request refer= s <-- This seems to be something particular to 1and1 and is unquoted</fo= nt> <br><font face=3D"Courier New" size=3D"2">"Referrer"</font> <br><font face=3D"Courier New" size=3D"2">"User agent"</font> <br><font face=3D"Courier New" size=3D"2">"-" <-- No idea what= this is meant to be!</font> </p> <p><font face=3D"Courier New" size=3D"2">So for example:</font> </p> <p><font face=3D"Courier New" size=3D"2">124.115.0.145 - - [07/Sep/2009:00:= 24:08 +0200] "GET / HTTP/1.1" 200 17616 </font><a><u><font face= =3D"Courier New" size=3D"2" color=3D"#0000ff">www.mydomain.com</font></u></= a><font face=3D"Courier New" size=3D"2"> "</font><a href=3D"http://www= .mydomain.com/" target=3D"_blank"><u><font face=3D"Courier New" size=3D"2" = color=3D"#0000ff">http://www.mydomain.com/</font></u></a><font face=3D"Cour= ier New" size=3D"2">" "Mozilla/4.0 (compatible; MSIE 6.0; Windows= NT 5.1)" "-"</font></p> <p><font face=3D"Courier New" size=3D"2">I'm presuming a LOGFORMAT woul= d fix it but no luck. I tried:</font> </p> <p><font face=3D"Courier New" size=3D"2">LOGFORMAT (%s - %u [%d/%M/%Y:%h:%n= :%j %j] "%j %r %j" %c %b %j %f %B %j)</font> </p> <p><font face=3D"Courier New" size=3D"2">Which when matched up with the log= record wold read as:</font> </p> <p><font face=3D"Courier New" size=3D"2">%s =A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0= =A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - Requestor IP</font> <br><font face=3D"Courier New" size=3D"2">-=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0= =A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 </font> <br><font face=3D"Courier New" size=3D"2">%u =A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - Username</font> <br><font face=3D"Courier New" size=3D"2">[%d/%M/%Y:%h:%n:%j %j]=A0 - Date/= time</font> <br><font face=3D"Courier New" size=3D"2">%j=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - GET </font> <br><font face=3D"Courier New" size=3D"2">%r=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - requested URL </font> <br><font face=3D"Courier New" size=3D"2">%j=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - HTTP version</font> <br><font face=3D"Courier New" size=3D"2">%c=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - HTTP response</font> <br><font face=3D"Courier New" size=3D"2">%b=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - Bytes</font> <br><font face=3D"Courier New" size=3D"2">%j=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - Domain to which the re= quest refers </font> <br><font face=3D"Courier New" size=3D"2">%f=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - Referrer</font> <br><font face=3D"Courier New" size=3D"2">%B=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - Browser/user agent</fo= nt> <br><font face=3D"Courier New" size=3D"2">%j=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0= =A0=A0 =A0=A0=A0=A0=A0=A0=A0 =A0=A0=A0=A0=A0=A0=A0 - "-" </font> </p> <p><font face=3D"Courier New" size=3D"2">Strangely, despite the LOGFORMAT c= ommand <errors.txt> reports:</font> </p> <p><font face=3D"Courier New" size=3D"2">analog: Warning F: Can't auto-= detect format of logfile</font> <br><font face=3D"Courier New" size=3D"2">=A0 Logs\20090907.log: ignoring i= t</font> </p> <p><font face=3D"Courier New" size=3D"2">Any ideas?</font> </p> <p><font face=3D"Courier New" size=3D"2">Many thanks.../Iain</font> </p> </div></div></div> <br>+----------------------------------------------------------------------= --<br> | =A0TO UNSUBSCRIBE from this list:<br> | =A0 =A0<a href=3D"http://lists.meer.net/mailman/listinfo/analog-help" tar= get=3D"_blank">http://lists.meer.net/mailman/listinfo/analog-help</a><br> |<br> | =A0Analog Documentation: <a href=3D"http://analog.cx/docs/Readme.html" ta= rget=3D"_blank">http://analog.cx/docs/Readme.html</a><br> | =A0List archives: =A0<a href=3D"http://www.analog.cx/docs/mailing.html#li= starchives" target=3D"_blank">http://www.analog.cx/docs/mailing.html#listar= chives</a><br> | =A0Usenet version: news://<a href=3D"http://news.gmane.org/gmane.comp.web= .analog.general" target=3D"_blank">news.gmane.org/gmane.comp.web.analog.gen= eral</a><br> +------------------------------------------------------------------------<b= r> <br></blockquote></div><br> --0016364eee547c41d004776641aa-- --===============1164933513== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline +------------------------------------------------------------------------ | TO UNSUBSCRIBE from this list: | http://lists.meer.net/mailman/listinfo/analog-help | | Analog Documentation: http://analog.cx/docs/Readme.html | List archives: http://www.analog.cx/docs/mailing.html#listarchives | Usenet version: news://news.gmane.org/gmane.comp.web.analog.general +------------------------------------------------------------------------ --===============1164933513==--