Re: Struggling with memory problem using Search Query

Zembower, Kevin <kzembowe-pdIAO+f/[email protected]> wrote:
> Aengus, thanks so much for your suggestion. Unfortunately, it doesn't
> seem to work for me. I made these entries in my analog config file:
> REFINCLUDE *.google.*,*.jhuccp.org*,*.yahoo.*
> 
> SEARCHENGINE    http://*.google.com/*   q,as_q,as_oq,as_epq,query
> SEARCHENGINE    http://*.google.co.*/*  q,as_q,as_oq,as_epq,query
> SEARCHENGINE    http://*.google.com.*/* q,as_q,as_oq,as_epq,query
> 
> This generates this section in the '--settings' output:
> Including (+) and excluding (-) the following referrers:
>  All excluded, then
>  + *.google.*
>  + *.jhuccp.org*
>  + *.yahoo.*
> 
> This leads me to believe that it's working correctly. I wanted these
> three referrers, because they make up most of my referrer report.
> However, I get this output when I run analog:
> ../analog-6.0/analog: analog version 6.0/Unix
> ../analog-6.0/analog: Warning M: Logfile
> /opt/analog/logdata/web1/access_log*
>  contains lines with no referrers, which are being filtered
>  (For help on all errors and warnings, see docs/errors.html)
> ../analog-6.0/analog: Warning M: Logfile
> /opt/analog/logdata/db/ccp-apps2/ex*
>  contains lines with no referrers, which are being filtered
> sh: line 20: 19024 Segmentation fault      ../analog-6.0/analog
> +gweb1.analog.cfg
> 
> This error message troubles me, as it seems to indicate that log
> entries without referrers were being dropped. I'll have trouble
> interepting this report, I think. Even if this report didn't seg
> fault, I'm not sure it would be useful to me.

If you want to generate a Search Report, you might as well ignore lines without referrers, because the Search report information comes from the Referrer.

(As a generale rule, the number of lines that have no referrer at all is usally pretty tiny - under normal circumstances, only bookmarks and e-mail links generate log entries with no referrer. 90% of the rest of the referrers will usually be "internal" - referrers from your own site).

> Unless I or anyone else on this list can think of another suggestion,
> I think that I just have two options:
> 1) Add more memory to this host.

You can test this very easily - exclude a couple of months from the report and see if you don't get the error when you're trying to generate a smaller report.

> 2) Generate a report with just the Search Query and Search Word
> sections, and minimize or eliminate everything else. Any guesses if
> this would work?

Sorry, I thought that you were already doing this. Yes - if you have to many logfiles to crunch in a single report, running seperate reports may allow you to get the information you need. I just turning off the other reports doesn't work, try to use the LOWMEM commands for everything except the referrers.

Aengus

+------------------------------------------------------------------------
|  TO UNSUBSCRIBE from this list:
|    http://lists.meer.net/mailman/listinfo/analog-help
|
|  Analog Documentation: http://analog.cx/docs/Readme.html
|  List archives:  http://www.analog.cx/docs/mailing.html#listarchives
|  Usenet version: news://news.gmane.org/gmane.comp.web.analog.general
+------------------------------------------------------------------------