I'm afraid I can't comment on whether your example would work - I don't
use it on any of my sites, because they all have some dynamic content,
(PHP, JSP etc) which doesn't work with direct access like this.
In most situations, there is no need for the requests to leave the local
machine. In my case, I need to use a proxy to be able to reach other
machines on the network, but from what you say you probably don't need
this set at all. If you are still getting traffic going out onto the
internet,you might want to experiment with indexing 'http://localhost/"
and using URL aliases to re-insert the proper domain name.
If you are still running for too long, it would probably be possible to
adjust things so that you did your indexing in several discrete steps,
or do it on a separate machine and transfer the databases once complete.
Hope that helps,
Mike
_____
From: [email protected]
[mailto:[email protected]] On Behalf Of Amaltas
Bohra
Sent: Wednesday, April 19, 2006 4:38 AM
To: [email protected]
Subject: [htdig] Re: htdig-general digest, Vol 1 #1799 - 2 msgs
I have already installed htdig with default configuration
settings, and just changed the database_dir and start_url. when I run
rundig, it takes long time because rundig tries to index local pages as
if they reside on different server and request goes to internet and
comes back to my server, this may also be consuming lot of bandwidth.
I just want to index my own website, and make rundig index pages
on local filesystem, but as if it is HTTP url.
I dont' konw about prxy settings, but will this work:
local_urls: http://www.mydomain.com/=/usr/local/apache/htdocs/
so after indexing htdig returns results like
http://www.mydomain.com/myindexpage.html
Thanks
Amaltas
On 4/19/06, [email protected] <
[email protected]
<mailto:[email protected]> > wrote:
Send htdig-general mailing list submissions to
[email protected]
To subscribe or unsubscribe via the World Wide Web,
visit
https://lists.sourceforge.net/lists/listinfo/htdig-general
or, via email, send a message with subject or body
'help' to
[email protected]
You can reach the person managing the list at
[email protected]
When replying, please edit your Subject line so it is
more specific
than "Re: Contents of htdig-general digest..."
Today's Topics:
1. Indexing local website (Amaltas Bohra)
2. RE: Indexing local website
([email protected])
--__--__--
Message: 1
Date: Tue, 18 Apr 2006 15:57:59 +0530
From: "Amaltas Bohra" <[email protected]>
To: [email protected]
Subject: [htdig] Indexing local website
------=_Part_24292_1398814.1145356079268
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
Hi,
I am new to indexing and htdig. I have two questions:
1) When we index local site (htdig and html documents
are in the same
folder), are they fast or do htdig indexes them as if it
is a different
website and all the requests come through internet. I
have some limits with
my webhost and I cannot run a script for more than 180
seconds.
2) Do htdig have any configuration settings wherein we
can put settings so
that it can differentiate between local documents and
documents on other
servers.
Thanks
------=_Part_24292_1398814.1145356079268
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
<div>Hi,<br>I am new to indexing and htdig. I have two
questions:<br>1) Whe=
n we index local site (htdig and html documents are
in the same folder=
), are they fast or do htdig indexes them as if it is a
different website a=
nd all the requests come through
internet. I have some limit=
s with my webhost and I cannot run a script for more
than 180 seconds.
<br>2) Do htdig have any configuration settings wherein
we can put settings=
so that it can differentiate between local documents and
documents on othe=
r servers.<br><br>Thanks</div>
------=_Part_24292_1398814.1145356079268--
--__--__--
Message: 2
Subject: RE: [htdig] Indexing local website
Date: Tue, 18 Apr 2006 11:45:10 +0100
From: <[email protected]>
To: <[email protected]>
This is a multi-part message in MIME format.
------_=_NextPart_001_01C662D5.2A14768C
Content-Type: text/plain;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
I'm not quite clear from this whether you are currently
running htdig,
or just thinking about it.
=20
The following settings are particularly relevant to this
situation; if
you have already installed htdig, can you tell us how
you have them
set?:
http_proxy: =09
http_proxy_exclude:
local_urls:=20
local_urls_only:=20
server_wait_time:=20
=20
More info on all of the above can be found at:
http://www.htdig.org/confindex.html
=20
Regards,
Mike
=20
_____ =20
From: [email protected]
[mailto: [email protected]
<mailto:[email protected]> ] On Behalf Of
Amaltas
Bohra
Sent: Tuesday, April 18, 2006 11:28 AM
To: [email protected]
Subject: [htdig] Indexing local website
=09
=09
Hi,
I am new to indexing and htdig. I have two
questions:
1) When we index local site (htdig and html
documents are in the
same folder), are they fast or do htdig indexes them as
if it is a
different website and all the requests come through
internet. I have
some limits with my webhost and I cannot run a script
for more than 180
seconds.=20
2) Do htdig have any configuration settings
wherein we can put
settings so that it can differentiate between local
documents and
documents on other servers.
=09
Thanks
------_=_NextPart_001_01C662D5.2A14768C
Content-Type: text/html;
charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0
Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Dus-ascii">
<META content=3D"MSHTML 6.00.2800.1522"
name=3DGENERATOR></HEAD>
<BODY>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial
color=3D#0000ff =
size=3D2><SPAN=20
class=3D998443910-18042006>I'm not quite clear from this
whether you are =
currently=20
running htdig, or just thinking about
it.</SPAN></FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial
color=3D#0000ff =
size=3D2><SPAN=20
class=3D998443910-18042006></SPAN></FONT> </DIV>
<DIV dir=3Dltr align=3Dleft><FONT face=3DArial
color=3D#0000ff =
size=3D2><SPAN=20
class=3D998443910-18042006>The following settings are
particularly =
relevant to=20
this situation; if you have already installed htdig, can
you tell us how =
you=20
have them set?:</SPAN></FONT></DIV><FONT color=3D#0000ff
size=3D2><SPAN=20
class=3D998443910-18042006>
<DIV dir=3Dltr align=3Dleft>
<TABLE>
<TBODY>
<TR>
<TD noWrap><FONT face=3DArial>http_proxy: =
</FONT></TD></TR></TBODY></TABLE></DIV>
<DIV dir=3Dltr align=3Dleft><FONT =
face=3DArial>http_proxy_exclude:</FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT
face=3DArial>local_urls: </FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT
face=3DArial>local_urls_only: =
</FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT
face=3DArial>server_wait_time: =
</FONT></DIV>
<DIV dir=3Dltr align=3Dleft><FONT
face=3DArial></FONT> </DIV>
<DIV dir=3Dltr align=3Dleft><SPAN
class=3D998443910-18042006><FONT =
face=3DArial>More=20
info on all of the above can be found at: <A=20
href=3D" http://www.htdig.org/confindex.html
<http://www.htdig.org/confindex.html> ">http://www.htdig.org/confind=
ex.html</A></FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN
class=3D998443910-18042006><FONT=20
face=3DArial></FONT></SPAN> </DIV>
<DIV dir=3Dltr align=3Dleft><SPAN
class=3D998443910-18042006><FONT=20
face=3DArial>Regards,</FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN
class=3D998443910-18042006><FONT=20
face=3DArial>Mike</FONT></SPAN></DIV>
<DIV dir=3Dltr align=3Dleft><SPAN=20
class=3D998443910-18042006></SPAN> </DIV></SPAN></FONT><BR>
<BLOCKQUOTE dir=3Dltr=20
style=3D"PADDING-LEFT: 5px; MARGIN-LEFT: 5px;
BORDER-LEFT: #0000ff 2px =
solid; MARGIN-RIGHT: 0px">
<DIV class=3DOutlookMessageHeader lang=3Den-us dir=3Dltr
align=3Dleft>
<HR tabIndex=3D-1>
<FONT face=3DTahoma size=3D2><B>From:</B>=20
[email protected]=20
[mailto:[email protected] ]
<B>On Behalf Of =
</B>Amaltas=20
Bohra<BR><B>Sent:</B> Tuesday, April 18, 2006 11:28
AM<BR><B>To:</B>=20
[email protected]<BR><B>Subject:</B>
[htdig] =
Indexing local=20
website<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV>Hi,<BR>I am new to indexing and htdig. I have two =
questions:<BR>1) When=20
we index local site (htdig and html documents are
in the same =
folder),=20
are they fast or do htdig indexes them as if it is a
different website =
and=20
all the requests come through internet. I
have some =
limits with=20
my webhost and I cannot run a script for more than 180
seconds. <BR>2) =
Do=20
htdig have any configuration settings wherein we can put
settings so =
that it=20
can differentiate between local documents and documents
on other=20
servers.<BR><BR>Thanks</DIV></BLOCKQUOTE></BODY></HTML>
------_=_NextPart_001_01C662D5.2A14768C--
--__--__--
_______________________________________________
ht://Dig general mailing list:
<[email protected]>
ht://Dig FAQ: http://htdig.sourceforge.net/FAQ.html
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-general
End of htdig-general Digest
lmpx.com only provides a reader for public news (NNTP) servers. It is not
affiliated with the servers or forums shown here and is not responsible for
the content of articles, which is written by their respective authors.