Re: htDig & ASP

ASP, PHP, JSP, cgi, and any other server-side scripting language sites  =

can be indexed just fine by htdig. It is true, however, that a  =

developer _may_ find a way to script a site such that it cannot be  =

properly indexed by htdig (or that the indexing/searching process would  =

not return optimal results).

It should also be noted that if the content to be searched lives in a  =

database and the only way to access the content is via a search form,  =

that a workaround would have to be implemented in order for htdig to be  =

able to index the content. This is because htdig indexes a site like a  =

spider: it crawls every link looking for content. If there are no links  =

to the content, the content will not be included in the index (nor in  =

the search results, obviously).

I can promise you that at least one major firm in the U.S. uses htdig  =

as the search engine for their intranet, which is done in PHP and  =

Fusebox (which does not produce Search Engine Friendly URLs, in other  =

words, there are plenty of query string parameters). The content lives  =

in a database and access is restricted by groups (you have to be a  =

member of the group to see the content). The search results are  =

filtered. Users only see content for which they have security  =

clearance. This is accomplished via a PHP wrapper.

I should note that I am NOT a developer (but I have provided an  =

installer package for Mac OS X and have used htdig since 1998).

HTH

Ted Stresen-Reuter
http://www.tedmasterweb.com/modules/mydownloads/viewcat.php?cid=3D1
http://www.clevernet.biz


On Aug 1, 2006, at 2:44 AM, JL Martin wrote:

> Hi,
> =A0
> My hosting company (Telus) claims that htDig cannot index websites  =

> that make use of ASP (they use version 3.1.6 and it worked in the  =

> past).
> =A0
> Here's an excerpt of their email .....
> =A0
> ....To summarize, the results that HtDig would receive from a site  =

> using a server side scripting language could easily be ambiguous if  =

> not meaningless. There really is no solution for this, as we cannot  =

> predict what a developer might chose to code or how they might choose  =

> the make the site work. The best that can be done is to realize that  =

> HtDig is a good tool for static (html) websites. Unfortunately, as a  =

> result of it's design and implementation, HtDig does not handle  =

> dynamically composed websites very well or in a consistant manner.
> =A0
> Any truth to this? Would upgrading to=A0v3.2 resolve this (scripting)  =

> issue?
> =A0
> JL  =

> Martin=A0---------------------------------------------------------------- =

> ---------
> Using Tomcat but need to do more? Need to support web services,  =

> security?
> Get stuff done quickly with pre-integrated technology to make your job  =

> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache  =

> Geronimo
> http://sel.as-us.falkag.net/sel? =

> cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D121642_________________________=
______ =

> ________________
> ht://Dig Developer mailing list:
> [email protected]
> List information (subscribe/unsubscribe, etc.)
> https://lists.sourceforge.net/lists/listinfo/htdig-dev

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easi=
er
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D1=
21642
_______________________________________________
ht://Dig Developer mailing list:
[email protected]
List information (subscribe/unsubscribe, etc.)
https://lists.sourceforge.net/lists/listinfo/htdig-dev