Announce: mnoGoSearch-3.3.10

Hi,

mnoGoSearch 3.3.10 is now available.


The changes in this release include:

- Content-Length is now stored into the database for FTP protocol when
CheckOnly access method is used. Previous it was set to 0.

- mconv conversion utility improvements were made. Input buffer size was
changed to 1Mb to avoid breaking apart multibyte characters when
processing a file with long lines. -x command line option was added to
display characters that can not be converted to the target character set
using hexadecimal SGML entities (e.g. ģ).

- zh-hans.utf8.lm and zh-hant.utf8.lm language maps were added to detect
Simplified and Traditional Chinese in UTF-8.

- Displaying Cached copies now works with mnoGoSearch PHP extension
module.

- Date formats "1997-07-16T19:20:30+01:00" and
"1997-07-16T19:20:30-01:00" are now understood in protocol headers and
when parsing XML files.

- The Sitemap Protocol is now supported to fetch the list of URLs
available for crawling on a website. A new command UseSitemap was added
to specify whether to use Sitemap Protocol (yes by default).

- Fixed that binding of integer parameters in SQL driver didn't work
well on 64-bit platforms which might cause indexer -Eblob failures.

- Fixed that ctlib (Sybase and Microsoft SQL Server client library)
driver returned CS_ROW_FAIL error on float numbers with high precision.

- QCache=yes and Suggest now work with Microsoft SQL Server.

- Running multiple indexer crawling processes is now possible with
Microsoft SQL Server. indexer uses the (TABLOCKX) table hint when
fetching targets from the database to avoid crawling of the same
documents by multiple indexer instances. Previously running multiple
crawling processes was possible only with MySQL, PostgreSQL and Oracle.

- indexer -Eblob now runs in non-locking mode (i.e. without search down
time) when working with Microsoft SQL Server. Previously non-locking
re-indexing was possible only with MySQL, PostgreSQL and Oracle.

- Bug#4220 "one-character HTML titles are not indexed" was fixed.

- Bug#501 "HoldBadHrefs don't work" was fixed.

- indexer now respects the -D command line option when dumping data. For
  example, indexer -Edumpdata -D2 will dump data from the second DBAddr
command in indexer.conf.

- New grouping mode GroupBySite rank was added.

- MonetDB and OpenLink Virtuoso databases is now supported.

- Bug#3963 "SQL injection possible with tag and URL parameter" was
fixed. Protection against SQL injection in other search parameters was
improved.

- New parameter MultiInsert=yes was added the DBAddr command, to enable
inserting of multiple records in a single INSERT SQL statement when
running "indexer -Eblob" with a MySQL database.

- Verbose output was improved to give more information about time spent
on various indexing steps (when running indexer -Eblob) and search steps
(when running search.cgi), to find performance bottlenecks easier.

- Minor indexer -Eblob performance improvement was made: more records
are now inserted per single statement prepare call.

- An XSS problem was fixed: typing this URL in Internet Explorer address
  bar popped up an alert message window:

http://localhost/cgi-bin/search.cgi?q=who>"><script>alert(123)</script><"=Search!

- It's now possible to specify an alternative name for the bdict table
name in DBAddr parameter:

   DBADdr mysql://root@localhost/test/?bdict=bdict_name

This can be used to build multiple search indexes in the same database,
for example, using different subsection filters.

- Bug#3792 "URL limit for htdb causes inefficient SQL query" was fixed.

- Bug#3806 "wrong usage of memset function" was fixed.

- The UserScore command can now be written in indexer.conf, so its
result is cached in the database during indexer -Eblob time and then
search.cgi can use the cached result at search time. This improves
performance in case of complex SQL queries in UserScore.

- Tika MSWord-to-text converter configuration instructions were added
into the manual.

Greetings!