Webboard: indexer -Eblob : more than 11 hours...
| Newsgroups | gmane.comp.web.mnogosearch.general |
|---|---|
| Message-ID | <[email protected]> |
Author: Christophe NOUVEL Email: [email protected] Message: > > <skip> > > > > > It seems your documents are huge enough. > > > An average internet HTML document is usually about 6-7Kb. > > > You have 39Gb for 1 million documents, which is about 39Kb > > > per document. > > > > Probably MS Frontpage! > > What does "SHOW TABLE STATUS LIKE 'bdicti'" return for you? Not easy to read after such a paste but here it is : mysql> SHOW TABLE STATUS LIKE 'bdicti'; +--------+--------+---------+------------+---------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-----------------+----------+---------------------------------------+---------+ | Name | Engine | Version | Row_format | Rows | Avg_row_length | Data_length | Max_data_length | Index_length | Data_free | Auto_increment | Create_time | Update_time | Check_time | Collation | Checksum | Create_options | Comment | +--------+--------+---------+------------+---------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-----------------+----------+---------------------------------------+---------+ | bdicti | MyISAM | 10 | Dynamic | 1105898 | 2360 | 3044688400 | 1099511627775 | 29620224 | 434418896 | NULL | 2007-05-31 07:55:27 | 2010-07-03 22:13:26 | 2008-10-13 17:27:12 | utf8_general_ci | NULL | max_rows=300000000 avg_row_length=512 | | +--------+--------+---------+------------+---------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+---------------------+-----------------+----------+---------------------------------------+---------+ 1 row in set (0.00 sec) mysql> > Looks fine. Did step=100000 help? Sure! Lines like 'Loading documents 1-100001' to 'Loading documents 2000001-2100001' are now logged. But in the end : Jul 3 21:51:13 fronton indexer[1191]: Total converted: 182589007 records, 75567 3713 bytes Jul 3 22:13:30 fronton indexer[1191]: Converting url data Jul 3 22:14:32 fronton indexer[1191]: Switching to new blob table. Jul 3 22:14:33 fronton indexer[1191]: Converting to blob finished 7445.71 Back to normal conversion times. By the way, I don't understand why Loading documents 2000001-2100001 if 1336677 total are reported by indexer -S... Christophe. Reply: <http://www.mnogosearch.org/board/message.php?id=21093>