Unified Patch to speed up Wordsite Generation
"John Hansen" <[email protected]> Wed, 1 Oct 2003 07:58:52 +1000
| Newsgroups | gmane.comp.web.aspseek.devel |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. ------_=_NextPart_001_01C3879E.74D46859 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Attached, a unified patch, that will dramatically increase word site = generation as the database grows. 160000 records took 35minutes to generate without this patch. Now takes 30 seconds. John ------_=_NextPart_001_01C3879E.74D46859 Content-Type: application/octet-stream; name="GenerateWordsite.patch" Content-Transfer-Encoding: base64 Content-Description: GenerateWordsite.patch Content-Disposition: attachment; filename="GenerateWordsite.patch" ZGlmZiAtcnUgYXNwc2Vlay0xLjIuMTAvaW5jbHVkZS9zcWxkYi5oIGFzcHNlZWstMS4yLjEwLnBh dGNoZWQvaW5jbHVkZS9zcWxkYi5oDQotLS0gYXNwc2Vlay0xLjIuMTAvaW5jbHVkZS9zcWxkYi5o CVR1ZSBKdW4gMTggMjE6MjA6MzggMjAwMg0KKysrIGFzcHNlZWstMS4yLjEwLnBhdGNoZWQvaW5j bHVkZS9zcWxkYi5oCVR1ZSBTZXAgMzAgMjA6MTM6MzggMjAwMw0KQEAgLTU2Nyw2ICs1NjcsMTAg QEANCiAJew0KIAkJcmV0dXJuIFNldFF1ZXJ5KCJTRUxFQ1QgdXJsX2lkLHNpdGVfaWQsbmV4dF9p bmRleF90aW1lIEZST00gdXJsd29yZCBXSEVSRSBkZWxldGVkPTAgQU5EIGhvcHM9OjEgQU5EIG5l eHRfaW5kZXhfdGltZT06MiAlcyIsIHBhcmFtKTsNCiAJfQ0KKwl2aXJ0dWFsIENTUUxRdWVyeSAq IFJlcGxhY2VXb3JkU2l0ZUkoQ1NRTFBhcmFtICpwYXJhbSkNCisJew0KKwkJcmV0dXJuIFNldFF1 ZXJ5KCJVUERBVEUgd29yZHNpdGUgU0VUIHNpdGVzID0gOjEgV0hFUkUgd29yZCA9IDoyIiwgcGFy YW0pOw0KKwl9DQogCXZpcnR1YWwgQ1NRTFF1ZXJ5ICogSW5zZXJ0V29yZFNpdGVJKENTUUxQYXJh bSAqcGFyYW0pDQogCXsNCiAJCXJldHVybiBTZXRRdWVyeSgiSU5TRVJUIElOVE8gd29yZHNpdGUo d29yZCwgc2l0ZXMpIFZBTFVFUyg6MSwgOjIpIiwgcGFyYW0pOw0KQEAgLTU4NSw3ICs1ODksNyBA QA0KIAl9DQogCXZpcnR1YWwgQ1NRTFF1ZXJ5ICogR2VuZXJhdGVXb3JkU2l0ZVMxKCkNCiAJew0K LQkJcmV0dXJuIFNldFF1ZXJ5KCJTRUxFQ1Qgd29yZCBGUk9NIHdvcmRzaXRlIik7DQorCQlyZXR1 cm4gU2V0UXVlcnkoIlNFTEVDVCB3b3JkLHN1YnN0cmluZyhzaXRlcywxLDE2MSkgRlJPTSB3b3Jk c2l0ZSIpOw0KIAl9DQogCXZpcnR1YWwgQ1NRTFF1ZXJ5ICogR2VuZXJhdGVXb3JkU2l0ZUQoQ1NR TFBhcmFtICpwYXJhbSkNCiAJew0KZGlmZiAtcnUgYXNwc2Vlay0xLjIuMTAvaW5jbHVkZS9zcWxk YmkuaCBhc3BzZWVrLTEuMi4xMC5wYXRjaGVkL2luY2x1ZGUvc3FsZGJpLmgNCi0tLSBhc3BzZWVr LTEuMi4xMC9pbmNsdWRlL3NxbGRiaS5oCVR1ZSBNYXkgMTQgMjE6NDc6MTYgMjAwMg0KKysrIGFz cHNlZWstMS4yLjEwLnBhdGNoZWQvaW5jbHVkZS9zcWxkYmkuaAlUdWUgU2VwIDMwIDE5OjU3OjQ1 IDIwMDMNCkBAIC04MSw2ICs4MSw3IEBADQogCXZvaWQgVXBkYXRlV29yZFNpdGUoY2hhciogaG9z dG5hbWUsIFVMT05HIHNpdGVfaWQpOwkvLy88IFVwZGF0ZXMgIndvcmRzaXRlIiB0YWJsZSBvbiAi cmVhbC10aW1lIiBpbmRleGluZw0KIAkvLy8gSW5zZXJ0cyAxIHJlY29yZCBpbnRvICJ3b3Jkc2l0 ZSIgdGFibGUNCiAJdm9pZCBJbnNlcnRXb3JkU2l0ZShjb25zdCBjaGFyKiB3cmQsIFVMT05HKiBz aXRlcywgaW50IHNpemUpOw0KKwl2b2lkIFJlcGxhY2VXb3JkU2l0ZShjb25zdCBjaGFyKiB3cmQs IFVMT05HKiBzaXRlcywgaW50IHNpemUpOw0KIAkvLy8gQWRkcyBVUkxzLCBwcm9kdWNlZCBieSBn aXZlbiBxdWVyeSB0byB0aGUgcXVldWUgb2YgVVJMcw0KIAlpbnQgQWRkVXJscyhDU1FMUXVlcnkg KnF1ZXJ5LCBkb3VibGUmIHRpbWUsIGludCYgbWF4dGltZSwgQ0ludFNldCogc2V0KTsNCiAJLy8v IFJldHVybnMgbmV4dCBVUkwgSUQgdG8gaW5kZXgsIHdoZW4gb3JkZXJpbmcgYnkgaG9wcyBpcyBv bg0KZGlmZiAtcnUgYXNwc2Vlay0xLjIuMTAvc3JjL3NxbGRiaS5jcHAgYXNwc2Vlay0xLjIuMTAu cGF0Y2hlZC9zcmMvc3FsZGJpLmNwcA0KLS0tIGFzcHNlZWstMS4yLjEwL3NyYy9zcWxkYmkuY3Bw CVR1ZSBKdW4gMTggMjE6MjA6MzggMjAwMg0KKysrIGFzcHNlZWstMS4yLjEwLnBhdGNoZWQvc3Jj L3NxbGRiaS5jcHAJVHVlIFNlcCAzMCAyMDowOTo1NSAyMDAzDQpAQCAtMzIxLDYgKzMyMSwxNyBA QA0KIAlzcWxfcmVhbF9xdWVyeShzcWxxdWVyeSk7DQogfQ0KIA0KK3ZvaWQgQ1NRTERhdGFiYXNl STo6UmVwbGFjZVdvcmRTaXRlKGNvbnN0IGNoYXIqIHdvcmQsIFVMT05HKiBzaXRlcywgaW50IHNp emUpDQorew0KKy8vICJVUERBVEUgd29yZHNpdGUgU0VUIHNpdGVzID0gJyVzJyBXSEVSRSB3b3Jk ID0gJyVzJzsNCisvLyAgICAgIHBxdWVyeSA9IG1lbWNweXEocHF1ZXJ5LCAoY2hhciopc2l0ZXMs IHNpemUgPDwgMik7DQorICAgICAgICBDU1FMUGFyYW0gcDsNCisJcC5BZGRQYXJhbUVzYygoY2hh ciopc2l0ZXMsIHNpemUgPDwgMik7DQorICAgICAgICBwLkFkZFBhcmFtKHdvcmQpOw0KKwlDU1FM UXVlcnkgKnNxbHF1ZXJ5ID0gbV9zcWxxdWVyeS0+UmVwbGFjZVdvcmRTaXRlSSgmcCk7DQorICAg ICAgICBzcWxfcmVhbF9xdWVyeShzcWxxdWVyeSk7DQorfQ0KKw0KIHZvaWQgQ1NRTERhdGFiYXNl STo6VXBkYXRlV29yZFNpdGUoY2hhciogaG9zdG5hbWUsIFVMT05HIHNpdGVfaWQpDQogew0KIAlo YXNoX21hcDxDV29yZCwgVUxPTkc+IGhtYXA7DQpAQCAtNTE1LDI5ICs1MjYsNTUgQEANCiAJfQ0K IAlpZiAoYW5zdykgZGVsZXRlIGFuc3c7DQogCWxvZ2dlci5sb2coQ0FUX0FMTCwgTF9JTkZPLCAi LiIpOw0KLS8vICJTRUxFQ1Qgd29yZCBGUk9NIHdvcmRzaXRlIik7DQorLy8gIlNFTEVDVCB3b3Jk LHN1YnN0cmluZyhzaXRlcywxLDE2MSkgRlJPTSB3b3Jkc2l0ZSIpOw0KIAlzcWxxdWVyeSA9IG1f c3FscXVlcnktPkdlbmVyYXRlV29yZFNpdGVTMSgpOw0KIAlhbnN3ID0gc3FsX3F1ZXJ5KHNxbHF1 ZXJ5KTsNCisJaW50IGNvdW50ZXIgPSAwOw0KIAl3aGlsZSAoYW5zdyAmJiBhbnN3LT5GZXRjaFJv dygpKQ0KIAl7DQorICAgICAgICAgICAgICAgIGlmKGNvdW50ZXIrKyA9PSAxMDAwMCkgew0KKwkJ ICAgIGxvZ2dlci5sb2coQ0FUX0FMTCwgTF9JTkZPLCAiLiIpOw0KKwkJICAgIGNvdW50ZXIgPSAw Ow0KKwkJfQ0KIAkJaWYgKHN0cmxlbihhbnN3LT5HZXRDb2x1bW4oMCkpIDw9IE1BWF9XT1JEX0xF TikNCiAJCXsNCiAJCQl3aXQgPSB3b3Jkcy5maW5kKGFuc3ctPkdldENvbHVtbigwKSk7DQotLy8g IkRFTEVURSBGUk9NIHdvcmRzaXRlIFdIRVJFIHdvcmQgPSAnJXMnIiwgcm93WzBdKTsNCi0JCQlD U1FMUGFyYW0gcDsNCi0JCQlwLkFkZFBhcmFtKGFuc3ctPkdldENvbHVtbigwKSk7DQotCQkJc3Fs cXVlcnkgPSBtX3NxbHF1ZXJ5LT5HZW5lcmF0ZVdvcmRTaXRlRCgmcCk7DQotCQkJc3FsX3F1ZXJ5 MihzcWxxdWVyeSk7DQorCQkJDQogCQkJaWYgKHdpdCAhPSB3b3Jkcy5lbmQoKSkNCiAJCQl7DQot CQkJCXZlY3RvcjxVTE9ORz4mIHNpdGVzID0gd2l0LT5zZWNvbmQ7DQotCQkJCUluc2VydFdvcmRT aXRlKGFuc3ctPkdldENvbHVtbigwKSwgJigqc2l0ZXMuYmVnaW4oKSksIHNpdGVzLnNpemUoKSk7 DQotCQkJCXdvcmRzLmVyYXNlKHdpdCk7DQorCQkJICAgIHZlY3RvcjxVTE9ORz4mIHNpdGVzID0g d2l0LT5zZWNvbmQ7DQorCQkJICAgIFVMT05HIGxlbiA9IGFuc3ctPkdldExlbmd0aCgxKTsNCisJ CQkgICAgaWYobGVuID09IDE2MUwpIHsNCisJCQkJUmVwbGFjZVdvcmRTaXRlKGFuc3ctPkdldENv bHVtbigwKSwgJigqc2l0ZXMuYmVnaW4oKSksIHNpdGVzLnNpemUoKSk7DQorCQkJICAgIH0gZWxz ZSB7DQorCQkJCWlmKHNpdGVzLnNpemUoKSAhPSAobGVuPj4yKSkgew0KKwkJCQkgICAgUmVwbGFj ZVdvcmRTaXRlKGFuc3ctPkdldENvbHVtbigwKSwgJigqc2l0ZXMuYmVnaW4oKSksIHNpdGVzLnNp emUoKSk7DQorCQkJCX0gZWxzZSB7DQorCQkJCSAgICBVTE9ORyogcG9sZCA9IChVTE9ORyopYW5z dy0+R2V0Q29sdW1uKDEpOw0KKwkJCQkgICAgaWYobWVtY21wKHNpdGVzLmJlZ2luKCksIHBvbGQs IGxlbikgIT0gMCkgew0KKwkJCQkJUmVwbGFjZVdvcmRTaXRlKGFuc3ctPkdldENvbHVtbigwKSwg Jigqc2l0ZXMuYmVnaW4oKSksIHNpdGVzLnNpemUoKSk7DQorCQkJCSAgICB9DQorCQkJCX0NCisJ CQkgICAgfQ0KKwkJCSAgICB3b3Jkcy5lcmFzZSh3aXQpOw0KKwkJCX0gZWxzZSB7DQorCQkJICAg IC8vICJERUxFVEUgRlJPTSB3b3Jkc2l0ZSBXSEVSRSB3b3JkID0gJyVzJyIsIHJvd1swXSk7DQor CQkJICAgIENTUUxQYXJhbSBwOw0KKwkJCSAgICBwLkFkZFBhcmFtKGFuc3ctPkdldENvbHVtbigw KSk7DQorCQkJICAgIHNxbHF1ZXJ5ID0gbV9zcWxxdWVyeS0+R2VuZXJhdGVXb3JkU2l0ZUQoJnAp Ow0KKwkJCSAgICBzcWxfcXVlcnkyKHNxbHF1ZXJ5KTsNCiAJCQl9DQogCQl9DQogCX0NCisJbG9n Z2VyLmxvZyhDQVRfQUxMLCBMX0lORk8sICIuIik7DQorCWNvdW50ZXIgPSAwOw0KIAlmb3IgKHdp dCA9IHdvcmRzLmJlZ2luKCk7IHdpdCAhPSB3b3Jkcy5lbmQoKTsgd2l0KyspDQogCXsNCisgICAg ICAgIAlpZihjb3VudGVyKysgPT0gMTAwMDApIHsNCisJICAgIAkgICAgbG9nZ2VyLmxvZyhDQVRf QUxMLCBMX0lORk8sICIuIik7DQorCSAgICAJICAgIGNvdW50ZXIgPSAwOw0KKwkJfQ0KIAkJdmVj dG9yPFVMT05HPiYgc2l0ZXMgPSB3aXQtPnNlY29uZDsNCiAJCUluc2VydFdvcmRTaXRlKHdpdC0+ Zmlyc3QuV29yZCgpLCAmKCpzaXRlcy5iZWdpbigpKSwgc2l0ZXMuc2l6ZSgpKTsNCiAJfQ== ------_=_NextPart_001_01C3879E.74D46859--