Re: Performance importing pages

On 29 May, 2005, at 6:31, [email protected] 
wrote:

> I have a very simple piece of code to import a set of articles into a
> MoinMoin Wiki.
>
> It goes like this;
>
> <pre>
> for PageName in UniqHeadings:
>     request = RequestCLI()
>     editor = PageEditor(request,PageName)
>     request.form={}
>     editor.saveText(editor.normalizeText(Articles[PageName]), 0)
> </pre>
>
> this has a list of around 4,500 UniqHeadings to get through. Average 
> article
> size is 2k. An Apache served MoinMoin instance is running.
>
> It is currently writing pages at the rate of around 8 per minute. Do I
> really have to wait 9 hours, or is something amiss?
>

I think you miss something - looks like this code will not work - how 
the code knows where is your wiki data dir with no url?

I tested this code on my idle machine (G5 Dual 2G):

import sys
sys.path = [# The path to the wiki directory
             '/Volumes/Home/nir/wiki/fix',
             # The path to moinmoin, not needed if its installed with 
setup.py
             '/Volumes/Home/nir/Projects/moin/fix'] + sys.path


from MoinMoin.PageEditor import PageEditor
from MoinMoin.request import RequestCLI


def save(url, pagename, text):
     request = RequestCLI(url=url, pagename=pagename)
     editor = PageEditor(request, pagename)
     text = editor.normalizeText(text)
     dummy, revision, exists = editor.get_rev()
     return editor.saveText(text, revision)


import os
url = 'localhost/fix/'
dir = 'imports'
files = [name for name in os.listdir(dir)
          if not name.startswith('.')]
for name in files:
     text = file(os.path.join(dir, name)).read()
     save(url, name, text)


I imported 80 2132 bytes pages in 10.4s.


Best Regards,

Nir Soffer



-------------------------------------------------------
This SF.Net email is sponsored by Yahoo.
Introducing Yahoo! Search Developer Network - Create apps using Yahoo!
Search APIs Find out how you can build Yahoo! directly into your own
Applications - visit http://developer.yahoo.net/?fr=offad-ysdn-ostg-q22005