manage_pasteObjects() *really* slow

Andreas Jung <[email protected]>
Newsgroups gmane.comp.web.zope.plone.archetypes.devel
Message-ID <DDCC955BD055AA5C67082B52@[192.168.2.121]>
Hi,

with a customer I am working on a migration of basically a simple folder 
structure (about 60k folders with up to 200 very simple AT instances inside 
each folder). For the migration we recreate the folder structure with a new
AT folder types and use manage_copy|pasteObjects() to copy over the 
subobjects. However this is horribly slow. The follow profile (for 3 folders
+ 214 subobjects) shows that a lot of time is spend (or should I say wasted)
in Referenceably._getCopy(). In addition we see 2.6 million(!) calls to 
serialize.persisent_id()..that does not make a much sense to me.  Any 
insights?

Andreas
--------------------------------

  List reduced from 859 to 25 due to restriction <25>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.001    0.001    4.951    4.951 migrate.py:7(write)
       13    0.000    0.000    4.449    0.342 _transaction.py:287(savepoint)
       13    0.000    0.000    4.448    0.342 _transaction.py:663(__init__)
       13    0.000    0.000    4.448    0.342 Connection.py:1005(savepoint)
       13    0.013    0.001    4.448    0.342 Connection.py:486(_commit)
     3533    0.070    0.000    4.413    0.001 
Connection.py:532(_store_objects)
     3580    0.006    0.000    4.248    0.001 serialize.py:381(serialize)
        3    0.001    0.000    4.248    1.416 
CopySupport.py:160(manage_pasteObjects)
     3580    2.967    0.001    4.242    0.001 serialize.py:409(_dump)
        6    0.003    0.000    4.040    0.673 CopySupport.py:559(_getCopy)
        6    0.000    0.000    4.022    0.670 _manager.py:109(savepoint)
        3    0.000    0.000    1.666    0.555 Referenceable.py:320(_getCopy)
  2636095    1.264    0.000    1.265    0.000 
serialize.py:184(persistent_id)
       21    0.001    0.000    0.309    0.015 
CatalogTool.py:389(catalog_object)
       27    0.001    0.000    0.307    0.011 
ZCatalog.py:551(catalog_object)
       27    0.002    0.000    0.306    0.011 Catalog.py:313(catalogObject)
        9    0.000    0.000    0.303    0.034 
CMFCatalogAware.py:193(manage_afterAdd)
        3    0.000    0.000    0.278    0.093 
constraintypes.py:265(invokeFactory)
        3    0.000    0.000    0.277    0.092 
PortalFolder.py:398(invokeFactory)
        3    0.000    0.000    0.277    0.092 
TypesTool.py:909(constructContent)
        3    0.000    0.000    0.277    0.092 
TypesTool.py:326(constructInstance)
        3    0.000    0.000    0.259    0.086 
TypesTool.py:551(_constructInstance)
        3    0.000    0.000    0.259    0.086 <string>:2(addExtraNetFolder)
        6    0.000    0.000    0.247    0.041 
CatalogMultiplex.py:19(indexObject)
       30    0.000    0.000    0.199    0.007 __init__.py:21(notify)
signature.asc (application/pgp-signature, 186 B)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFESJB4CJIWIbr9KYwRAo+XAKDY01umXDUrv0ED/Wwt5mMIPftFwACg4ISN
XqpldSwkWykvfyGvfy+zxQ8=
=5faq
-----END PGP SIGNATURE-----
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.