FileSystemStorage / CMFEditions integration

Sylvain Thénault <[email protected]> Fri, 16 Jun 2006 11:25:00 +0200
Newsgroups gmane.comp.web.zope.plone.versioning,gmane.comp.web.zope.plone.archetypes.devel
Message-ID <[email protected]>
Hi there !

This morning I took the time to extract proper patches that I had have
to write to have FileSystemStorage fields working properly with
CMFEditions versioning. For this I had to patch:

* CMFEditions, to:
  * give a rollback flag to afterRetrieveModifier, allowing to know
    in a modifier if an object is retieved for a rollback or not (simple
    view of an old version)
  * handle a .retrieved_version_id attribute on versioned object to know
    the cmfedition version number of an object (at the time at wrote the
    patch, I didn't find any attribute giving this information as I'ld
    have expected, version_id wasn't suiting my need)

* FileSystemStorage, to:
  * handle version number in generated file name
  * store fssinfo into different attribute according to object's version
    (I don't know enough about cmfeditions'internals to explain that,
    but I had have to do this to properly deals with file deletion and
    maybe other problems i don't remember right now)

* Archetypes'file widget macro to include version number into its
  download link

and finally to write a modifier, dealing with filesystem files on
version creation/rollback.

As you can see, the problem is that it introduces some cmfeditions
related code into FSS and Archetypes. However this is in restricted
place and doesn't cause any problem if cmfeditions is not used or even 
installed, and this integration seems to me potentially interesting for
others, so I wish to integrate it into the main stream of those 3
products. Is someone object on this or have any comment regarding the
patches? (see attached files) 

regards,
-- 
Sylvain Thénault                               LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
Développement logiciel sur mesure:       http://www.logilab.fr/services
Python et calcul scientifique:           http://www.logilab.fr/science

_______________________________________________
Collective-versioning mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/collective-versioning
cmfedition_fss.patch (text/plain, 7.8 KB)
Index: ArchivistTool.py
===================================================================
--- ArchivistTool.py	(révision 24858)
+++ ArchivistTool.py	(copie de travail)
@@ -299,13 +299,13 @@
                             prepared_obj.metadata)
             
     security.declarePrivate('retrieve')
-    def retrieve(self, obj=None, history_id=None, selector=None, preserve=()):
+    def retrieve(self, obj=None, history_id=None, selector=None, preserve=(), rollback=False):
         """
         """
         # retrieve the object by accessing the right history entry
         # the histories storage called by LazyHistory knows what to do
         # with a None selector
-        history = self.getHistory(obj, history_id, preserve)
+        history = self.getHistory(obj, history_id, preserve, rollback)
         try:
             return history[selector]
         except StorageRetrieveError:
@@ -314,11 +314,11 @@
                 % (obj, selector))
     
     security.declarePrivate('getHistory')
-    def getHistory(self, obj=None, history_id=None, preserve=()):
+    def getHistory(self, obj=None, history_id=None, preserve=(), rollback=False):
         """
         """
         try:
-            return LazyHistory(self, obj, history_id, preserve)
+            return LazyHistory(self, obj, history_id, preserve, rollback)
         except StorageUnregisteredError:
             raise ArchivistUnregisteredError(
                 "Retrieving a version of an unregistered object is not "
@@ -389,7 +389,7 @@
     """
     __implements__ = (IHistory, )
     
-    def __init__(self, archivist, obj, history_id, preserve=()):
+    def __init__(self, archivist, obj, history_id, preserve=(), rollback=False):
         """Sets up a lazy history. Takes an object which should be the original object
            in the portal, and a history_id for the storage lookup.  If the history id
            is omitted then the history_id will be determined by dereferencing the obj.
@@ -400,6 +400,7 @@
         storage = getToolByName(archivist, 'portal_historiesstorage')
         self._obj, history_id = dereference(obj, history_id, archivist)
         self._preserve = preserve
+        self._rollback = rollback
         self._history = storage.getHistory(history_id)
         
     def __len__(self):
@@ -433,7 +434,8 @@
         # 6. call the after retrieve modifier
         refs_to_be_deleted, attr_handling_references, preserved_data = \
             self._modifier.afterRetrieveModifier(self._obj, repo_clone, 
-                                                 self._preserve)
+                                                 self._preserve,
+                                                 self._rollback)
         
         return VersionData(data, refs_to_be_deleted, 
                            attr_handling_references, preserved_data, 
Index: CopyModifyMergeRepositoryTool.py
===================================================================
--- CopyModifyMergeRepositoryTool.py	(révision 24858)
+++ CopyModifyMergeRepositoryTool.py	(copie de travail)
@@ -72,6 +72,18 @@
          'enable': 'enablePolicyOnTypeHook',
          'disable': 'disablePolicyOnTypeHook'}
 
+def _notify_new_version(obj):
+    """notify version by calling an external new_version_created script
+    or method. This may be acquired or not exists.
+    """
+    try:
+        obj.new_version_created()
+    except AttributeError:
+        import traceback
+        traceback.print_exc()
+        pass
+
+
 class CopyModifyMergeRepositoryTool(UniqueObject,
                                     SimpleItem,
                                     ActionProviderBase):
@@ -259,6 +271,8 @@
         self._recursiveSave(obj, metadata,
                             self._prepareSysMetadata(comment),
                             autoapply=True)
+        _notify_new_version(obj)
+        obj.retrieved_version_id = None
 
     security.declarePublic('save')
     def save(self, obj, comment='', metadata={}):
@@ -268,7 +282,9 @@
         self._recursiveSave(obj, metadata,
                             self._prepareSysMetadata(comment),
                             autoapply=self.autoapply)
-
+        _notify_new_version(obj)
+        obj.retrieved_version_id = None
+        
     security.declarePublic('revert')
     def revert(self, obj, selector=None):
         """See interface.
@@ -284,6 +300,7 @@
             obj._setId(original_id)
             #parent.manage_renameObject(obj.getId(), original_id)
             #parent._setObject(original_id, obj, set_owner=0)
+        obj.retrieved_version_id = None
 
         # run fixups
         self._doInplaceFixups(fixup_queue, True)
@@ -293,8 +310,10 @@
         """See interface.
         """
         self._assertAuthorized(obj, AccessPreviousVersions, 'retrieve')
-        return self._retrieve(obj, selector, preserve)
-
+        retrieved = self._retrieve(obj, selector, preserve)
+        obj.retrieved_version_id = None
+        return retrieved
+    
     security.declarePublic('restore')
     def restore(self, history_id, selector, container, new_id=None):
         """See interface.
@@ -437,7 +458,8 @@
         if hasBeenDeleted:
             # if the object to retreive doesn't have a counterpart in the tree
             # build a new one before retrieving an old state
-            vdata = portal_archivist.retrieve(obj=obj, history_id=history_id, selector=selector, preserve=preserve)
+            vdata = portal_archivist.retrieve(obj=obj, history_id=history_id, selector=selector, preserve=preserve,
+                                              rollback=inplace)
             repo_clone = vdata.data.object
             obj = portal_reffactories.invokeFactory(repo_clone, source)
             hasBeenMoved = False
@@ -458,14 +480,15 @@
 
         if hasBeenMoved:
             if getattr(aq_base(source), obj.getId(), None) is None:
-                vdata = portal_archivist.retrieve(obj=obj, history_id=history_id, selector=selector, preserve=preserve)
+                vdata = portal_archivist.retrieve(obj=obj, history_id=history_id, selector=selector, preserve=preserve,
+                                                  rollback=inplace)
                 repo_clone = vdata.data.object
                 obj = portal_reffactories.invokeFactory(repo_clone, source)
             else:
                 # What is the desired behavior
                 pass
 
-        vdata = portal_archivist.retrieve(obj, history_id, selector, preserve)
+        vdata = portal_archivist.retrieve(obj, history_id, selector, preserve, rollback=inplace)
 
         # Replace the objects attributes retaining identity.
         _missing = object()
Index: ModifierRegistryTool.py
===================================================================
--- ModifierRegistryTool.py	(révision 24858)
+++ ModifierRegistryTool.py	(copie de travail)
@@ -234,7 +234,7 @@
         return inside_crefs, outside_crefs
     
     security.declarePrivate('afterRetrieveModifier')
-    def afterRetrieveModifier(self, obj, repo_clone, preserve=[]):
+    def afterRetrieveModifier(self, obj, repo_clone, preserve=[], rollback=False):
         """See IModifier
         """
         # before letting the after retrieve modifiers replace 
@@ -252,7 +252,11 @@
         refs_to_be_deleted = []
         attrs_handling_subobjects = []
         for ignored_id, mod in self._collectModifiers(obj, ISaveRetrieveModifier, reversed=True):
-            to_be_del, attrs, preserve = mod.afterRetrieveModifier(obj, repo_clone)
+            try:
+                to_be_del, attrs, preserve = mod.afterRetrieveModifier(obj, repo_clone, rollback=rollback)
+            except TypeError:
+                # doesn't support the new rollback argument ?
+                to_be_del, attrs, preserve = mod.afterRetrieveModifier(obj, repo_clone)
             refs_to_be_deleted.extend(to_be_del)
             attrs_handling_subobjects.extend(attrs)
             preserved.update(preserve)
fss_cmfedition.patch (text/plain, 2.8 KB)
Index: FileSystemStorage.py
===================================================================
--- FileSystemStorage.py	(révision 24859)
+++ FileSystemStorage.py	(copie de travail)
@@ -409,16 +415,28 @@
     security = ClassSecurityInfo()
     
     security.declarePrivate('getFSSInfoVarname')
-    def getFSSInfoVarname(self, name):
-        """ """
-        
-        return '%s_filesystemstorage_info' % name
+    def getFSSInfoVarname(self, name, instance=None):
+        """return the filessystemstorage_info variable name according
+        to version number
+        """
+        if instance is None:
+            return '%s_filesystemstorage_info' % name
+        version_id = getattr(instance, 'retrieved_version_id', None)
+        if version_id is None:
+            try:
+                request = instance.REQUEST
+                version_id = int(request.form['version'])
+            except (AttributeError, KeyError, ValueError), ex:
+                pass
+        if version_id is None:
+            return '%s_filesystemstorage_info' % name
+        return '%s_filesystemstorage_info_v%s' % (name, version_id)
     
     security.declarePrivate('getFSSInfo')
     def getFSSInfo(self, name, instance, **kwargs):
         """Get fss info"""
         
-        info_varname = self.getFSSInfoVarname(name)
+        info_varname = self.getFSSInfoVarname(name, instance)
         return getattr(aq_base(instance), info_varname, None)
     
     
@@ -426,15 +444,15 @@
     def delFSSInfo(self, name, instance, **kwargs):
         """Delete fss info attribute"""
         
-        info_varname = self.getFSSInfoVarname(name)
+        info_varname = self.getFSSInfoVarname(name, instance)
         delattr(aq_base(instance), info_varname)
     
     security.declarePrivate('setFSSInfo')
     def setFSSInfo(self, name, instance, value, **kwargs):
         """Set new value in fss info"""
         
-        info_varname = self.getFSSInfoVarname(name)
+        info_varname = self.getFSSInfoVarname(name, instance)
         field = self.getField(name, instance, **kwargs)

         # Check types
@@ -526,9 +544,18 @@
     def getFilename(self, name, instance, **kwargs):
         """
         """
        
         uid = kwargs.get('uid', instance.UID())
-        return '%s_%s' % (uid, name)
+        version_id = getattr(instance, 'retrieved_version_id', None)
+        if version_id is None:
+            try:
+                request = instance.REQUEST
+                version_id = int(request.form['version'])
+            except (AttributeError, KeyError, ValueError), ex:
+                pass
+        if version_id is None:
+            return '%s_%s' % (uid, name)
+        return '%s_%s_v%s' % (uid, name, version_id)
 
     security.declarePrivate('getFilePath')
     def getFilePath(self, name, instance, **kwargs):
archetypes_cmfedition+fss.patch (text/plain, 497 B)
22,23c22,25
<           <tal:link condition="showLink">
<               <a href="" tal:attributes="href string:${here/absolute_url}/at_download/$fieldName">
---
>           <tal:link condition="showLink"
>                     define="version here/retrieved_version_id | nothing;
>                             versionstr python: version is not None and '?version=%s' % version or '';">
>               <a href="" tal:attributes="href string:${here/absolute_url}/at_download/$fieldName$versionstr">
FSSEditionsModifier.py (text/x-python, 3.9 KB)
# Copyright (c) 2005-2006 LOGILAB S.A. (Paris, FRANCE).
# http://www.logilab.fr/ -- mailto:[email protected]
#
# This program is free software; you can redistribute it and/or modify it under
# the terms of the CECILL license, available at
# http://www.inria.fr/valorisation/logiciels/Licence.CeCILL-V1.pdf
#

from os.path import exists

from Products.CMFCore.utils import getToolByName

from Products.CMFEditions.interfaces.IModifier import ISaveRetrieveModifier
from Products.CMFEditions.Modifiers import ConditionalModifier, manage_addModifierForm

from Products.FileSystemStorage.FileSystemStorage import FileSystemStorage
from Products.FileSystemStorage.FileUtils import copy_file

def fss_fields(obj):
    """return a list of fields using FileSystemStorage on the object"""
    try:
        schema = obj.Schema()
    except AttributeError:
        return ()
    return [f for f in schema.fields() if isinstance(f.storage, FileSystemStorage)]

class HandleFileSystemStorage:
    """Modifier to handle AT objects using FileSystemStorage
    """

    __implements__ = (ISaveRetrieveModifier, )

    def beforeSaveModifier(self, obj, clone):
        # check we have some fields to consider on the object
        fields = fss_fields(obj)
        if not fields:
            return [], []
        archivist = getToolByName(obj, 'portal_archivist')
        new_version_id = len(archivist.queryHistory(obj))
        # copy fields
        clone.retrieved_version_id = new_version_id
        for field in fields:
            name = field.getName()
            working_path = field.storage.getFilePath(name, obj)
            if not exists(working_path):
                continue
            clone_path = field.storage.getFilePath(name, clone)
            #print 'copying', working_path, 'to', clone_path
            copy_file(working_path, clone_path)
            # XXX I've to dump fss info manually for some obscur reason related
            # to file deletion...
            info_varname = field.storage.getFSSInfoVarname(name, obj)
            try:
                info = getattr(obj, info_varname)
            except AttributeError, ex:
                continue
            infocopy = info.__class__(**info.__dict__)
            setattr(clone, field.storage.getFSSInfoVarname(name, clone), infocopy)
            setattr(obj, field.storage.getFSSInfoVarname(name, clone), infocopy)
        return [], []

    def afterRetrieveModifier(self, obj, repo_clone, preserve=(), rollback=False):
        # check if the modifier is called with a valid working copy
        if obj is None:
            return [], [], {}
        # check we have some fields to consider on the object
        fields = fss_fields(obj)
        if not fields:
            return [], [], {}
        repo_clone.retrieved_version_id = repo_clone.version_id
        obj.retrieved_version_id = None
        if rollback:
            for field in fields:
                clone_path = field.storage.getFilePath(field.getName(), repo_clone)
                if not exists(clone_path):
                    continue
                working_path = field.storage.getFilePath(field.getName(), obj)
                copy_file(clone_path, working_path)
        return [], [], {}


def manage_addHandleFileSystemStorage(self, id, title=None, REQUEST=None):
    """HandleFileSystemStorage modifier factory"""
    modifier = HandleFileSystemStorage()
    self._setObject(id, ConditionalModifier(id, modifier, title))
    if REQUEST is not None:
        REQUEST['RESPONSE'].redirect(self.absolute_url()+'/manage_main')

MODIFIERS = ( 
    {
        'id': 'HandleFileSystemStorage',
        'title': "Handle AT objects with fields using a FileSystemStorage.",
        'enabled': True,
        'wrapper': ConditionalModifier,
        'modifier': HandleFileSystemStorage,
        'form': manage_addModifierForm,
        'factory': manage_addHandleFileSystemStorage,
        'icon': 'www/modifier.gif',
    },
)