[ZCM] [ZC] 2342/ 5 Comment "bug in Catalog.py metadata not updated with keyword index"

Issue #2342 Update (Comment) "bug in Catalog.py metadata not updated with keyword index"
 Status Rejected, Catalog/bug medium
To followup, visit:
  http://www.zope.org/Collectors/Zope/2342

==============================================================
= Comment - Entry #5 by perry on Jul 23, 2007 1:59 pm

I think the problem lies in the fact, that the

 if data.get(index, 0) != newDataRecord:

always returns False, cause the data.get(index,0) already returns the updated index-value. So 

 data[index] = newDataRecord never gets executed. 

If I do that statement unconditionally it works as exspected.

________________________________________
= Comment - Entry #4 by tseaver on Jul 23, 2007 10:31 am

Your application is storing a mutable (a Python list) in the
metadata, and then mutating it without triggering the persistence
machinery for the record object, which means that the change is
not written to the ZODB.

It should be possible to see the symptom without restarting Zope,
if you load the ZMI view enough times (and quickly enough) to
prevent reusing the same connection.

I wouldn't recommend it, but you could perhaps work around this
behavior by setting '_p_changed' to a true value on the "brain"
(the "record" object) after mutating the list.
________________________________________
= Comment - Entry #3 by perry on Jul 23, 2007 2:37 am

I don't want to update metadata only

Here is a more detailed description of our usecase:

In our catalog we have a keyword index registered_courses and a metadata field registered_courses. If a student registers a new course an event is triggered, which adds the new course to the index.  In our event-handler we have:


  res = students_catalog(id = student_id)
  if not res:
      return
  student_rec = res[0]
  registered_courses = getattr(student_rec,'registered_courses',None)
  if not registered_courses:
      registered_courses = []
  #import pdb;pdb.set_trace()
  if event_type == "sys_add_object":
      if course_id not in registered_courses:
          registered_courses.append(course_id)
      else:
          return
  elif registered_courses and event_type == "sys_del_object":
      removed = False
      while course_id in registered_courses:
           removed = True
           registered_courses.remove(course_id)
      if not removed:
           return
      record_data = {}
      for field in self.schema() + self.indexes():
          record_data[field] = getattr(student_rec, field)
      # Add the updated data:
      record_data.update(data)
      self.catalog_object(dict2ob(record_data), student_id)

where dict2obj does just that.

That updates the meta_data only on index_creation, and after a zope restart the meta_data for registered_courses only containes the first course.

For example after adding two courses the zmi view of the catalog record looks like:
 Metadata Contents: ...
 registered_courses  	 ['LAL311', 'LAL312']

 Index Contents:
 registered_courses  	 ['LAL311', 'LAL312']

after a Zope restart it is:

 Metadata Contents: ...
 registered_courses  	 ['LAL311']

 Index Contents:
 registered_courses  	 ['LAL311', 'LAL312']

which is certainly wrong.

I more information is needed I can provide this. 


________________________________________
= Reject - Entry #2 by tseaver on Jul 20, 2007 4:19 pm

 Status: Pending => Rejected

The metadata columns are not updated when reindexing only a
sing index (or a set of them).  Note that there is, in fact, no relationship at all between them;  if you want to update the
metadata, you have to update *everything*.


________________________________________
= Request - Entry #1 by perry on Jul 20, 2007 5:06 am

we use ZCatalogs as tables, to store data for rapid access and searching. Recently we added a keyword index to one table, and found that the metadata of the index is not updated, when a keyword is added.The name of the index and metadata is the same "registered_courses". I found in Catalog.py updateMetadata around line 306

        else:
            if data.get(index, 0) != newDataRecord:
                data[index] = newDataRecord

the if condition evaluates always to false, cause the data.get(index,0) accesses the data of the index and not of the metadata field as long as Zope was not restarted that worked fine. But after a restart the metadata field always only contained the first added keyword, the index
was still ok.
After I changed the above code to:

       else:
            if True or data.get(index, 0) != newDataRecord:
                data[index] = newDataRecord

==============================================================