Re: Trellis-fork

Oh yeah, one more thing. We could try to use deadlock-breaking and
given that automatic retries are OK with Trellis, it would work. But
lets just not do that. :)

On 2009-06-23, Sergey Schetinin <[email protected]> wrote:
> Next thing I want to do it introduce thread safety. There's some code
>  to help do that in Trellis already, but I plan to remove that
>  implement a very simple model. If I am not mistaken, Phillip planned
>  for cells to have locks in their manager attribute, so that
>  ctrl.lock(cell) uses that just like other managers, calling __enter__
>  / __exit__. For locks that acquires and releases them so, given
>  ctrl.lock is called in all appropriate places the access to cells is
>  synchronous.
>
>  The problem with this is that if different cells have different locks
>  as their managers the deadlocks are very likely. There's no way of
>  telling what cells will be used in a transaction and in what order so
>  two concurrent transactions are very likely to get into a deadlock.
>
>  If all cells have the same lock as their manager there are no
>  deadlocks, but the whole ctrl.lock() business is pointless as there's
>  only one lock to acquire and release.
>
>  Another possible case is when there are two or more sets of cells that
>  form disjoint networks. All cells in each of these sets would have the
>  same lock. This way there's a guarantee of no deadlocks, but the only
>  reason to use locks in this case is if there will be concurrent
>  transactions on the same cell network.
>
>  I think the best tradeoff for now is a GIL-like solution -- just
>  disable all this cell.manager / ctrl.lock stuff and just hold the
>  global lock in ctrl._process. This doesn't support the last case
>  described above, but let's first have an actual use case for that and
>  see how it could be supported then. One simple solution that could
>  work for that is to have a controller per cell group / network with
>  its own lock.
>
>  Another problem is how cells store their values -- they change them
>  in-place and log that for undo. The problem is that if that cell is
>  accessed from another thread, and from *outside* a transaction the
>  changed value will be seen -- that's not good at all. For example see
>  _ReadValue.get_value implementation:
>
>     def get_value(self):
>         if ctrl.active:
>             # if atomic, make sure we're locked and consistent
>             used(self)
>         return self._value
>
>  If not atomic there's no guarantee of getting valid value which isn't
>  easily resolved.
>
>  One way to fix that is to store changes in a different attribute and
>  read that only if inside transaction. A development of this approach
>  is to let the controller store all of cell values. This also makes way
>  for properly concurrent transactions without locking because the
>  controller can store the consistent data snapshot in one location and
>  let the transactions work on a copy. Each transaction would get its
>  own copy so there's no need to lock at this point. At commit time
>  however the changes need to be merged, and because all of value
>  storage is handled by controller it can verify at this point if other
>  transactions changed any inputs used by this transaction since it
>  started. (For more on this see
>  http://code.google.com/p/trellis-fork/wiki/Transactions ) Reads from
>  outside transactions would just get a value from the latest committed
>  snapshot.
>
>  The unknown in this approach is how fast is it for single threaded
>  applications. I don't see any reasons for it too especially slow, but
>  it could be a bit slower than current implementation. It might be
>  faster too though. If anyone could work on performance measuring for
>  Trellis I'd really appreciate that.
>
>  Also, one more thing to mention in regards to thread safety -- even
>  with the GIL-like fix in place the controller has to be thread-local
>  because even while there's only one concurrent transaction possible
>  not all operations on controller are transactional: ctrl.active checks
>  in other threads will happen outside transactions and need to return
>  False.
>
>
>  On 2009-06-22, Sergey Schetinin <[email protected]> wrote:
>  > I've just brought the ForkChanges page up to date.
>  >  http://code.google.com/p/trellis-fork/wiki/ForkChanges
>  >
>  >  Any feedback on changes (also the "future changes" section) is welcome.
>  >
>  >
>  >  Also the "Transaction" page have grown into a quite long overview of
>  >  what's similar between Trellis rules and concurrent transactions and
>  >  some ideas on how Trellis could evolve. Still incomplete, but about
>  >  2/3 done, I think. If you see any mistakes in terminology or factual
>  >  errors, please report.
>  >
>  > http://code.google.com/p/trellis-fork/wiki/Transactions
>  >
>  >
>  >
>  >
>  >
>  > On 2009-06-17, Sergey Schetinin <[email protected]> wrote:
>  >  > On 2009-06-12, Sergey Schetinin <[email protected]> wrote:
>  >  >  > This is a thread for trellis-fork updates and discussion. At least it
>  >  >  >  should suffice at first.
>  >  >  >
>  >  >  >  http://code.google.com/p/trellis-fork/
>  >  >  >  http://pypi.python.org/pypi/Trellis-fork
>  >  >
>  >  >
>  >  > I've already finished quite a lot of work and test finally pass (had
>  >  >  to make a few edits, see commit log) and the changes to the code
>  >  >  structure are bigger than I imagined at first, but I'm quite happy
>  >  >  with the state of things overall.
>  >  >
>  >  >  Anyway, I'll work on it some more before reporting on the progress, so
>  >  >  if you want to watch the changes as they happen see
>  >  >  http://code.google.com/p/trellis-fork/updates/list
>  >  >  or the rss feed: http://code.google.com/feeds/p/trellis-fork/updates/basic
>  >  >
>  >  >  The actual reason for this email is that I want to get some feedback
>  >  >  on an article I started to write, explaining the use of transactions
>  >  >  in Trellis, it's work in progress, but it's something I think that can
>  >  >  help a lot with understanding Trellis.
>  >  >
>  >  >  http://code.google.com/p/trellis-fork/wiki/Transactions
>  >  >  Does it help?
>  >  >  Should it be more verbose or the maybe the opposite?
>  >  >  Any factual mistakes, typos, hints on structure, things to mention etc?
>  >  >
>  >  >
>  >  >
>  >  >
>  >  >  --
>  >  >  Best Regards,
>  >  >  Sergey Schetinin
>  >  >
>  >  >  http://s3bk.com/ -- S3 Backup
>  >  >  http://word-to-html.com/ -- Word to HTML Converter
>  >  >
>  >
>  >
>  >  --
>  >  Best Regards,
>  >  Sergey Schetinin
>  >
>  >  http://s3bk.com/ -- S3 Backup
>  >  http://word-to-html.com/ -- Word to HTML Converter
>  >
>
>
>
> --
>
> Best Regards,
>  Sergey Schetinin
>
>  http://s3bk.com/ -- S3 Backup
>  http://word-to-html.com/ -- Word to HTML Converter
>


-- 
Best Regards,
Sergey Schetinin

http://s3bk.com/ -- S3 Backup
http://word-to-html.com/ -- Word to HTML Converter