Re: Tree rest resource and sorting

That's a great outcome.  Thanks for sticking with this, David!

On Tue, Jun 28, 2011 at 1:00 PM, David Steinberger <[email protected]>wrote:

> **
>
>  Gentlemen,
>
> That issue should be solved. Blake from *Two Toasters*** pushed 2 changes
> that allow us to preserve the order of the tree.
>
> Commits:
>
> https://github.com/twotoasters/RestKit/commit/0949337749e3152c87ab32743f07c8cc5910575c
>
> https://github.com/twotoasters/RestKit/commit/851df3bcbaa624a3845a5447eb391167d347043c
>
> I used it in my app in the following way:
> I added a category on the model that injects the index when it's dealing
> with the tree object.
> Need to do some more testing, but looks solid.
>
> @implementation RKObjectLoaderTTModel (positionInAlbum)
>
>
> #pragma mark RKModelLoaderDelegate
>
> - (void)objectLoader:(RKObjectLoader*)loader willMapData:(inout id
> *)mappableData {
>      if (loader.objectMapping.objectClass == [RKMTree class]) {
>          NSArray* origEntities = [*mappableData valueForKeyPath:@"entity"];
>
>          NSMutableArray* newEntity = [[NSMutableArray alloc]
> initWithCapacity:[origEntities count]];
>
>          int i = 0;
>          for (NSDictionary* origEntity in origEntities) {
>              NSMutableDictionary* oneEntity = [origEntity mutableCopy];
>
>              // inject the position in the array
>              NSMutableDictionary* entity = ([(NSDictionary*)[oneEntity
> objectForKey:@"entity"] mutableCopy]);
>              [entity setObject:[NSString stringWithFormat:@"%i", i]
> forKey:@"positionInAlbum"];
>
>              [oneEntity removeObjectForKey:@"entity"];
>              [oneEntity setObject:entity forKey:@"entity"];
>
>              [newEntity addObject:oneEntity];
>              [oneEntity release];
>              [entity release];
>              i++;
>          }
>
>          [*mappableData removeObjectForKey:@"entity"];
>          [*mappableData setObject:newEntity forKey:@"entity"];
>          [newEntity release];
>      }
>  }
>
>
> @end
>
>
>
> **On 6/28/11 7:04 AM, Bharat Mediratta wrote:
>
>
> On Mon, Jun 27, 2011 at 4:06 AM, David Steinberger <[email protected]>wrote:
>
>> Can u add me to the list of frustrated users?
>> When, after weeks of waiting, someone suggests a module as THE solution
>> ... well, that hurts too.
>> K, let's not get stuck here.
>>
>
>  Yes, I think that's totally fair.  I don't think that it's appropriate to
> do this in a module :-/
>
>
>>
>> I have my objections against the injection of elements into the payload
>> but it's worth spending some time on.
>> I'll sit down tonight and give it a try.
>>
>
>  Thanks.  If that doesn't pan out, I promise you I'll sit down and write
> the code to inject it on the G3 side and we'll all move on.
>
>
>>
>> -------- Original-Nachricht --------
>> > Datum: Sun, 26 Jun 2011 22:12:12 -0700
>> > Von: Bharat Mediratta <[email protected]>
>> > An: David Steinberger <[email protected]>
>> > CC: [email protected]
>> > Betreff: Re: [Gallery-devel] Tree rest resource and sorting
>>
>>  > On Sun, Jun 26, 2011 at 2:39 PM, David Steinberger
>> > <[email protected]>wrote:
>> >
>> > > **
>> > >
>> > >  Regarding injection of order field:
>> > > AFAIK it's not possible. We can register a delegate (see
>> > RKObjectMapper.m)
>> > > and get notified before the mapping operation begins, but RK continues
>> > to
>> > > map in the background.
>> > > So long story short:  Haven't found a good possibility for that.
>> > >
>> >
>> > Ok, so I see at
>> >
>> https://github.com/twotoasters/RestKit/blob/master/Code/ObjectMapping/RKObjectMapper.m#L208that
>> > it makes a call to
>> > objectMapperWillBeginMapping:self on the delegate. I don't fully
>> > understand
>> > the model there, but isn't that synchronous? It seems like at that point
>> > we
>> > could grab the sourceObject and do a traversal.
>> >
>> >
>> > > My main objection here is that this manipulation is a heavy operation
>> > for
>> > > the client.
>> > > So I think the query param would be best (but cannot really judge how
>> > much
>> > > effort this is on gallery3 side).
>> > >
>> >
>> > Is it that expensive?  To update tree_rest it's an O(N) operation across
>> > an
>> > ordered list, right?  I'd have to spend a lot more time with the code
>> than
>> > I
>> > have.
>> >
>> > My (and Tim's) frustration with this is that this limitation in the iOS
>> > code
>> > is really bone-headed.  It's frustrating for us to have to work around
>> it
>> > on
>> > the server side when it makes no sense at all to me that they would make
>> > this design decision.  We pass down an ordered list and the library
>> stores
>> > it in an unordered fashion -- they're throwing out relevant data.
>>  Putting
>> > the data back in again is doable, but I feel like we're going to have to
>> > repeat this exercise over and over again until they fix their API.
>>  Sorry
>> > that this is dragging on so long, but I'm sure there is *some*
>> alternative
>> > here.
>> >
>> >
>> > >
>> > > FYI: The tutorial is way outdated, so don't rely on that. Get their
>> > source
>> > > from Github (https://github.com/twotoasters/RestKit) and check the
>> > > examples inside. The handling is now much easier than back in Feb.
>> > RestKit
>> > > rulez :)!
>> > >
>> > >  On 6/26/11 9:09 PM, Bharat Mediratta wrote:
>> > >
>> > >
>> > >  Ok, so the problem so far are:
>> > > 1) If we don't provide ordering, the iOS client won't work properly
>> > > 2) If we provide ordering in the core app it breaks our abstractions
>> > >  3) If we provide ordering in a 3rd party module then it'll be a pain
>> > and
>> > > we'll lose users
>> > >
>> > >  I suspect that doing #2 with a query param is the way to go.  But
>> I've
>> > > been looking at the RestKit code and docs, specifically
>> > > http://mobile.tutsplus.com/tutorials/iphone/restkit_ios-sdk/ -- what
>> I'm
>> > > trying to figure out is if we can intercept the JSON response on that
>> > side
>> > > and inject the order field in the client, then parse the JSON.  If we
>> > could
>> > > do that, it would solve all problems.  David, is that doable?
>> > >
>> > > On Sun, Jun 26, 2011 at 12:34 AM, David Steinberger
>> > <[email protected]>wrote:
>> > >
>> > >>
>> > >>  I really don't think this should go into a module. Reasons:
>> > >>
>> > >>    - We'd replicate existing functionality (heavily).
>> > >>    - From a mobile client we always have to check that the module is
>> > >>    installed, if not have to tell the user to contact an admin for
>> that
>> > ...
>> > >>    this needs to work out of the box
>> > >>    - Regarding *Apple*: So far it's vendor specific. But we might ran
>> > >>    into this on other platforms as well.
>> > >>    We just don't know this as of now.
>> > >>
>> > >> How about adding this as a query parameter?
>> > >> So those who need it will get it, those who don't will get the same
>> as
>> > of
>> > >> today?
>> > >>
>> > >>   On 6/26/11 4:32 AM, Tim Almdal wrote:
>> > >>
>> > >> God forbid, we add this to every entity... I'm opposed to adding
>> custom
>> > >> anything to make something work for one vendor...  Add it as a module
>> > and
>> > >> then stuff it into -contrib, so those of us that never use Apple
>> don't
>> > have
>> > >> to carry this baggage around.
>> > >>
>> > >> On 6/25/2011 11:45 AM, David Steinberger wrote:
>> > >>
>> > >>
>> > >> Yeah you're right: My patch is not sufficient. If we could add this
>> to
>> > >> every entity that would be much better.
>> > >>
>> > >> The reason why I pushed so hard for tree_rest was that here a missing
>> > >> sorting attribute *destroys* the whole concept of core data. I have *
>> > >> Entity* modeled as a core data entity and relate it to the item
>> entity
>> > as
>> > >> well as to the tree-entity. However when I retrieved the tree entity
>> > the
>> > >> sorting was gone.
>> > >>
>> > >> Quick question: I realized that my sorting attribute has another
>> issue.
>> > I
>> > >> just gave the *parent* an id of -1.
>> > >> But an album can also be seen as a member of a parent-album at a
>> > specific
>> > >> index. So my sorting attribute change. How could this be prevented
>> > (although
>> > >> that's not a deal breaker for me)?
>> > >>
>> > >>  On 6/25/11 8:31 PM, Bharat Mediratta wrote:
>> > >>
>> > >>
>> > >>  Ok, I'm convinced that Apple needs this.  :-)  But won't your patch
>> be
>> > >> somewhat insufficient?  From my reading, you're going to have this
>> > problem
>> > >> with any ordered list that we send back, right?  Instead of putting
>> > this
>> > >> into tree_rest, would it make sense to add a new query param like
>> > >> "add_indexes" and then have rest::reply walk the entire structure
>> it's
>> > about
>> > >> to return and add an "_index" field to every entity in every list
>> that
>> > it
>> > >> finds?  I could probably bang that out pretty quickly, if you think
>> > that
>> > >> there's a need for it beyond tree_rest.
>> > >>
>> > >> On Mon, Jun 6, 2011 at 9:40 AM, David Steinberger
>> > <[email protected]>wrote:
>> > >>
>> > >>> Best doc is the one from Apple:
>> > >>>
>> > >>>
>> >
>> http://developer.apple.com/library/mac/#documentation/Cocoa/Conceptual/CoreData/Articles/cdFAQ.html#//apple_ref/doc/uid/TP40001802-CJBDBHCB
>> > >>>
>> > >>> See 'How do I fetch objects in the same order I created them?' about
>> > >>> collections and core data.
>> > >>>
>> > >>> I'm using core data 90% via RestKit:
>> > >>> https://github.com/twotoasters/RestKit
>> > >>>
>> > >>> If you want checkout the example 'RKTwitterCoreData' to see how it
>> > works
>> > >>> (branch 67-object-mapping-2.0).
>> > >>> I can also send a mockup how I mapped gallery3 json responses (tree
>> &
>> > >>> item).
>> > >>>
>> > >>> -------- Original-Nachricht --------
>> > >>> > Datum: Sun, 5 Jun 2011 16:44:49 -0700
>> > >>> > Von: Bharat Mediratta <[email protected]>
>> > >>> > An: David Steinberger <[email protected]>
>> > >>> > CC: [email protected]
>> > >>> > Betreff: Re: [Gallery-devel] Tree rest resource and sorting
>> > >>>
>> > >>> > On Sun, Jun 5, 2011 at 3:44 PM, David Steinberger
>> > >>> > <[email protected]>wrote:
>> > >>> >
>> > >>> > >
>> > >>> > > Right after I sent the message I realized that I'd be beaten up
>> by
>> > >>> that.
>> > >>> > >
>> > >>> > > So let me give details: Core Data doesn't store NSArray, but
>> > NSSet:
>> > >>> > >
>> > >>> > >
>> > >>> >
>> > >>>
>> >
>> http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foundation/Classes/NSSet_Class/Reference/Reference.html
>> > >>> > >
>> > >>> > > Yes it's platform specific. but right now iOS is the most common
>> > one.
>> > >>> > > Core Data is the best way to serialize objects (IMHO) and I
>> really
>> > >>> want
>> > >>> > to
>> > >>> > > use it.
>> > >>> > >
>> > >>> >
>> > >>> > Ok.  I'm reading through the Core Data docs, but I'm new to it and
>> > >>> there
>> > >>> > are
>> > >>> > a ton of docs out there.  Can you point me at relevant docs that
>> > talk
>> > >>> > about
>> > >>> > how Core Data does its underlying storage?
>> > >>> >
>> > >>> > I assume that at some point you're going to coerce the NSSet into
>> an
>> > >>> > NSArray
>> > >>> > by applying a sort order over the given field.  But it's a little
>> > >>> > surprising
>> > >>> > that the initial ordering isn't preserved somewhere...
>> > >>> >
>> > >>> >
>> > >>> > >
>> > >>> > > Why should it be wrong to indicate the sort by an attribute?
>> > >>> > > It doesn't impact the server-side at all.
>> > >>> > >
>> > >>> >
>> > >>> > But it doesn't belong on the server side, either.  We're already
>> > >>> sending
>> > >>> > an
>> > >>> > ordered list -- adding an ordering field is redundant.  This is
>> > really
>> > >>> a
>> > >>> > limitation on the client side and it should be fixed there.  Yes,
>> > I'm
>> > >>> > being
>> > >>> > a bit of a purist but I hate crapping up our APIs to handle edge
>> > cases.
>> > >>> > If
>> > >>> > this is the only way to do it, I'll be grumpy but I'll fiddle with
>> > the
>> > >>> > server side code.  But I'd like to be convinced that there  isn't
>> > some
>> > >>> way
>> > >>> > to do it in Core Data.  Surely this must be a common use case.
>> > >>> >
>> > >>> >
>> > >>> > >
>> > >>> > >
>> > >>> > > On 6/6/11 12:16 AM, Bharat Mediratta wrote:
>> > >>> > >
>> > >>> > >
>> > >>> > >  Hm.  An array is an ordered list, and I would expect NSArray to
>> > >>> > maintain
>> > >>> > > the ordering --
>> > >>> > >
>> > >>> >
>> > >>>
>> >
>> http://developer.apple.com/library/mac/#documentation/Cocoa/Reference/Foundation/Classes/NSArray_Class/NSArray.htmlseems
>> > >>> > > to bear that out.  Are you seeing it lose the ordering?
>> > >>> > >
>> > >>> > >  Even if it was, I'd expect that this is a platform specific
>> > problem
>> > >>> so
>> > >>> > > I'd expect it'd be something you'd fix on your side by updating
>> > the
>> > >>> > entity
>> > >>> > > after you load it.  Doing it on the server side smells wrong to
>> > me.
>> > >>> > >
>> > >>> > >  Thoughts?
>> > >>> > >
>> > >>> > > On Wed, Jun 1, 2011 at 10:55 AM, David Steinberger
>> > >>> > <[email protected]>wrote:
>> > >>> > >
>> > >>> > >>  Team,
>> > >>> > >>
>> > >>> > >> I'm reworking the data model of my iPhone app.
>> > >>> > >> One thing I want to revisit is how I deal with the tree rest
>> > >>> resource
>> > >>> > >> (tree_rest.php).
>> > >>> > >>
>> > >>> > >> Reason:
>> > >>> > >> I know that the tree_rest.php delivers the objects in the right
>> > >>> order,
>> > >>> > but
>> > >>> > >> this doesn't help me much.
>> > >>> > >> I need to store and retrieve the array and by doing such an
>> > >>> operation
>> > >>> > with
>> > >>> > >> an array the order might be lost.
>> > >>> > >>
>> > >>> > >> Proposed solution:
>> > >>> > >> The patch attached adds a relative_position attribute to each
>> > >>> entity.
>> > >>> > Via
>> > >>> > >> this attribute the entity order can be reconstructed at any
>> time.
>> > >>> > There's no
>> > >>> > >> impact on performance and overall structure.
>> > >>> > >>
>> > >>> > >> Please review and merge to master or propose something even
>> > better.
>> > >>> > >>
>> > >>> > >> Thanks,
>> > >>> > >>   David
>> > >>> > >>
>> > >>> > >> ---
>> > >>> > >>
>> > >>> > >> FYI Details:
>> > >>> > >> I use core data to cache all objects (tree/album, items, tags,
>> > ...).
>> > >>> > Core
>> > >>> > >> data is kind of a database for objects on iOS. The tree is
>> > modeled
>> > >>> via
>> > >>> > a
>> > >>> > >> Class that has a NSString property that holds the url (that's
>> the
>> > >>> > primary
>> > >>> > >> key) and a NSArray that holds all the entities.
>> > >>> > >> But when I retrieve the object-graph iOS applies any sorting.
>> ==>
>> > A
>> > >>> > >> sort-attribute is needed.
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> > >>
>> > >>> >
>> > >>>
>> >
>> ------------------------------------------------------------------------------
>> > >>> > >> Simplify data backup and recovery for your virtual environment
>> > with
>> > >>> > >> vRanger.
>> > >>> > >> Installation's a snap, and flexible recovery options mean your
>> > data
>> > >>> is
>> > >>> > >> safe,
>> > >>> > >> secure and there when you need it. Data protection magic?
>> > >>> > >> Nope - It's vRanger. Get your free trial download today.
>> > >>> > >> http://p.sf.net/sfu/quest-sfdev2dev
>> > >>> > >> __[ g a l l e r y - d e v e l ]_________________________
>> > >>> > >>
>> > >>> > >> [ list info/archive --> http://gallery.sf.net/lists.php ]
>> > >>> > >> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>> > >>> > >>
>> > >>> > >
>> > >>> > >
>> > >>>
>> > >>>  --
>> > >>> NEU: FreePhone - kostenlos mobil telefonieren!
>> > >>> Jetzt informieren: http://www.gmx.net/de/go/freephone
>> > >>>
>> > >>>
>> > >>
>> > >>
>> >
>> ------------------------------------------------------------------------------
>> > >> All the data continuously generated in your IT infrastructure
>> contains
>> > a
>> > >> definitive record of customers, application performance, security
>> > >> threats, fraudulent activity and more. Splunk takes this data and
>> makes
>> > >> sense of it. Business sense. IT sense. Common sense..
>> > http://p.sf.net/sfu/splunk-d2d-c1
>> > >>
>> > >>
>> > >> __[ g a l l e r y - d e v e l ]_________________________
>> > >>
>> > >> [ list info/archive --> http://gallery.sf.net/lists.php ]
>> > >> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>> > >>
>> > >>
>> > >>
>> > >> No virus found in this message.
>> > >> Checked by AVG - www.avg.com
>> > >> Version: 10.0.1388 / Virus Database: 1513/3726 - Release Date:
>> 06/25/11
>> > >>
>> > >>
>> > >>
>> >
>> ------------------------------------------------------------------------------
>> > >> All the data continuously generated in your IT infrastructure
>> contains
>> > a
>> > >> definitive record of customers, application performance, security
>> > >> threats, fraudulent activity and more. Splunk takes this data and
>> makes
>> > >> sense of it. Business sense. IT sense. Common sense..
>> > http://p.sf.net/sfu/splunk-d2d-c1
>> > >>
>> > >>
>> > >> __[ g a l l e r y - d e v e l ]_________________________
>> > >>
>> > >> [ list info/archive --> http://gallery.sf.net/lists.php ]
>> > >> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>> > >>
>> > >>
>> > >>
>> > >>
>> >
>> ------------------------------------------------------------------------------
>> > >> All of the data generated in your IT infrastructure is seriously
>> > valuable.
>> > >> Why? It contains a definitive record of application performance,
>> > security
>> > >> threats, fraudulent activity, and more. Splunk takes this data and
>> > makes
>> > >> sense of it. IT sense. And common sense.
>> > >> http://p.sf.net/sfu/splunk-d2d-c2
>> > >> __[ g a l l e r y - d e v e l ]_________________________
>> > >>
>> > >> [ list info/archive --> http://gallery.sf.net/lists.php ]
>> > >> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>> > >>
>> > >
>> > >
>>
>>  --
>> NEU: FreePhone - kostenlos mobil telefonieren!
>> Jetzt informieren: http://www.gmx.net/de/go/freephone
>>
>>
>> ------------------------------------------------------------------------------
>>  All of the data generated in your IT infrastructure is seriously
>> valuable.
>> Why? It contains a definitive record of application performance, security
>> threats, fraudulent activity, and more. Splunk takes this data and makes
>> sense of it. IT sense. And common sense.
>> http://p.sf.net/sfu/splunk-d2d-c2
>> __[ g a l l e r y - d e v e l ]_________________________
>>
>> [ list info/archive --> http://gallery.sf.net/lists.php ]
>> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>>
>>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.http://p.sf.net/sfu/splunk-d2d-c2
>
>
> __[ g a l l e r y - d e v e l ]_________________________
>
> [ list info/archive --> http://gallery.sf.net/lists.php ]
> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>
>
>
> ------------------------------------------------------------------------------
> All of the data generated in your IT infrastructure is seriously valuable.
> Why? It contains a definitive record of application performance, security
> threats, fraudulent activity, and more. Splunk takes this data and makes
> sense of it. IT sense. And common sense.
> http://p.sf.net/sfu/splunk-d2d-c2
> __[ g a l l e r y - d e v e l ]_________________________
>
> [ list info/archive --> http://gallery.sf.net/lists.php ]
> [ gallery info/FAQ/download --> http://gallery.sf.net ]
>

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security 
threats, fraudulent activity, and more. Splunk takes this data and makes 
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2

__[ g a l l e r y - d e v e l ]_________________________

[ list info/archive --> http://gallery.sf.net/lists.php ]
[ gallery info/FAQ/download --> http://gallery.sf.net ]