Re: Pagination and sort design in ddd

The focus of the Domain in DDD is validation and business rules. Reading, data persistence and performance is not a primary concern. If you look at CQS or CQRS (Greg Young) you will see some differences.

Pagination and ensuring bite sized data chunks in general is a responsibility you don't get much guidance on in any of these discussions. For the most part I think it is an implementation detail they could stand to focus on a bit, but it does belong in the database layer. 

Young has an interesting take on Caches with AtomPub. But when data changes frequently it doesn't work very well. Using it for immutable data, which there is a lot more of than you might think, is a good option. AtomPub will stick to most caches.

When I feel the need for pagination I provide a few pre-digested filter types with a few set sort orders. That makes query design simple. eBay does it. And yes, paginate to a standard record count people can accept. I don't give options there. I also tend to pre-load the next page, and cache it as the user is reading the current page. 

For good simulation of high record counts, I also want to put a 100ms lag into data responders in the dev environment to encourage good data reading habits. I want to try this for my current project. The lag is calculated based on the row count. 3ms per row is a good average. But after 40 records returned in a chunk the lag goes UP a bit per record. This makes coders calculate their responsiveness budgets carefully. I want to try RabbitMQ to set up the lag. It is a penalty on top of everything else causing lag. We also monitor and automatically question the slowest queries regularly in code review meetings.

I also encourage datamarts for reporting if the storage overhead can be tolerated. Most people appreciate pre-aggregated data in star schemas once they have them. Cubes still rule in the younger executive suite. 

I am also avidly watching Young and his EventStore for signs of mature capacity. This thing will never replace databases, but it could do most of the heavy lifting for temporal analysis and OLAP one day.


--- In [email protected], "raymond_kolbe" <rkolbe@...> wrote:
>
> 
> 
> --- In [email protected], "eben_roux" <me@> wrote:>>
> Hello,> > I subscribe to the school of thought that believes one should
> not query the domain model.  If an aggregate can provide you with the
> required data then fine but one should not be fiddling with the domain
> model to satisfy querying.> > The solution alluded to by yourself and
> Rickard I believe is the correct one.> > If you want to go for a
> seperate bounded context that handles pagination/sorting then that is ok
> too.  If you are so inclined you can represent your sql statements using
> some query object (having a query object represent a raw sql statement
> is also fine in some cases).> > You could go as far as having this BC
> execute the query and store the paginated/sorted results in working set
> data structures that can simply return a generic structure for
> presentation purposes.  These would of course expire after some timeout
> and can then be cleaned out.> > Hope that helps.> > Eben>
> Hi all,
> I'm bringing this thread back from the dead in the interest of
> cataloging the implementation aspects of pagination/sort w/regard to DDD
> repositories (attempting to not break the UL under certain use-cases).
> Assuming that pagination is not part of the UL, but rather an aspect of
> performance/view concern, I understand the following implementation
> solutions to exist (including, but not limited to):
> 1) Return a Query Object and allow the client to add additional
> constraints. I would argue this goes against the UL for a BC and I would
> urge people to avoid when possible. This is a leaky abstraction into
> what the repository is responsible for in my opinion.
> 2) Return a Lazy Collection (for a lack of a better term). Essentially
> this looks and acts like a regular collection but it may choose to defer
> populating the  collection until the first item in the collection is
> requested. Alternatively, calling slice on the collection would return a
> specific portion of the collection, possibly returning another Lazy
> Collection.
> 3) Return a Paged Result obj. My understanding is that this would be a
> fully instantiated collection where getIterator returns the current
> slice and count returns the total collection sum. E.g. Paged Result
> objs. are a slice of the total collection so no lazy fetching is
> performed.
> 4) Return a full Collection and rely on caching on the client side.
> To give some context to how I got here I'll explain the problem I'm
> working on. I have an aggregate root called Assignment which contains an
> association to an Employee, Topic, and Records. The records provide us
> information about whether or not the employee is current with their
> topic training (Topic), and also provide an audit trail of when the
> topic was taken. Some  information is kept on the Assignment such as
> "required", "last taken on" and "status". Status and last taken on being
> VOs, telling us whether the employee is "Current", "Expired", or
> "Unknown".
> Anyway, the AssignmentRepository has a method called
> getRequiredAssignmentsForTopic($topicId). This is used in the domain by
> the web application (for viewing) and for a reminder system (for
> notifying of upcoming expiring topic training). In two use case we ask
> for all Assignments for $topicId (reminders and exporting to CSV, PDF,
> whatever), and in a third use case we want paginated results for the
> web.
> How do others handle this? I'm aware of other solutions as well but I
> would like to hear what others think.
>




------------------------------------