RE: Needs to know everything; was: RE: [AM] ZZZ

(Responding to Dagna)

First, sorry for the delay, was 400 miles from my machine
for a couple of days.

 EVERYONE NEEDS TO KNOW EVERYTHING!

>>> (Dagna)
>>> ... the dba may not see it as 'traditional' or 'agile', but as 'risk'.
> 
>> (Paul)
>> That's fair enough, as long as all risks are evaluated 
>> fairly, including the risk of being unable to respond to 
>> change in a timely manner when it needs to happen.
>
> DG: One way of reducing that risk would be to have a dba involved
> in the project (in my youth, I made sure my grandmother was well
> up to date in the latest egg-sucking techniques... <g>).

Agreed.

> While all the dbas I have ever encountered have been obliging,
> flexible, co-operative team players (was that a snigger at the
> back?),

It was, ignore them, they don't know their own faults...  ;-)

> some tend to refuse everything or set up great fences to be
> got over, to stop people messing up 'their' data. Or so I am
> told. ;-) But if you can get them involved (and get over the two
> standard responses - "you should have involved me sooner"
> and "why are you dragging me in? I don't need to be involved
> yet!"), then 'your' data is also 'theirs' - which helps a lot. If this
> sounds like I am anti-dba - apologies, I am not, and I do
> appreciate how much they have to do. But I have also suffered
> from their attitudes.

Yes, if the whole of 'systems' is acting as one diversified
team, things will in theory progress a lot better.  My take on
'Generalising Specialist' is that we should know at least the
basics of the other person's problems.  If we get the DBA
involved early, we can get an early idea of which issues
need to be discussed with DBA, and which issues we don't 
need to pass along for inspection and comment.

>> (Paul)

>> ... one of the first solutions was to encapsulate the data and
>> put it and behaviour behind interfaces, so that the ripple effect
>> was stopped.  That's now standard in programming, but not
>> yet in database....

> DG: This  one took some thought. And will do for some time, I think. 
>
> First thoughts - if it has been encapsulated, then surely that works
> both ways? Answer - no, it almost certainly doesn't. If a data
> change is needed, then encapsulation can protect those apps,
> calls and so on that don't use the new structure (assuming that the
> old one is still supported). If the underlying data remains the same,
> and maybe a new index or whatever is added, or a table split into
> master and detail, then the outside world needs to know nothing.
> But if the format or contents of a field changes, or the field is no
> longer used, or a new one appears, then there is an impact on
> the outside world.

'We' have the view that the data is there primarily to support the
applications.  I assume 'you' have the view that the data is there
primarily to support the business.  Whereupon 'we' would say 
"Yes, but the data supports the business by supporting the
appications that support the business".  In theory, there should
never be any change that reduces the types of information held
in the database except where the business no longer needs
that information - and by extension no longer needs the applications
that use that information.  Of course there will be granularity
probems... the bit of unwanted functionality may be hard to unpick
from wanted functionality.

If we consider the function of the encapsulating interface to be
mapping the information that an application needs or provides
to the data stored in the database, then in theory changing the
data structure, format, etc. should not cause changes that
ripple beyond the interface.  However, there will be exceptions.
Suppose the database needs extra information when an
application is adding new information to the database - information 
without which the database cannot store its new data 'correctly'.
Where this changes, the application now has a new requirement -
to supply this additional information.

I think this is an important way of looking at the situation; the
database is a stakeholder in the application development,
and conversely the application is a stakeholder in the database
development.  Each supplies requirements to the other.

> Especially if it is something like the infamous 'name field split'
> scenario, which may require a change to the way the apps
> present the name to the database for insertion and update.

Indeed.  Here the database is supplying requirements to the
application, probably as a result of requirements it received
from a different application.

> Second thought - how about views? But they are a form of
> encapsulation or the underlying tables, and the same applies.

Agreed.

> Third thought - there is an amount of pain that is going to occur
> somewhere, and 'you' pushed it 'our' way? That was mean of
> you. ;->

Hey, do you want hints on solutions too?  ;-)  If applications
developers hadn't done encapsulation first, 'you' might be
teaching 'us'.  The pain exists, the solutions seem to exist.  If
we're all part of the same team, we're being helpful, not mean.
(Should've been a spin doctor? ;-)  )

> (See also Function Point Analysis, if you ever can't sleep.
> Something that works (I have seen it work), but makes "BDUF
> incorporating every little thing that anyone may want to consider
> for the next ten years" look like prototyping... But it is a way of
> measuring the amount of work needed to achieve a design,
> and (it is a long time since I read the book, and I forget the
> details) if a set amount of work has to be done, then I guess
> it gets done somewhere.

Right.  Throw in another rule of thumb - if each bit of the work
gets done in the 'right place', then the system will be very
flexible.  Unfortunately there are one or two things that throw this
nice picture out of kilter (If we could do all reporting from the object
layer rather than direct from the database, many things would be
much easier...).

> I am going to ponder this one some more. A few train journeys,
> glasses of wine, sleepless nights (not that I *intend* to  lose
> sleep over it, but those nights  when I can't sleep anyway), then
> I will see if I can come up with something to float at dm-discuss,
> and bring it back here.

Sounds like a good idea.


<snip>

>> (Paul)
>> In a data-rich system, the structure of the data drives the 
>> problem, and though programmers would start with an object 
>> model, starting with a data model would give perfectly 
>> adequate results.  For behaviour-rich systems, the behaviour
>> of the objects is significant, and if we are to have any 
>> reasonable chance at building the system economically 
>> and in a way that can respond readily to change, then
>> correct apportionment of behaviour to objects is paramount.
>> the data needed is determined by the behaviour needed.
>> In control-rich systems, behaviour is also important, but 
>> we have the added complication that the behaviour of 
>> objects changes significantly with time.
> 
>> In reality, encapsulation of the data will save the day,
>> we hope, because the persistence layer can map
>> between the business object representation and the
>> underlying data representation, should they need to 
>> differ.  Yet awkward mappings can be performance
>> hogs.

> DG: I hope this doesn't come across as a really stupid question...
> I think it might, but please  remember that I am a poor, confused,
> data dinosaur.
> Ahem.

Agreed... Oops - make that "Okay"  :-)

> How do you encapsulate the data when you then tie it up into
> objects? Can you split it out under those conditions? Or have
> I totally missed the point of objects? (I am quite prepared to
> accept that I have, but the reference to the object having
> behaviours seems to indicate that an object involves both
> the data and the processes it performs or has performed on it,
> which is my original understanding. Unless the object has no
> data. Help!)

It may be better thinking in terms of Information or Knowledge
rather than Data.  Potentially, an object has both knowledge
and behaviour.  Some objects have no knowledge at all,
they are totally stateless, they merely help other objects do
what they need to do with *their* knowledge.  All objects have
behaviour, but in some cases this behaviour is so trivial
that it is nothing more than accepting and supplying
information.  

From the applications programmer's point of view, this is
all that needs to happen - databases are unnecessary -
except for a coulpe of things that objects don't do very 
well.  The first is persistence.  If the power gets turned off,
we don't want to lose all the information that the business needs.
The second is reporting.  We could do reporting from the 
object layer, but it tends to be much slower than doing it from
a relational database for many types of report.

As long as the object behaves in the way it is designed to behave,
and deals adequately with the type of information it has
responsibility for, it can do whatever it wants inside its
interface, such as storing the information as data in a database,
for example.  It can change what it does behind the interface
with none of its clients being any the wiser (except where the
performance characteristics change).  The responsibilities
that an object appears to support may be delegated to other
objects behind the interface - in this way the data can be
'split out', if I understand your question.



>> (Paul)
>> That's probably not relevant - I can envisage cases
>> where the same data is used by a 30-year-old core
>> business dinosaur and an ultra-modern agile added
>> value miracle (carefully displaying extreme bias ;-)  ).
>
> The 30-year-old core dinosaur being closely coupled while
> the young whippersnapper goes through the APIs? (Biased?
> Me??) If only - can the young upstart access the 30 year old
> file system? Or does it get an extract and build its own version?
> (Replication! Duplication! *Bad* agile thingy... <eg>)

In effect, any 'object' is an in-memory replica of a snippet
of the database (unless it contains no data that persists).
This causes problems, but we have transaction mechanisms
to cope with those problems, so the data doesn't get out of
step where that's important.  Remember, data on its own is
pretty useless; to be useful there must be associated behaviour.
Sometimes this is mediated by the mind of a business person,
sometimes there's an application to do the 'donkey work'.
A modern application written by experienced programmers
would almost certainly have a 'data adaptor' or 'persistence'
layer that would translate between the 'ideal' format and the
'persistence' format, whatever that happened to be.

Personally, I wouldn't know how to go about accessing a
30 year old file system if it were on magnetic tape, unless
I were working in Cobol.  I'm sure it could be done.  I keep
thinking I'd transfer the tape to CD-Rom and chuck away 
the tapes and tape drives - maybe that's naive?


Paul Oldfield

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
www.aptprocess.com

any opinions expressed herein are not necessarily those of
Mentors of Cally or the Appropriate Process Movement
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

For more information about AM, visit the Agile Modeling Home Page at www.agilemodeling.com
--^----------------------------------------------------------------
This email was sent to: [email protected]

EASY UNSUBSCRIBE click here: http://topica.com/u/?bUrKDA.bWnbtk.Z2NtYS1h
Or send an email to: [email protected]

TOPICA - Start your own email discussion group. FREE!
http://www.topica.com/partner/tag02/create/index2.html
--^----------------------------------------------------------------