Data Stuff was: RE: [am] old diagrams for old people
- Just wanted to make people aware of
www.agiledata.org/essays/agileDataModeling.html and www.agiledata.org in
A few comments below.
Answering a couple at once.
Also, cross posted to the Agile Database list due to the relevance of the
discussion. My apologies if you get multiple copies.
At 12:41 PM 7/31/2004, Steve Gordon wrote:
>Pragmatically, the data model has to integrate with the enterprise-wideActually, it requires the "guardians" to choose to work together with us
>data model, which requires waiting for its guardians to cooperate. This
>makes prototyping with a data model slow, and also makes changes quite
effectively. Most choose to work otherwise. There is fundamentally
nothing special about data modeling. Unfortunately the data community,
IMHO, is currently burdened with some very traditional thinkers. To
paraphrase Kuhn, I suspect that we'll need to wait for them to either
retire or pass away until we can see a major change of thinking within that
community. Until then all we can do is invite those folks to join us. It
is possible for them to be agile, I think I've shown fairly well how they
can do so at www.agiledata.org, but they need to choose to change their
ways. It really is a choice.
>This is getting into design issues. You may choose to work this way or you
>The initial functionality should, of course, not read and write directly
>to flat files, but instead utilize data objects that are implemented in
>terms of flat files (using XML, the work can be quite minimal). Then,
>once the functionality is refined sufficiently by informed feedback, you
>only have to reimplement the data objects appropriately for the corporate
>data model. We keep the quick and dirty data object implementation around
>to support unit testing.
may choose to work other ways. I can't help but think that a Java team who
is familiar with Hibernate (or another persistence framework) could choose
to work with that right away.
>If you need to fit into the corporate data model, which is virtually always
>One of the biggest benefits to this approach is that we can keep making
>progress while the guardians of the corporate data model resist our
>changes or otherwise impede our progress on the data front.
the case, then why can you simply consider it a constraint as you develop
in an evolutionary? Why can't you follow enterprise naming
conventions? Why can't they work with you as an active member of the
team? I provide some insight at
> -----Original Message-----You're choosing to work that way. Many people choose to not work that way
> From: sialbs@... [mailto:sialbs@...]
> Sent: Sat 7/31/2004 9:20 AM
> To: firstname.lastname@example.org
> Subject: RE: [am] old diagrams for old people
> That has always been a question in the back of my mind: How you
> were able to support data modeling in your posts and also be so ardently
> agile. As an inveterate data modeler for 25 years or so I've always gone
> for the data model first and then based all processes on that. Our
> evolutionary prototyping approach requires the creation of a data model
> first. This is partly because we work that way anyway, but also because
> the tools we use require a data model to be created in order to be able
> to define the fields we place on the screens.
and they're very successful.
>It depends on the situation. I've been on projects where the data folks
> I guess the idea of building temporary files (kind of a living
> data model because you are actually creating a model using the flat
> files) to demonstrate the solution always seemed like double work. Once
> they like the solution then you have to undo all the flat files and
> replace them with relational or whatever after defining the data model,
> not to mention rewriting code with SQL or whatever. But the way you state
> it here makes elegant sense.
expected us to wait several months for them to set up the database. We
could have done the flat file thing almost instantly. However, luckily I
have a few contacts at Oracle and obtained some licenses so we could set up
our own DBs (the client had a corporate licence, but the data folks felt
the need to control access to them).
> Now, my question is how does this fit in with Scott's approach toYes, assuming you have DB access and cooperative data folks why can't you
> agile data modeling? Isn't he espousing do a little data model for the
> first iteration and then augment and add more detail to the data model
> with each successive iteration? I'm having trouble quite grasping that
> whole thing anyway. How does your approach fit in with Scott's?
work that way?
>Why would you want to do that? By adding a data model profile to the UML
> From: Dagna [mailto:dagna@...]
> Sent: Friday, July 30, 2004 7:46 PM
> And please can I nominate 'UML data model' as the Oxymoron of the Week?
> Dagna Gaythorpe
it could achieve the following benefits:
- It would put a recognized standards body behind a data modeling notation,
something which has yet to occur within the data community.
- It would signal to the modeling tool vendors that it's about time they
added REAL support for data modeling to their development tools.
- It would signal to developers that they should learn something about data
modeling (the data community clearly hasn't been able to make basic data
skills attractive to developers).
- It would provide a way to depict data models in a notation which
developers would likely recognize. Shouldn't data folks be interested in
communicating their work?
At 04:47 PM 8/1/2004, Dagna wrote:
>Steven,Which is probably why they should find ways to work effectively with the
>Pragmatically, the Enterprise Data Model guardians are concerned with how
>everything fits together... Not just how one system works, but the whole
>enterprise. And a quick, agile fix now can be a major pain in the future in
>terms of transformation and translation needed to integrate with other
agile folks. See my many writings on the subject at
www.agiledata.org. Otherwise the agile folks will view the data guardians
as an impediment to success and will very likely find ways to block
them. I've lost track of the number of times I've heard a data person
lament about development teams that went rogue, yet I can't help but think
that many times the reason why they went rogue was because the data folks
weren't able to meet the needs of the team (even if that means explaining
why they can't do what they want to do and then finding another way to do
>Your third paragraph is the reason I joined this list - there is a huge gulfExactly. We need to work together effectively, and the way do that is to
>in understanding between enterprise data people and agile developers. We all
>have very good reasons for acting the way we do, and if we don't manage to
>understand each other better then we will carry right on wasting huge
>amounts of money transforming data as we move it around, and being unable to
>report across systems (which the enterprise people want to avoid), and
>putting in systems late and over budget and not working properly (which the
>agile people want to avoid).
start by trying to understand what the other folks are trying to
achieve. In fact, working together is philosophy #5 of 6 of the Agile Data
method, see http://www.agiledata.org/essays/vision.html#Philosophies .
>If you have an enterprise data model, then you should be getting 'starterBetter yet, why not have one or more people on the team who are up to speed
>packs' from it for every development, showing the basic, standard data that
>already exists and what format it is in. If you aren't, then why do you have
on the enterprise data model (and have access to it, gotta think it changes
over time), as well as the enterprise naming conventions, ... ?
>an enterprise data model? It should be a tool for speeding things up, not anWish the rest of the people attending the DAMA conferences would come and
>obstacle course to enforce conformity without any payback. They should also
>be able to tell you which systems own the data you need, and if there are
>any existing interfaces that you can re-use for the inputs you need. If they
>aren't doing these things and want to know how, then they should come and
>talk to me at a DAMA conference some time (or email me).
talk with you. Having attended several I've come away completely depressed
about the state of the data community. It's like a time warp back to the
mid-1980s (for the most part).
I'll definitely try to make one of your talks next time I'm at a conference
><snip>At 05:17 PM 8/1/2004, Steve Gordon wrote:
>Dagna,To be fair this is a reflection of your environment, it doesn't necessarily
>I am not in that kind of environment presently. The enterprise data model
>and agile software
>development move at quite different speeds. There may well be some good
>reasons for the
>difference in speed, but agile developers cannot let the slower speed or
>the resistance to
>change at the enterprise level hold up their project.
have to be the case.
Also to be fair, this seems to be the norm. My experiences with the data
community is that they are burdened with many traditional, non-agile
folks. Worse yet, many of them don't seem very willing to change. Some
do, many don't. The IT world is changing and I suspect that Corporate
Darwinism will take care of those folks who don't want to change.
My advice is to work with them, communicate to them how you work, and point
out that you need them to work this way and that it is clearly possible for
them to do so if they choose to step up to the challenge. If they choose
not to then find ways to fend for yourself.
>Sounds like you're doing this extra work to go around them. Not ideal, but
>If we present EDM changes we need to do iteration N, and we cannot be
>iteration N+2, how are we do to iterative development? If the feedback to
>software indicates the need to do something different, we may need to ask
>EDM to undo the
>previous change and make a different accomodation. This will definitely
>not sit well with the
>EDM people. This is why I advocate not asking for the EDM accomodations
>business functionality and its implementation have been validated by
>customers and users.
>This requires implementing them in the simplest, well-designed way
>possible in the meantime,
>which usually means using data objects that read and write flat files.
then again sounds like they're not stepping up to the challenge. Sigh.
Can you get them to experiment a bit? To assign their most agile, or at
least open minded, person to work with your team? To learn the new
ways? To break some of their current rules so that they can learn (the
hard way) new ways to work with agile teams? Bottom line is that agile
software development is real, it's growing, and isn't going away. They
need to accept this fact and start adjusting to it.
Scott W. Ambler
Senior Consultant, Ronin International, Inc.
www.agiledata.org -:- www.agilemodeling.com -:- www.ambysoft.com -:-
www.databaserefactoring.com -:- www.enterpriseunifiedprocess.com -:-