Loading ...
Sorry, an error occurred while loading the content.
 

Re: [agileDatabases] The Cultural Impedance Mismatch (between developers and data professionals)

Expand Messages
  • Scott Ambler
    ... The issue isn t how hard you re trying, it s how well you re succeeding. If you re not finding ways to motivate people to learn this stuff, then we ve
    Message 1 of 12 , Aug 7, 2009
      --- On Thu, 8/6/09, David Portas <dportas@...> wrote:

      > >
      >
      > I do agree with Clifford's fine summary. Scott's view that
      > data
      > management professionals aren't trying hard enough to
      > educate
      > application developers is not my experience. Instead I've
      > sometimes
      > found it hard to motivate developers to learn about
      > enterprise data
      > management issues, and that includes agile teams as well.

      The issue isn't how hard you're trying, it's how well you're succeeding. If you're not finding ways to motivate people to learn this stuff, then we've still got a problem.

      > Perhaps
      > especially agile teams in fact. Increasingly, those
      > developers rely
      > too much on mapping their object models to relational
      > models and
      > assume they don't need to bother with relational database
      > modelling at
      > all.

      Exactly. And unless you find ways to motivate people to go beyond that, the situation will never get better. The OR framework folks found a way to motivate people to get interesting in OR mapping -- They solved a perceived need and provided (relatively) consumable tooling to help address that.

      The development community must perceive the need to improve data quality, or at least not make the problem worse, and I suspect for the most part that they do. They might not understand the solution, and they likely don't have decent tools (yet).

      >
      > On explaining to one developer why join dependency was an
      > important
      > consideration in database design I was told that it was
      > "too
      > theoretical" and not relevant to real development. Of
      > course it may be
      > that I'm not a very good teacher. Perhaps I'm not. But most
      > of the
      > learning material that developers are exposed to totally
      > fails to deal
      > with fundamental data management issues, which means there
      > is a
      > mountain to climb when it comes to teaching them how to
      > solve
      > practical problems.

      As to my previous point, I'm sure you tried hard but it didn't work out. I see this all the time too. Some data professionals are trying to transfer skills, I'm sure some are succeeding, some are trying and not succeeding as often as they'd like, but many aren't even trying.

      Perhaps if we discussed issues like this in terms that developers are (hopefully) interested in, such as quality and performance, we'd likely have better results.

      I'd be surprised to discover that more than 1 in 3 orgs choose to give developers even something as simple as a 1 or 2 day course in database design, let alone significant training and mentoring.

      >
      > Scott, if you really believe in educating developers about
      > data
      > management issues then I can't understand why you seem to
      > discourage
      > them from learning database fundamentals
      > (http://www.agiledata.org/essays/relationalTheory.html).

      Because there's very little of practical value in it for developers. Most developers are tired of wading through all the theory to get to the few nuggets of practical information that they can use to do a good job. Until the data community comes up with a viable way of communicating the nuggets, and doing so in a non-theoretical-sounding manner, the development community will very likely continue to ignore them. The story that you told above is an example, or perhaps a symptom, of this very thing.

      > The urgent
      > need for educators in my opinion is not to teach "applying
      > SQL in
      > practice"(!!) instead more education is needed to give IT
      > professionals a grounding in the basics so that they will
      > then stand a
      > chance of understanding more advanced issues and
      > techniques.

      And how many decades has the data community been thrashing on this? Do you honestly see a light at the end of this tunnel?

      - Scott
      Scott W. Ambler
      Chief Methodologist/Agile, IBM Rational
      Agile at Scale blog: http://www.ibm.com/developerworks/blogs/page/ambler
      Follow me on Twitter: http://twitter.com/scottwambler



      __________________________________________________________________
      Connect with friends from any web browser - no download required. Try the new Yahoo! Canada Messenger for the Web BETA at http://ca.messenger.yahoo.com/webmessengerpromo.php
    • Devdas Bhagat
      ... Perhaps the problem is the people who think of Objects as the be-all-and-end-all of all programming solutions, even when they are clearly not appropriate?
      Message 2 of 12 , Aug 7, 2009
        On 8/7/09, Scott Ambler <scottwambler@...> wrote:
        >
        >
        >
        > --- On Thu, 8/6/09, David Portas <dportas@...> wrote:
        >
        >> >
        >>
        >> I do agree with Clifford's fine summary. Scott's view that
        >> data
        >> management professionals aren't trying hard enough to
        >> educate
        >> application developers is not my experience. Instead I've
        >> sometimes
        >> found it hard to motivate developers to learn about
        >> enterprise data
        >> management issues, and that includes agile teams as well.
        >
        > The issue isn't how hard you're trying, it's how well you're succeeding. If
        > you're not finding ways to motivate people to learn this stuff, then we've
        > still got a problem.
        >
        Perhaps the problem is the people who think of Objects as the
        be-all-and-end-all of
        all programming solutions, even when they are clearly not appropriate? A lot of
        programmers do not understand object theory, and would claim that their
        procedural code is object-oriented because it uses classes and is written in a
        language which claims to be objects all the way down.

        As a self-taught professional, I have problems when the developers I
        work with refuse to
        learn about functional programming. "If it can't be done in my
        language of choice,
        using my paradigm of choice, then I will make my solution your
        problem" has been a
        fairly common response to attempts to help people learn about data modelling.

        One approach I have been trying to get traction for is to make the
        application developers responsible for actually supporting the
        application. The developers are on call, they are
        tech support, they are responsible for operations and data management
        issues as well.

        I don't care how you do it, as long as you get the right answers fast
        enough, without
        spending too much.

        >> Perhaps
        >> especially agile teams in fact. Increasingly, those
        >> developers rely
        >> too much on mapping their object models to relational
        >> models and
        >> assume they don't need to bother with relational database
        >> modelling at
        >> all.
        >
        > Exactly. And unless you find ways to motivate people to go beyond that, the
        > situation will never get better. The OR framework folks found a way to
        > motivate people to get interesting in OR mapping -- They solved a perceived
        > need and provided (relatively) consumable tooling to help address that.
        >
        The problem is that the tooling works only in certain simple cases.
        When you hit any
        sort of complexity, the tools give you a bad solution. You don't
        realise the solution is bad until you hit a performance/data quality
        wall.

        Most OO type developers would realise a data encapsulation problem in
        code like this:
        class foo {
        public int bar;
        }

        However, this is not seen to be a problem:
        TABLE foo (bar integer);

        If you need to encapsulate data in code, then why not encapsulate it
        in the database as
        well? Stored procedures/functions work quite well for this.

        > The development community must perceive the need to improve data quality, or
        > at least not make the problem worse, and I suspect for the most part that
        > they do. They might not understand the solution, and they likely don't have
        > decent tools (yet).
        >
        At some point, the tool is the human brain. Developers need to
        understand relational
        theory to be able to use any tools which will help in maintaining data
        quality. The same
        tool applies to security.

        Developers who only know imperative languages are a significant
        disadvantage to any
        development team.

        I am sure there are developers who can write perfectly good SQL, but
        they still don't
        understand what they are actually doing.
        <snip>
        >
        > Because there's very little of practical value in it for developers. Most
        > developers are tired of wading through all the theory to get to the few
        > nuggets of practical information that they can use to do a good job. Until

        How do you know you are doing a good job unless you actually understand the
        mathematics behind the system you are using?

        IMHO, a lot of developers have a very short term way of thinking,
        which is encouraged by
        agile programming techniques. Data professionals have a longer term
        view of thinking.
        (This doesn't mean I disagree with agile techniques, or the
        principles, it just means that I
        have a longer term goal in mind as well when I am applying these techniques).

        How many developers think in terms of code needing to run for decades? How many
        programs even run for that long? How much data is used after decades?

        > the data community comes up with a viable way of communicating the nuggets,
        > and doing so in a non-theoretical-sounding manner, the development community
        > will very likely continue to ignore them. The story that you told above is
        > an example, or perhaps a symptom, of this very thing.
        >
        >> The urgent
        >> need for educators in my opinion is not to teach "applying
        >> SQL in
        >> practice"(!!) instead more education is needed to give IT
        >> professionals a grounding in the basics so that they will
        >> then stand a
        >> chance of understanding more advanced issues and
        >> techniques.
        >
        > And how many decades has the data community been thrashing on this? Do you
        > honestly see a light at the end of this tunnel?
        >
        Do you have any alternative suggestion(s)? I would be glad to learn of
        alternative
        techniques.

        Devdas Bhagat
      • Scott Ambler
        ... ... That s a very good strategy. I ve been writing about the need to include an explicit production phase in SDLCs for years to put it in
        Message 3 of 12 , Aug 8, 2009
          --- On Fri, 8/7/09, Devdas Bhagat <devdas.b@...> wrote:

          <snip>
          >
          > One approach I have been trying to get traction for is to
          > make the
          > application developers responsible for actually supporting
          > the
          > application. The developers are on call, they are
          > tech support, they are responsible for operations and data
          > management
          > issues as well.
          >

          That's a very good strategy. I've been writing about the need to include an explicit production phase in SDLCs for years to put it in everyone's face that someone is going to have to operate and support the system once it's delivered. I don't trust developers that haven't had to maintain and support an existing system running in production, and prefer to work with people who have had to do so both for systems that they've been involved with building and with systems that they haven't. These sorts of developers have a significantly better understanding of what it truly takes to deliver a system.


          >
          > If you need to encapsulate data in code, then why not
          > encapsulate it
          > in the database as
          > well? Stored procedures/functions work quite well for
          > this.

          There's a few trade-offs when doing this, and you've still got to design the schema that's being encapsulated.

          <snip>
          > How do you know you are doing a good job unless you
          > actually understand the
          > mathematics behind the system you are using?

          That's a huge assumption. See http://www.agiledata.org/essays/relationalTheory.html for some thoughts.


          >
          > IMHO, a lot of developers have a very short term way of
          > thinking,
          > which is encouraged by
          > agile programming techniques. Data professionals have a
          > longer term
          > view of thinking.

          But if they can't interact effectively with development teams, it really doesn't matter much, does it?

          - Scott
          Scott W. Ambler
          Chief Methodologist/Agile, IBM Rational
          Agile at Scale blog: http://www.ibm.com/developerworks/blogs/page/ambler
          Follow me on Twitter: http://twitter.com/scottwambler





          __________________________________________________________________
          Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail. Click on Options in Mail and switch to New Mail today or register for free at http://mail.yahoo.ca
        • Devdas Bhagat
          On 8/8/09, Scott Ambler wrote: ... I am not denying the need to design the schema. It s just easier to change the schema as
          Message 4 of 12 , Aug 8, 2009
            On 8/8/09, Scott Ambler <scottwambler@...> wrote:
            <snip>
            >> If you need to encapsulate data in code, then why not
            >> encapsulate it in the database as well? Stored procedures/functions
            >> work quite well for this.
            >
            > There's a few trade-offs when doing this, and you've still got to design the
            > schema that's being encapsulated.
            >
            I am not denying the need to design the schema. It's just easier to change the
            schema as needed by the data itself if the data storage is encapsulated.

            > <snip>
            >> How do you know you are doing a good job unless you
            >> actually understand the
            >> mathematics behind the system you are using?
            >
            > That's a huge assumption. See
            > http://www.agiledata.org/essays/relationalTheory.html for some thoughts.
            >
            >
            >>
            >> IMHO, a lot of developers have a very short term way of thinking,
            >> which is encouraged by agile programming techniques. Data
            >> professionals have a longer term view of thinking.
            >
            > But if they can't interact effectively with development teams, it really
            > doesn't matter much, does it?
            >
            Why do I never hear of an example of a programmer actually going and asking
            a database professional, "Do I have a source for this data already? What is the
            best way to store the data so that it will be useful after my program has been
            replaced? How can I make your life easier?"

            I am not sure about how much blame should be put on the data professionals
            and how much on the programmers (IMHO, both sides are to blame for the lack
            of communication. One side trying to communicate is simply not going to work).

            Devdas Bhagat
          • Scott Ambler
            ... That s a huge assumption. For relational databases that s clearly not true if you understand how to do database refactoring. Theory tells us that
            Message 5 of 12 , Aug 9, 2009
              --- On Sat, 8/8/09, Devdas Bhagat <devdas.b@...> wrote:

              > From: Devdas Bhagat <devdas.b@...>
              > <snip>
              > >> If you need to encapsulate data in code, then why
              > not
              > >> encapsulate it in the database as well? Stored
              > procedures/functions
              > >> work quite well for this.
              > >
              > > There's a few trade-offs when doing this, and you've
              > still got to design the
              > > schema that's being encapsulated.
              > >
              > I am not denying the need to design the schema. It's just
              > easier to change the
              > schema as needed by the data itself if the data storage is
              > encapsulated.

              That's a huge assumption. For relational databases that's clearly not true if you understand how to do database refactoring. Theory tells us that refactoring databases is hard, practice shows us that it's not.



              >
              > I am not sure about how much blame should be put on the
              > data professionals
              > and how much on the programmers (IMHO, both sides are to
              > blame for the lack
              > of communication. One side trying to communicate is simply
              > not going to work).

              Agreed. The article is clear that both sides are to blame. Surveys appear to show that data professionals seem to have a greater share of it though.

              - Scott
              Scott W. Ambler
              Chief Methodologist/Agile, IBM Rational
              Agile at Scale blog: http://www.ibm.com/developerworks/blogs/page/ambler
              Follow me on Twitter: http://twitter.com/scottwambler



              __________________________________________________________________
              Make your browsing faster, safer, and easier with the new Internet Explorer® 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
            Your message has been successfully submitted and would be delivered to recipients shortly.