Loading ...
Sorry, an error occurred while loading the content.

Re: [agileDatabases] Moving a Data Warehouse Team to Agile

Expand Messages
  • Clifford Heath
    Marty, I have a friend who s doing the same thing - coaching a DW team to agile. AIUI, his efforts are being spent, in the main, figuring out how to TDD test
    Message 1 of 5 , Jan 5, 2009
    • 0 Attachment
      Marty,

      I have a friend who's doing the same thing - coaching a DW team
      to agile. AIUI, his efforts are being spent, in the main, figuring out
      how to TDD test (and get DW developers to test) the ETL process.
      I believe they're using Ruby to automate the testing process.

      I've BCC'd him on this message, so hopefully he'll respond to you
      privately.

      Clifford Heath, Data Constellation, http://dataconstellation.com
      Agile Information Management and Design

      On 06/01/2009, at 6:58 AM, Bland, Marty wrote:

      > Agile Databases,
      >
      > I am an agile coach beginning to move a data warehouse to agile.
      > Can you provide example user stories for a data warehouse team? The
      > team is working in an waterfall methodology. What is best way to move
      > them to be more agile? What are other best practices in migrating a
      > data warehouse team to agile principles. Thanks in advance for the
      > help.
      >
      > Marty Bland
      > Agile Project Leader
      > Cell 804-310-3049
      > 804-284-1848 (TIE 433)
      >
      >
      >
      >
      >
      > The information contained in this e-mail is confidential and/or
      > proprietary
      > to Capital One and/or its affiliates. The information transmitted
      > herewith
      > is intended only for use by the individual or entity to which it is
      > addressed. If the reader of this message is not the intended
      > recipient,
      > you are hereby notified that any review, retransmission,
      > dissemination,
      > distribution, copying or other use of, or taking of any action in
      > reliance
      > upon this information is strictly prohibited. If you have received
      > this
      > communication in error, please contact the sender and delete the
      > material
      > from your computer.
      >
      >
      >
      > [Non-text portions of this message have been removed]
      >
      >
      > ------------------------------------
      >
      > Yahoo! Groups Links
      >
      >
      >
    • Gabriel Tanase
      Hello, I would like to second Clifford s recommendation, for several reasons: 1. ETL is usually the biggest part of the development effort in an DW project. It
      Message 2 of 5 , Jan 7, 2009
      • 0 Attachment
        Hello,

        I would like to second Clifford's recommendation, for several reasons:
        1. ETL is usually the biggest part of the development effort in an DW
        project. It makes sense to break it down into sprints and give incremental,
        immediately usable value to business.
        2. Reconciliation of figures between source systems and the DW is the basis
        of the trust the DW team needs to build into the minds of DW's business
        users. The sooner the team can show consistent, successful reconciliation,
        the higher the chances of success for the entire project
        3. Although ETL does not deliver reports and OLAP cubes as specified (these
        being usually the desired end result), developing ETL procedures quickly and
        reliably may allow selected business users ("power users") to browse and
        query data areas much sooner than in a waterfall project. The benefits can
        be large:
        - users gain confidence early
        - specifications and development of reports are based on real data, rather
        than mock data or no data
        - it is clear earlier what is possible and what is not possible with the
        real data, which may positively realign expectations and requirements for
        reports
        - many unexpected requirements can be unearthed and clarified
        ... and so on

        This is not to say that other development items should be treated with less
        attention, but getting data in correctly is the foundation of everything
        else.

        One caveat, though: the tests that DW ETL developers must design and develop
        should be more forward-thinking that normally. Tests should be developed (at
        some point) to handle use cases that may not appear critical in the short
        term but are bound to happen in the long term and may impact the data
        integrity in the DW if not handled correctly.
        I am thinking e.g. of handling corrections / reversals or of updates /
        renumbering of keys that may happen in the source systems.


        Kind regards,
        Gabriel


        2009/1/5 Clifford Heath <clifford.heath@...>

        > Marty,
        >
        > I have a friend who's doing the same thing - coaching a DW team
        > to agile. AIUI, his efforts are being spent, in the main, figuring out
        > how to TDD test (and get DW developers to test) the ETL process.
        > I believe they're using Ruby to automate the testing process.
        >
        > I've BCC'd him on this message, so hopefully he'll respond to you
        > privately.
        >
        > Clifford Heath, Data Constellation, http://dataconstellation.com
        > Agile Information Management and Design
        >
        >
        > On 06/01/2009, at 6:58 AM, Bland, Marty wrote:
        >
        > > Agile Databases,
        > >
        > > I am an agile coach beginning to move a data warehouse to agile.
        > > Can you provide example user stories for a data warehouse team? The
        > > team is working in an waterfall methodology. What is best way to move
        > > them to be more agile? What are other best practices in migrating a
        > > data warehouse team to agile principles. Thanks in advance for the
        > > help.
        > >
        > > Marty Bland
        > > Agile Project Leader
        > > Cell 804-310-3049
        > > 804-284-1848 (TIE 433)
        > >
        > >
        > >
        > >
        > >
        >
        >
        >
        >


        [Non-text portions of this message have been removed]
      Your message has been successfully submitted and would be delivered to recipients shortly.