Loading ...
Sorry, an error occurred while loading the content.

Re: [agileDatabases] Managing Dependent Systems

Expand Messages
  • RĂ©mi Levasseur
    ... He didn t said the dependent systems were analytic-treatment based. Perhaps are these OLTP subsystems only used for load sharing or historical reasons. The
    Message 1 of 6 , Feb 15, 2006
    • 0 Attachment
      Steve Jorgensen wrote:
      essawit wrote:
      
        
      All,
      
      I am working on a large OLTP data.  This database is copied every 12 
      hours about 6 times.  Once this is done secondary systems read the data 
      and and use it for other processes.  
      
      The issue is that these systems are very tightly coupled to the tables 
      and columns with the OLTP database.
       
      
          
      By "these systems" I understand you to mean "secondary systems".
      
        
      Whenever we attempt to change anything in the database we have what 
      amounts to a summit meeting to discuss the change and when the 
      estimates from downstream systems are collected a simple change end up 
      taking months to complete.  I know many systems must face this 
      challenge with ETL processes that are dependent on the primary OLTP 
      schema.  Does anyone know of any good books/articles on how 
      dependencies can be managed effectivley?
       
      
          
      One simple and not exacly new concept is that the databases that receive 
      data should not have schemas particularly like the OLTP database at all, 
      because they are OLAP applications and should query from OLAP -friendly 
      schemas.  In this case, you never copy the OLTP database at all (except 
      perhaps as a process stage).  Instead, you use the OLTP data to populate 
      the OLAP databases.
      
      With this structure in place, most upstream schema changes are isolated 
      from the downstream.  Changes in either schema may require changes to 
      the data transformation queries or scripts, but probably won't require 
      changes to the other schema, and therefore won't require changes to code 
      that employs the other schema.
        
      He didn't said the dependent systems were analytic-treatment based. Perhaps are these OLTP subsystems only used for load sharing or historical reasons. The first question is indedded interesting.
      If as suggested the sub systems are here for analytic purposes, ETL design is the key.

      Regards,
      Rémi Levasseur



    • Steve Jorgensen
      ... Ah - but if it s for load sharing, then the approach should be to reduce code duplication between the applications that make use of the data. This way, the
      Message 2 of 6 , Feb 15, 2006
      • 0 Attachment
        Rémi Levasseur wrote:
        ...
        One simple and not exacly new concept is that the databases that receive 
        data should not have schemas particularly like the OLTP database at all, 
        because they are OLAP applications and should query from OLAP -friendly 
        schemas.  In this case, you never copy the OLTP database at all (except 
        perhaps as a process stage).  Instead, you use the OLTP data to populate 
        the OLAP databases.
        
        With this structure in place, most upstream schema changes are isolated 
        from the downstream.  Changes in either schema may require changes to 
        the data transformation queries or scripts, but probably won't require 
        changes to the other schema, and therefore won't require changes to code 
        that employs the other schema.
          
        He didn't said the dependent systems were analytic-treatment based. Perhaps are these OLTP subsystems only used for load sharing or historical reasons. The first question is indedded interesting.
        If as suggested the sub systems are here for analytic purposes, ETL design is the key.
        Ah - but if it's for load sharing, then the approach should be to reduce code duplication between the applications that make use of the data.  This way, the updated code components could be rolled out simultaneously with the schema changes.

        - Steve J.
      Your message has been successfully submitted and would be delivered to recipients shortly.