Loading ...
Sorry, an error occurred while loading the content.

Refactoring in a different enviroment

Expand Messages
  • Lorenzo Jorquera
    Hello, I want to share my experience using refactoring in enviroment that is very different to the OOP enviroment where this technique is mostly used. I
    Message 1 of 3 , Nov 30, 2004
    • 0 Attachment
      Hello,

      I want to share my experience using refactoring in enviroment that is very different to the OOP enviroment where this technique is mostly used. I apologize for my english, it is not my native language...

      I am currently working with an ETL tool called Datastage. We use this tool to extract data from different sources and then transform this data to build datawarehouses. In Datastage you program by drawing a graph. Each node is a transformations you want to apply to the data while the links between nodes indicate how the data flows.
      I inherited three huge jobs (this is how programs are called in datastage). They were really big and messy. What is more, much of the data transformations they were doing was not needed at all.

      I did several atempts of improving the design of this jobs, always trying to understand completly what they were doing before modifing them, but I kept failing, as they were too big and it was impossible for me to keep all the details in my head at once ... until I realised that what I was trying to do was to refactor the jobs, and I begun to do the same I would do to improve a Java or Ruby. I created a way of testing that I was not changing the functionality of the jobs (I kept copies of their original output), and then I started to attack the little problems I could see all over the jobs (the smells), one at a time (and running the tests after each change).

      It was amazing to see how fast the design of the jobs improved by tackling one little problem at a time. It also was very educative to apply the principles of Refactoring in a such a differente enviroment.



      ---------------------------------
      Do You Yahoo!?
      Todo lo que quieres saber de Estados Unidos, América Latina y el resto del Mundo.
      Visíta Yahoo! Noticias.


      [Non-text portions of this message have been removed]
    • Tom Copeland
      ... Nice story! The same thing applies to complex SQL queries; I ll pipe the output to a file and then I can refactor the SQL to replace, say, a SELECT
      Message 2 of 3 , Nov 30, 2004
      • 0 Attachment
        On Tue, 2004-11-30 at 14:36, Lorenzo Jorquera wrote:
        > I created a way of testing that I was not changing the functionality of the jobs (I kept copies of their original output),

        Nice story!

        The same thing applies to complex SQL queries; I'll pipe the output to a
        file and then I can refactor the SQL to replace, say, a SELECT DISTINCT
        with a subquery. It's a very satisfying way to work...

        Yours,

        Tom
      • robrambusch
        ... functionality of the jobs (I kept copies of their original output), ... to a ... DISTINCT ... It also works quite well when replacing a complex unix script
        Message 3 of 3 , Nov 30, 2004
        • 0 Attachment
          --- In extremeprogramming@yahoogroups.com, Tom Copeland <tom@i...>
          wrote:
          > On Tue, 2004-11-30 at 14:36, Lorenzo Jorquera wrote:
          > > I created a way of testing that I was not changing the
          functionality of the jobs (I kept copies of their original output),
          >
          > Nice story!
          >
          > The same thing applies to complex SQL queries; I'll pipe the output
          to a
          > file and then I can refactor the SQL to replace, say, a SELECT
          DISTINCT
          > with a subquery. It's a very satisfying way to work...
          >
          > Yours,
          >
          > Tom

          It also works quite well when replacing a complex unix script on an
          automated scheduler.

          Rob Rambusch
        Your message has been successfully submitted and would be delivered to recipients shortly.