Loading ...
Sorry, an error occurred while loading the content.
 

Re: [XP] TDD Quandary

Expand Messages
  • Laurent Bossavit
    Erik, ... When such code is buggy, in exactly what ways does it tend to be broken ? How can you tell ? Cheers, -[Laurent]- It is tempting to suppose that
    Message 1 of 10 , Oct 2, 2004
      Erik,

      > How in the world do I test drive graphics (DirectX) code.

      When such code is buggy, in exactly what ways does it tend to be
      broken ? How can you tell ?

      Cheers,

      -[Laurent]-
      It is tempting to suppose that everyone might fully realize
      his powers and that some at least can become complete exemplars
      of humanity. But this is impossible. It is a feature of human
      sociability that we are by ourselves but parts of what we might
      be. We must look to others to attain the excellences that we
      must leave aside, or lack altogether. -- John Rawls
    • Ken Boucher
      ... What strikes me as painful about this is tt seems that the tests involved are mostly testing Direct X, not your code, and as far as that goes, they re only
      Message 2 of 10 , Oct 2, 2004
        > How in the world do I test drive graphics (DirectX) code. That is,
        > code that really does very little data manipulation at all, but
        > basically just works with an API to get things done. And by done, I
        > mean drawing objects on the screen.

        What strikes me as painful about this is tt seems that the tests
        involved are mostly testing Direct X, not your code, and as far as
        that goes, they're only testing how the version of Direct-X you have
        installed on that machine actually works. It makes me wonder if TDD
        is the right tool for that particular problem or if you want a different
        tool for that layer.

        Software tools that may help:
        http://www.parasoft.com/jsp/products/article.jsp?articleId=1500 for C++
        http://www.mvps.org/directx/smalltalk/articles/startmtdx8.htm for smalltalk
      • Ron Jeffries
        ... Good questions, Laurent [and Erik]. This will help us decide which tests are necessary. TDD, as I do it, is for testing the logic of my program. What s
        Message 3 of 10 , Oct 2, 2004
          On Saturday, October 2, 2004, at 6:41:48 AM, Laurent Bossavit wrote:

          >> How in the world do I test drive graphics (DirectX) code.

          > When such code is buggy, in exactly what ways does it tend to be
          > broken ? How can you tell ?

          Good questions, Laurent [and Erik]. This will help us decide which tests
          are necessary.

          TDD, as I do it, is for testing the "logic" of my program. What's the logic
          of the programs you are writing, Erik?

          Ron Jeffries
          www.XProgramming.com
          This is how I develop software.
          Take the parts that make sense to you.
          Ignore the rest.
        • Phlip
          ... Write a temporary test function called reveal() . It will activate the MS Windows event queue, and allow a testee to paint on the screen, and stay up
          Message 4 of 10 , Oct 2, 2004
            bigbaseballer wrote:

            > How in the world do I test drive graphics (DirectX)
            > code. That is,
            > code that really does very little data manipulation
            > at all, but
            > basically just works with an API to get things done.
            > And by done, I
            > mean drawing objects on the screen. Not something I
            > can really "test"
            > in the sense of having inputs and outputs and
            > verifying them, it is
            > just so complex that the closest thing to
            > traditional unit testing I
            > could do is verify if certain pixels were certain
            > colors ... and I
            > assure you that would be a worthless test.

            Write a temporary test function called 'reveal()'. It
            will activate the MS Windows event queue, and allow a
            testee to paint on the screen, and stay up until you
            close it.

            Call it at the bottom of your test case. The case will
            set up a scenario, then each time you hit the test
            button a window appears, and you can visually inspect
            the results. When you like the appearance, comment out
            the 'reveal()'.

            Sadly, this technique boosts early velocity, and
            without real assertions in your code the technique
            will lead to source cannot refactor without high risk.

            > The idea of mock objects comes up. Except, that idea
            > has two
            > implications. First of all, it means that I need to
            > mock the DirectX
            > API ... which I could do. It would be a bit
            > annoying, but I could do
            > it.

            The style is called "Mock Graphics", and one should
            use it as a last resort.

            I recently wrote an OpenGL application using Mock
            Graphics directly at the OpenGL boundary, based on the
            glTrace package by Phil Frisbie, Jr. (When Kent Beck
            finishes reviewing this effort, I might get to publish
            it...)

            The unwisely curious can get the sample code early, in
            the CVS repository at http://sf.net/projects/flea

            > But my BIGGEST issue with this approach is
            > this: I have
            > historically test driven a public interface, and
            > then refactored some
            > internal stuff. But basically, the test shows how
            > the code is supposed
            > to work from an external viewpoint. I get complete
            > code coverage this
            > way, and from talking to some people it sounds like
            > I am on the right
            > track. But what about this? My external interface is
            > not supposed to
            > really tell anything about the DirectX api. How do
            > mocks come into
            > play when I can't actually put them into use without
            > making the
            > interface aware of DirectX?

            Here is a "does this help?" answer: Copy your code
            into a scratch sandbox project, outside your version
            control system, and then do Exploratory Testing and
            Spike Solutions in it. Those are big words for "noodle
            around". Make the code do what you want, and neglect
            the tests. Then write beautiful tests that look like
            they somehow came before the tested code.

            Then copy the tests, line by line, back into the main
            project, and use them to truly test-first the design
            your scratch project arrived at.


            =====
            Phlip
            http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces



            _______________________________
            Do you Yahoo!?
            Declare Yourself - Register online to vote today!
            http://vote.yahoo.com
          • bigbaseballer
            ... the logic ... Ok, to the first question, buggy would be either a) it crashes, or b) there are visual artifacts that we don t like. That second one is sort
            Message 5 of 10 , Oct 2, 2004
              --- In extremeprogramming@yahoogroups.com, Ron Jeffries
              <ronjeffries@X...> wrote:
              > On Saturday, October 2, 2004, at 6:41:48 AM, Laurent Bossavit wrote:
              >
              > >> How in the world do I test drive graphics (DirectX) code.
              >
              > > When such code is buggy, in exactly what ways does it tend to be
              > > broken ? How can you tell ?
              >
              > Good questions, Laurent [and Erik]. This will help us decide which tests
              > are necessary.
              >
              > TDD, as I do it, is for testing the "logic" of my program. What's
              the logic
              > of the programs you are writing, Erik?

              Ok, to the first question, buggy would be either a) it crashes, or b)
              there are visual artifacts that we don't like. That second one is sort
              of broad ... but really that kind of comes down to acceptance testing
              rather than unit testing.

              And there is logic in this part of the code, I need to manage the
              DirectX state, and how I actually draw certain things. But other than
              managing DirectX, most of the actual logic is in performance
              optimizations, and I am looking to stay away from that (for now). But
              when I start, I want to have something to fallback on.

              I think I will do what Philip said. It is actually a really good idea,
              under the circumstances ... I really think I could write decent tests
              if I knew exactly what I was doing, versus just knowing what I want
              the end result to be.
            • Laurent Bossavit
              Erik, ... I would think unit-testing-at-all would be sufficient to catch most of these ? Or is there something else I should know about the kind of DirectX
              Message 6 of 10 , Oct 2, 2004
                Erik,

                > Ok, to the first question, buggy would be either a) it crashes,

                I would think unit-testing-at-all would be sufficient to catch most
                of these ? Or is there something else I should know about the kind of
                DirectX coding errors, specifically, that make it crash ?

                > b) there are visual artifacts that we don't like.

                Can you recall a specific occasion where you had written code that
                produced a visual artifact, and the code should have been written
                another way ?

                Cheers,

                -[Laurent]-
                The greatest obstacle to transforming the world is that we lack the
                clarity and imagination to conceive that it could be different.
                Roberto Mangabeira Unger
              • J. B. Rainsberger
                ... I might use the Gold Master technique here. Draw the screen and check it by hand once. Now capture it and check future draws against the output you
                Message 7 of 10 , Oct 2, 2004
                  bigbaseballer wrote:

                  > How in the world do I test drive graphics (DirectX) code. That is,
                  > code that really does very little data manipulation at all, but
                  > basically just works with an API to get things done. And by done, I
                  > mean drawing objects on the screen. Not something I can really "test"
                  > in the sense of having inputs and outputs and verifying them, it is
                  > just so complex that the closest thing to traditional unit testing I
                  > could do is verify if certain pixels were certain colors ... and I
                  > assure you that would be a worthless test.

                  I might use the Gold Master technique here. Draw the screen and check it
                  by hand once. Now capture it and check future draws against the output
                  you verified by hand. If a future draw is different from the Gold
                  Master, the test fails. Time to investigate. It's not TDD, but it
                  provides a refactoring safety net.

                  Another technique is to think of driving the DirectX API as generating a
                  sequence of events that a DirectXEventListener catches and sends to the
                  API. The techniques for testing event sources and event listeners are
                  well documented. (See _JUnit Recipes_, chapter 14.) Now the goal is to
                  verify that your code generates the proper set (or sequence) of events,
                  the result of which would be a correctly-drawn screen. The set of events
                  could be quite big. If it is, then I highly recommend testing this with
                  FIT/FitNesse, rather than xUnit.

                  I hope this helps.
                  --
                  J. B. (Joe) Rainsberger
                  Diaspar Software Services
                  http://www.diasparsoftware.com :: +1 416 791-8603
                  Predictable, repeatable, quality delivery
                Your message has been successfully submitted and would be delivered to recipients shortly.