Loading ...
Sorry, an error occurred while loading the content.
 

Re: [XP] Real or Unreal Test. Help me argue with my colleagues

Expand Messages
  • nader1talai@gmail.com
    Here is my understanding of the problem and suggestion Input is entered via B, A reads values and makes the values available to C C transmits values over the
    Message 1 of 7 , Apr 26, 2011
      Here is my understanding of the problem and suggestion

      Input is entered via B,
      A reads values and makes the values available to C
      C transmits values over the air to PC.

      I suggest that you need tests at system the whole end to end, module A, B, C and unit test level.

      You may build up the end to end tests through mocks and you will also need end to end test with the real system.

      Nader
      Sent from my BlackBerry� wireless device

      -----Original Message-----
      From: Steven Woody <narkewoody@...>
      Sender: extremeprogramming@yahoogroups.com
      Date: Tue, 26 Apr 2011 22:00:11
      To: <extremeprogramming@yahoogroups.com>
      Reply-To: extremeprogramming@yahoogroups.com
      Subject: [XP] Real or Unreal Test. Help me argue with my colleagues

      Hi,

      Our product is an embedded device. Looking externally, it is a
      powered box, say A. On one side of the box, their is a USB interface
      that need to be connected with another device B. Inside the box, there
      exists a wireless modem chip C. Our system, basically, is to read
      values from A, and make it readable via the wireless modem from a
      client program running on PC.

      User cannot directly play with the box A, he has to operate with
      device B and play with a PC read program on a PC.

      Now, we concern how the acceptance test for our box A should be done.
      My options is to mock device A and modem C, since I think the system
      under test is just A. But many colleagues think this kind of test is
      unreal and want to put the real device B and real modem C together in
      a test setup.

      My reasoning has not yet satisfying them so much. Would you please
      tell me your opinion and help me in argue with them?

      Thanks in advance.


      --
      Life is the only flaw in an otherwise perfect nonexistence
      � � -- Schopenhauer

      narke
      public key at http://subkeys.pgp.net:11371 (narkewoody@...)



      [Non-text portions of this message have been removed]
    • Steven Woody
      ... Thanks for the opinion. Maybe not to call it acceptance test, but system test? I am not so familiar to these definitions. The problem here is How to
      Message 2 of 7 , Apr 26, 2011
        On 26 April 2011 22:09, Curtis Cooley <curtis.cooley@...> wrote:

        >
        >
        > On Apr 26, 2011 7:00 AM, "Steven Woody" <narkewoody@...> wrote:
        > >
        > >
        > >
        > > Hi,
        > >
        > > Our product is an embedded device. Looking externally, it is a
        > > powered box, say A. On one side of the box, their is a USB interface
        > > that need to be connected with another device B. Inside the box, there
        > > exists a wireless modem chip C. Our system, basically, is to read
        > > values from A, and make it readable via the wireless modem from a
        > > client program running on PC.
        > >
        > > User cannot directly play with the box A, he has to operate with
        > > device B and play with a PC read program on a PC.
        > >
        > > Now, we concern how the acceptance test for our box A should be done.
        > > My options is to mock device A and modem C, since I think the system
        > > under test is just A. But many colleagues think this kind of test is
        > > unreal and want to put the real device B and real modem C together in
        > > a test setup.
        > >
        > > My reasoning has not yet satisfying them so much. Would you please
        > > tell me your opinion and help me in argue with them?
        > >
        >
        > In my opinion, acceptance testing should include as much of the 'real
        > world'
        > as practical. These are tests that don't necessarily have to run fast. They
        > are your regression suite that tells you when something is broken.
        >
        > A test/qa lab with real world equipment will provide the highest level of
        > confidence in the system.
        >
        >
        Thanks for the opinion. Maybe not to call it acceptance test, but system
        test? I am not so familiar to these definitions.

        The problem here is "How to control the real world in the terms of
        testing?". Making a value change to the device B can sometimes be a very
        hard job in our application, but to device A, we only need to ensure it can
        read a value back from B. If extern to the wireless mode, things can be
        even impossible, since there is no idea to control the GSM/GPRS network.



        > [Non-text portions of this message have been removed]
        >
        >
        >



        --
        Life is the only flaw in an otherwise perfect nonexistence
        -- Schopenhauer

        narke
        public key at http://subkeys.pgp.net:11371 (narkewoody@...)


        [Non-text portions of this message have been removed]
      • Steven Smith
        I think you re both right. In any system, you want to have a mixture of unit, integration, and acceptance tests. The exact mix of how many of these you d
        Message 3 of 7 , Apr 26, 2011
          I think you're both right. In any system, you want to have a mixture of
          unit, integration, and acceptance tests. The exact mix of how many of these
          you'd like to have (and how many you actually end up writing) will vary
          based on many factors, such as your team's experience with testing (and each
          kind of test) and the details of the application itself. Generally, you
          want to write as many tests as possible at the unit and integration end of
          the scale, as these tend to run faster and break less frequently than
          acceptance tests, which in many cases must be done manually in addition to
          having many more dependencies. I think the best way to approach the
          argument with your peers is to accept their side of the argument, but make
          the point that this kind of testing is a "defense in depth" approach, and
          there are things you could be catching with your unit and integration tests
          more easily and cheaply than with the acceptance tests on real hardware.

          Steve


          On Tue, Apr 26, 2011 at 10:00 AM, Steven Woody <narkewoody@...> wrote:

          >
          >
          > Hi,
          >
          > Our product is an embedded device. Looking externally, it is a
          > powered box, say A. On one side of the box, their is a USB interface
          > that need to be connected with another device B. Inside the box, there
          > exists a wireless modem chip C. Our system, basically, is to read
          > values from A, and make it readable via the wireless modem from a
          > client program running on PC.
          >
          > User cannot directly play with the box A, he has to operate with
          > device B and play with a PC read program on a PC.
          >
          > Now, we concern how the acceptance test for our box A should be done.
          > My options is to mock device A and modem C, since I think the system
          > under test is just A. But many colleagues think this kind of test is
          > unreal and want to put the real device B and real modem C together in
          > a test setup.
          >
          > My reasoning has not yet satisfying them so much. Would you please
          > tell me your opinion and help me in argue with them?
          >
          > Thanks in advance.
          >
          > --
          > Life is the only flaw in an otherwise perfect nonexistence
          > -- Schopenhauer
          >
          > narke
          > public key at http://subkeys.pgp.net:11371 (narkewoody@...)
          >
          >



          --
          Steve Smith
          http://SteveSmithBlog.com/
          http://twitter.com/ardalis


          [Non-text portions of this message have been removed]
        • Curtis Cooley
          ... This characterizes what I meant, but much more clearly. Thanks Steven. -- Curtis Cooley curtis.cooley@gmail.com home:http://curtiscooley.com
          Message 4 of 7 , Apr 26, 2011
            On Tue, Apr 26, 2011 at 7:38 AM, Steven Smith <ssmith.lists@...> wrote:
            > I think you're both right.  In any system, you want to have a mixture of
            > unit, integration, and acceptance tests.  The exact mix of how many of these
            > you'd like to have (and how many you actually end up writing) will vary
            > based on many factors, such as your team's experience with testing (and each
            > kind of test) and the details of the application itself.  Generally, you
            > want to write as many tests as possible at the unit and integration end of
            > the scale, as these tend to run faster and break less frequently than
            > acceptance tests, which in many cases must be done manually in addition to
            > having many more dependencies.  I think the best way to approach the
            > argument with your peers is to accept their side of the argument, but make
            > the point that this kind of testing is a "defense in depth" approach, and
            > there are things you could be catching with your unit and integration tests
            > more easily and cheaply than with the acceptance tests on real hardware.
            >

            This characterizes what I meant, but much more clearly. Thanks Steven.
            --
            Curtis Cooley
            curtis.cooley@...
            home:http://curtiscooley.com
            blog:http://ponderingobjectorienteddesign.blogspot.com
            ===============
            Leadership is a potent combination of strategy and character. But if
            you must be without one, be without the strategy.
            -- H. Norman Schwarzkopf
          • Steven Woody
            Thanks a lot!! I basically think same with you, but your language is so much better! On Apr 26, 2011 10:39 PM, Steven Smith wrote:
            Message 5 of 7 , Apr 26, 2011
              Thanks a lot!! I basically think same with you, but your language is so much
              better!
              On Apr 26, 2011 10:39 PM, "Steven Smith" <ssmith.lists@...> wrote:


              [Non-text portions of this message have been removed]
            Your message has been successfully submitted and would be delivered to recipients shortly.