Loading ...
Sorry, an error occurred while loading the content.

Re: [XP] Is there anyone who uses xp methods in embedded applications?

Expand Messages
  • William Rutiser
    This podcast may be of interest. Nancy Van Schooenderwoer and Bob Payne talk about her work in the agile embedded software space. She talks about her work,
    Message 1 of 14 , Jun 1, 2006
    • 0 Attachment
      This podcast may be of interest.
      Nancy Van Schooenderwoer and Bob Payne "talk about her work in the agile
      embedded software space. She talks about her work, the challenges of
      instrumenting embedded software for automated testing and the future of
      a Fit Fixture for the embedded software world."

      http://agiletoolkit.libsyn.com/index.php?post_id=2081

      -- Bill Rutiser


      Greg Akins wrote:
      > You can try this article http://atomicobject.com/papers.page
      >
      > I know a guy at Union Switch & Signal in Pittsburgh who is trying (with
      > varying levels of success) to do unit testing of embedded C; but don't know
      > much of the details.
      >
      > On 6/1/06, Micron Engineering <micronpn@...> wrote:
      >
      >> I would like to know some opinions about unit test implementations in
      >> embedded applications (mainly in C and C++).
      >> Thanks,
      >>
    • Russel Hill
      ... We re already using Fit with embedded software. To be fair, these platforms have 8MB RAM, 4MB Flash, and a network connection. One these platforms we also
      Message 2 of 14 , Jun 1, 2006
      • 0 Attachment
        On 6/1/06, William Rutiser <wruyahoo05@...> wrote:
        > This podcast may be of interest.
        > Nancy Van Schooenderwoer and Bob Payne "talk about her work in the agile
        > embedded software space. She talks about her work, the challenges of
        > instrumenting embedded software for automated testing and the future of
        > a Fit Fixture for the embedded software world."

        We're already using Fit with embedded software. To be fair, these
        platforms have 8MB RAM, 4MB Flash, and a network connection. One these
        platforms we also run a test suite against the FPGAs behavior.

        We've done XP on less capable embedded platforms, but they predate our
        adoption of Fit. We had tests for that code running on our dev box but
        not on the embedded platform. It was reasonable painful when we did
        run into a platform specific issue (sqrt on the embedded target didn't
        work). However, since we had good test coverage (on the dev box), we
        pretty much knew we were looking for either a compiler or a library
        bug. If I had to do that over agian, I'd find / make a way for the
        tests to run against the embedded hardware. I don't think a Fit
        fixture would be all that hard...
      • Ian Collins
        ... Yes, lots. All the recent embedded products at my last company where developed using XP. My preferred unit test framework is one of the CppUnit variants.
        Message 3 of 14 , Jun 1, 2006
        • 0 Attachment
          Micron Engineering wrote:

          >I would like to know some opinions about unit test implementations in
          >embedded applications (mainly in C and C++).
          >
          >
          Yes, lots.

          All the recent embedded products at my last company where developed
          using XP. My preferred unit test framework is one of the CppUnit variants.

          All of the products were developed using TDD in a host environment, with
          acceptance tests that cold be run against a simulation on the host, or a
          real target. We tried various levels of hardware simulations, including
          a very detailed model of the embedded microcontroller which enabled us
          to TDD device drivers.

          We used the same tools for both C and C++ targets, the detailed
          simulations required the C code to be built as C++ which has its pros
          and cons.

          If you want any specific detail, please ask.

          Ian
        • Micron Engineering
          ... I read some papers about unit tests and would like to compare my actual approach with the list to see if my be improved or if it has a bad implementation.
          Message 4 of 14 , Jun 1, 2006
          • 0 Attachment
            Ian Collins ha scritto:
            > Micron Engineering wrote:
            >
            >
            >> I would like to know some opinions about unit test implementations in
            >> embedded applications (mainly in C and C++).
            >>
            >>
            >>
            > Yes, lots.
            >
            > All the recent embedded products at my last company where developed
            > using XP. My preferred unit test framework is one of the CppUnit variants.
            >
            > All of the products were developed using TDD in a host environment, with
            > acceptance tests that cold be run against a simulation on the host, or a
            > real target. We tried various levels of hardware simulations, including
            > a very detailed model of the embedded microcontroller which enabled us
            > to TDD device drivers.
            >
            > We used the same tools for both C and C++ targets, the detailed
            > simulations required the C code to be built as C++ which has its pros
            > and cons.
            >
            > If you want any specific detail, please ask.
            >

            I read some papers about unit tests and would like to compare my actual
            approach with the list to see if my be improved or if it has a bad
            implementation.
            Mainly I develop embedded sw for custom boards based around a
            microcontroller (from 8 to 32 bits) and some sw on normal PCs (mainly
            for Windows o.s. and sometimes for Linux o.s.). So for little
            microcontrollers I use C compilers for all the other applications C++
            compilers (for Windows Borland C++ Buolder, for Linux Kylix3 and gcc C++
            compilers and WxWidgets).

            C projects
            I divide the sw in modules where module = file and every module has its
            test functions and its main() (for embedded applications the startup
            function is common to every module). This means that I write one or more
            test functions for every function contained in the file. Now, to manage
            better the test functions I decide to use this approach:
            //------------------------------------------------------------------------------
            // MODULE CODE STARTS HERE
            //------------------------------------------------------------------------------
            ....

            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            // UNIT TEST FILE
            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            #if REGRESSION_NET == 1
            #include ".\UnitTests\NetRegressionTest.c"
            #endif

            so simply using a define inside the module or at compile time I may
            enable or not unit test code and confine it in a different file (also in
            a different directory for CVS purpose, in this way I may have also a
            unit test project thinking in terms of cvs).
            I read about stub functions to substitute functions called inside a
            function to test. Actually I am not using them also if I understand that
            may be better isolate any single test but I really don't understand the
            way to do that.

            For example:

            long IntSum(int a, int b)
            {
            return(a + b);
            }

            int Series(int *iSum, int *iData, int len)
            {
            int i, j;
            for(i=0, j=0; i<len; i+=2, j++)
            iSum[j] = IntSum(iData[i], iData[i+1]);
            return(j);
            }

            to test IntSum() I need to test the returned value to see if it is
            correct. To test Series() I may understand what to test but I don't
            understand what may be the stub function and if it is a very advantage
            to write the stub function.

            C++ projects
            In this case test module = class so test functions are inside the class
            in public section (inside a #if ... #endif block).
            I read some papers about unit tests and would like to compare my actual
            approach with the list to see if my be improved or if it has a bad
            implementation.
            Mainly I develop embedded sw for custom boards based around a
            microcontroller (from 8 to 32 bits) and some sw on normal PCs (mainly
            for Windows o.s. and sometimes for Linux o.s.). So for little
            microcontrollers I use C compilers for all the other applications C++
            compilers (for Windows Borland C++ Buolder, for Linux Kylix3 and gcc C++
            compilers and WxWidgets).

            C projects
            I divide the sw in modules where module = file and every module has its
            test functions and its main() (for embedded applications the startup
            function is common to every module). This means that I write one or more
            test functions for every function contained in the file. Now, to manage
            better the test functions I decide to use this approach:
            //------------------------------------------------------------------------------
            // MODULE CODE STARTS HERE
            //------------------------------------------------------------------------------
            ....

            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            // UNIT TEST FILE
            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            //------------------------------------------------------------------------------
            #if REGRESSION_NET == 1
            #include ".\UnitTests\NetRegressionTest.c"
            #endif

            so simply using a define inside the module or at compile time I may
            enable or not unit test code and confine it in a different file (also in
            a different directory for CVS purpose, in this way I may have also a
            unit test project thinking in terms of cvs).
            I read about stub functions to substitute functions called inside a
            function to test. Actually I am not using them also if I understand that
            may be better isolate any single test but I really don't understand the
            way to do that.

            For example:

            long IntSum(int a, int b)
            {
            return(a + b);
            }

            int Series(int *iSum, int *iData, int len)
            {
            int i, j;
            for(i=0, j=0; i<len; i+=2, j++)
            iSum[j] = IntSum(iData[i], iData[i+1]);
            return(j);
            }

            to test IntSum() I need to test the returned value to see if it is
            correct. To test Series() I may understand what to test but I don't
            understand what may be the stub function and if it is a very advantage
            to write the stub function.

            C++ projects
            In this case test module = class so test functions are inside the class
            in public section (inside a #if ... #endif block).

            Are there better ways to realize unit tests?

            > Ian
            >
            >
            > To Post a message, send it to: extremeprogramming@...
            >
            > To Unsubscribe, send a blank message to: extremeprogramming-unsubscribe@...
            >
            > ad-free courtesy of objectmentor.com
            > Yahoo! Groups Links
            >
            >
            >
            >
            >
            >
            >
            >
            >
            >


            ----------

            No virus found in this outgoing message.
            Checked by AVG Free Edition.
            Version: 7.1.394 / Virus Database: 268.7.4/351 - Release Date: 29/05/2006


            [Non-text portions of this message have been removed]
          • Ian Collins
            ... I get the impression you are adding unit tests rather than doing TDD? Have a look at CppUnit or CppUnitLight. I haven t used the latter, but it appears to
            Message 5 of 14 , Jun 1, 2006
            • 0 Attachment
              Micron Engineering wrote:

              >Ian Collins ha scritto:
              >
              >
              >>Micron Engineering wrote:
              >>
              >>
              >>
              >>
              >>>I would like to know some opinions about unit test implementations in
              >>>embedded applications (mainly in C and C++).
              >>>
              >>>
              >>>
              >>>
              >>>
              >>Yes, lots.
              >>
              >>All the recent embedded products at my last company where developed
              >>using XP. My preferred unit test framework is one of the CppUnit variants.
              >>
              >>All of the products were developed using TDD in a host environment, with
              >>acceptance tests that cold be run against a simulation on the host, or a
              >>real target. We tried various levels of hardware simulations, including
              >>a very detailed model of the embedded microcontroller which enabled us
              >>to TDD device drivers.
              >>
              >>We used the same tools for both C and C++ targets, the detailed
              >>simulations required the C code to be built as C++ which has its pros
              >>and cons.
              >>
              >>If you want any specific detail, please ask.
              >>
              >>
              >>
              >
              >I read some papers about unit tests and would like to compare my actual
              >approach with the list to see if my be improved or if it has a bad
              >implementation.
              >
              >

              I get the impression you are adding unit tests rather than doing TDD?

              Have a look at CppUnit or CppUnitLight. I haven't used the latter, but
              it appears to be popular.

              Look how these frameworks build their test cases and suites, notice that
              you do not have to pollute your target code with test code.

              I use CppUnit to test all of my C code, it works well with a couple of
              fiddles, mainly a macro for 'static' to make static data and function
              visible to the test harness.

              Have a good look and come back with some specific questions, I can give
              you some examples.

              Cheers,

              Ian.
            • Russel Hill
              ... I like to split my tests into a separate compilation unit. This has a couple of distinct advantages: 1) I have the liberty of writing C++ test against C
              Message 6 of 14 , Jun 1, 2006
              • 0 Attachment
                On 6/1/06, Micron Engineering <micronpn@...> wrote:
                > C projects
                > I divide the sw in modules where module = file and every module has its
                > test functions and its main() (for embedded applications the startup
                > function is common to every module). This means that I write one or more
                > test functions for every function contained in the file. Now, to manage
                > better the test functions I decide to use this approach:

                I like to split my tests into a separate compilation unit. This has a
                couple of distinct advantages:

                1) I have the liberty of writing C++ test against C code (I like this one)
                2) I can compile application code _once_ and then use the linker to
                control whether the test harness or main is in control.
                3) I like it better that way.

                > I read about stub functions to substitute functions called inside a
                > function to test. Actually I am not using them also if I understand that
                > may be better isolate any single test but I really don't understand the
                > way to do that.

                Stub functions are particularly troublesome in 'C' (and some C++). It
                is possible to stub functions (or entire libraries) at link time. This
                is not a trick that I keep close to the top of my bag but it is
                occassionally useful. This would be another reason to keep test/stub
                code in separate compilation units.

                > to test IntSum() I need to test the returned value to see if it is
                > correct. To test Series() I may understand what to test but I don't
                > understand what may be the stub function and if it is a very advantage
                > to write the stub function.

                In your example, there isn't much reason for a stub. Consider:

                void toggleResetBit()
                {
                *controlRegister |= resetBit;
                *controlRegister &= ~resetBit;
                }

                void reconfigure()
                {
                toggleResetBit()
                // do a bunch of other stuff
                }

                If you want to test that reconfigure actually calls toggleResetBit(),
                then you could link in a different implementation of toggleResetBit(),
                such as:

                void toggleResetBit()
                {
                someoneCalledToggleResetBit = TRUE;
                }

                and then:

                void testReconfigure()
                {
                reconfigure();
                Assert(someoneCalledToggleResetBit);
                }

                Whether this is an important test to write, or not, is between you,
                your team, and your customer(s).

                Testing that toggleResetBit does what you expect is an entirely
                different problem. Is left as an exercise for the interested reader.
              • Micron Engineering
                ... I have a separate c file but I have to include it on original file to test (using #if...#endif to compile or not) and this to test also static functions
                Message 7 of 14 , Jun 1, 2006
                • 0 Attachment
                  Russel Hill ha scritto:
                  > On 6/1/06, Micron Engineering <micronpn@...> wrote:
                  >
                  >> C projects
                  >> I divide the sw in modules where module = file and every module has its
                  >> test functions and its main() (for embedded applications the startup
                  >> function is common to every module). This means that I write one or more
                  >> test functions for every function contained in the file. Now, to manage
                  >> better the test functions I decide to use this approach:
                  >>
                  >
                  > I like to split my tests into a separate compilation unit. This has a
                  > couple of distinct advantages:
                  >
                  I have a separate c file but I have to include it on original file to
                  test (using #if...#endif to compile or not) and this to test also static
                  functions and variables.
                  > 1) I have the liberty of writing C++ test against C code (I like this one)
                  >
                  This means that you have a C++ compiler for the embedded cpu, I haven't
                  for all cpu I used (example Z8Encore, eZ80Acclaim)
                  > 2) I can compile application code _once_ and then use the linker to
                  > control whether the test harness or main is in control.
                  >
                  How do you test static functions? And how can you use static variables?
                  > 3) I like it better that way.
                  >
                  >
                  Th
                  >> I read about stub functions to substitute functions called inside a
                  >> function to test. Actually I am not using them also if I understand that
                  >> may be better isolate any single test but I really don't understand the
                  >> way to do that.
                  >>
                  >
                  > Stub functions are particularly troublesome in 'C' (and some C++). It
                  > is possible to stub functions (or entire libraries) at link time. This
                  > is not a trick that I keep close to the top of my bag but it is
                  > occassionally useful. This would be another reason to keep test/stub
                  > code in separate compilation units.
                  >
                  >
                  >> to test IntSum() I need to test the returned value to see if it is
                  >> correct. To test Series() I may understand what to test but I don't
                  >> understand what may be the stub function and if it is a very advantage
                  >> to write the stub function.
                  >>
                  >
                  > In your example, there isn't much reason for a stub. Consider:
                  >
                  Yes, my example was a "difficult" example, I choose it because I read
                  articles about a stub for every dependant function so I choose one
                  difficult to implement to understand better; the most difficult are
                  functions where the returned type depends from one or more input parameters.
                  >


                  ----------

                  No virus found in this outgoing message.
                  Checked by AVG Free Edition.
                  Version: 7.1.394 / Virus Database: 268.7.4/351 - Release Date: 29/05/2006


                  [Non-text portions of this message have been removed]
                • Russel Hill
                  ... I try to minimize my use of static and, given the choice, I prefer using the trick Ian mentioned when I need to directly access statics. ... Actually, it
                  Message 8 of 14 , Jun 1, 2006
                  • 0 Attachment
                    On 6/1/06, Micron Engineering <micronpn@...> wrote:
                    > I have a separate c file but I have to include it on original file to
                    > test (using #if...#endif to compile or not) and this to test also static
                    > functions and variables.

                    I try to minimize my use of static and, given the choice, I prefer
                    using the trick Ian mentioned when I need to directly access statics.


                    > > 1) I have the liberty of writing C++ test against C code (I like this one)
                    > >
                    > This means that you have a C++ compiler for the embedded cpu, I haven't
                    > for all cpu I used (example Z8Encore, eZ80Acclaim)

                    Actually, it means that my programmer tests run in an environment that
                    has a C++ compiler. Which need not be on the embedded CPU. We develop
                    Coldfire apps on x86 linux boxes. Virtually all the programmer tests
                    run on the x86. When we move to the Coldfire, we mostly run acceptance
                    tests (using Fitnesse).

                    > How do you test static functions? And how can you use static variables?

                    1) Minimize the use of static
                    2) Clever use of macros:

                    #ifdef TESTING
                    #define STATIC
                    #else
                    #define STATIC static
                    #endif

                    Or, something like that...

                    > >
                    > > In your example, there isn't much reason for a stub. Consider:
                    > >
                    > Yes, my example was a "difficult" example, I choose it because I read
                    > articles about a stub for every dependant function so I choose one
                    > difficult to implement to understand better; the most difficult are
                    > functions where the returned type depends from one or more input parameters.

                    I only stub or mock where I must, or where it makes my life simpler.
                    Dogmatically stubbing every dependent functions isn're required and
                    wouldn't make my life simpler.
                  • Keith Braithwaite
                    ... A couple of jobs ago I did a lot of unit test intensive work in Symbian C++. Out of that sprang SymbianOSUnit (http://www.symbianosunit.co.uk/ a CxxUnit
                    Message 9 of 14 , Jun 2, 2006
                    • 0 Attachment
                      --- In extremeprogramming@yahoogroups.com, Micron Engineering
                      <micronpn@...> wrote:
                      >
                      > I would like to know some opinions about unit test implementations in
                      > embedded applications (mainly in C and C++).

                      A couple of jobs ago I did a lot of unit test intensive work in
                      Symbian C++. Out of that sprang SymbianOSUnit
                      (http://www.symbianosunit.co.uk/ a CxxUnit port) You might find some
                      ideas there.

                      Also, Nancy Van Schooenderwoert and I did a presentation at XPDay 3
                      that has some discussion of unit testing for embedded work. See
                      http://www.keithbraithwaite.demon.co.uk/professional/presentations/index.html#embedded

                      Keith
                    • James Grenning
                      There are people using XP for embedded. We have clients using CppTestTools, a variant of CppUnitLite that also includes support for FitNesse. One of the
                      Message 10 of 14 , Jun 8, 2006
                      • 0 Attachment
                        There are people using XP for embedded. We have clients using
                        CppTestTools, a variant of CppUnitLite that also includes support for
                        FitNesse.

                        One of the challenges is dual targeting your code. Getting a fast
                        turn around time build environment is key to using XP and TDD for
                        embedded. In a nutshell you get your code working first in the
                        friendly confines for your development machine. When you have test
                        passing you periodically build and test for the target. I have a few
                        papers on the topic on our website
                        http://www.objectmentor.com/resources/listArticles?key=topic&topic=Embedded%20Software

                        We have clients doing dual targeted Unit tests and FitNesse based
                        acceptance testing on the development machine. We also have a couple
                        clients that use PC based FitNesse with fixtures that communicate to
                        the target either through serial, parallel or USB. Some of this work
                        is done using already exiting serial command line communications, or
                        the adding of an RPC kind of mechanism to poke the buttons on the
                        embedded software.

                        James

                        ------------------------
                        James W Grenning
                        Object Mentor, Inc.
                        Gurnee, IL 60031
                        www.objectmentor.com
                        blog.objectmentor.com
                        (847)438-9942
                        ------------------------



                        --- In extremeprogramming@yahoogroups.com, Micron Engineering
                        <micronpn@...> wrote:
                        >
                        > I would like to know some opinions about unit test implementations in
                        > embedded applications (mainly in C and C++).
                        > Thanks,
                        >
                        > ----------
                        >
                        > No virus found in this outgoing message.
                        > Checked by AVG Free Edition.
                        > Version: 7.1.394 / Virus Database: 268.7.4/351 - Release Date:
                        29/05/2006
                        >
                        >
                        > [Non-text portions of this message have been removed]
                        >
                      • James Grenning
                        Running the tests in the target is really important. Ideally you won t find many problems when you get there when you have a host based testing practice in
                        Message 11 of 14 , Jun 8, 2006
                        • 0 Attachment
                          Running the tests in the target is really important. Ideally you
                          won't find many problems when you get there when you have a host based
                          testing practice in place

                          In porting CppTestTools to the hitachi SH3 we found that the good old
                          c library function strstr does not work the same as the all the other
                          strstr implementations. The tests really helped to narrow it down.

                          James
                          ------------------------
                          James W Grenning
                          Object Mentor, Inc.
                          Gurnee, IL 60031
                          www.objectmentor.com
                          blog.objectmentor.com
                          (847)438-9942
                          ------------------------

                          --- In extremeprogramming@yahoogroups.com, "Russel Hill"
                          <russh347@...> wrote:
                          >
                          > On 6/1/06, William Rutiser <wruyahoo05@...> wrote:

                          > work). However, since we had good test coverage (on the dev box), we
                          > pretty much knew we were looking for either a compiler or a library
                          > bug. If I had to do that over agian, I'd find / make a way for the
                          > tests to run against the embedded hardware. I don't think a Fit
                          > fixture would be all that hard...
                          >
                        Your message has been successfully submitted and would be delivered to recipients shortly.