Loading ...
Sorry, an error occurred while loading the content.

Re: [junit] Re: Runner for maven

Expand Messages
  • Dawid Weiss
    ... I was thinking about ANT actually, not maven (surefire). I admit we also needed some extra control over suite allocation scheduling and more detailed
    Message 1 of 12 , Jan 5, 2012
    View Source
    • 0 Attachment
      > maven-surefire now also supports concurrent forks, and they don't have any of the interaction problems you describe - as you know. Since surefire is an > all-purpose runner, we try to support any "sensible" mode of execution, and there are good use cases for both models of execution.

      I was thinking about ANT actually, not maven (surefire). I admit we
      also needed some extra control over suite allocation scheduling and
      more detailed reporting, so surefire wasn't a perfect fit for us. It
      is a great tool though, I agree with that.

      Dawid
    • Dawid Weiss
      Oh, I also didn t mention one more thing -- if you have tests that fork subthreads which in turn write to System.out/err then you will also need a smarter way
      Message 2 of 12 , Jan 5, 2012
      View Source
      • 0 Attachment
        Oh, I also didn't mention one more thing -- if you have tests that
        fork subthreads which in turn write to System.out/err then you will
        also need a smarter way to track which suite a thread belongs to.
        This can be done with a parent thread group (in most cases works) but
        like you said -- I think it's just a pain and there is no way to
        implement it correctly for all possible use cases.

        Dawid

        On Thu, Jan 5, 2012 at 12:52 PM, Dawid Weiss <dawid.weiss@...> wrote:
        >> maven-surefire now also supports concurrent forks, and they don't have any of the interaction problems you describe - as you know. Since surefire is an > all-purpose runner, we try to support any "sensible" mode of execution, and there are good use cases for both models of execution.
        >
        > I was thinking about ANT actually, not maven (surefire). I admit we
        > also needed some extra control over suite allocation scheduling and
        > more detailed reporting, so surefire wasn't a perfect fit for us. It
        > is a great tool though, I agree with that.
        >
        > Dawid
      • Esko Luontola
        I m already working on something like that. I m writing a new test runner called Jumi http://jumi.fi/ which will have native support for running tests in
        Message 3 of 12 , Jan 6, 2012
        View Source
        • 0 Attachment
          I'm already working on something like that. I'm writing a new test
          runner called Jumi http://jumi.fi/ which will have native support for
          running tests in parallel in the same JVM, and it take care of capturing
          the System.out/err of individual tests.

          Jumi will do the output capturing by replacing System.out/err with its
          own implementation, which saves all print events (the same way it saves
          test lifecycle events), synchronizes both out and err (I think all
          current IDEs and build tools fail this:
          https://github.com/orfjackal/misc-tools/blob/master/src/test/java/net/orfjackal/bugs/SystemOutErrBugTest.java),
          and also takes care of capturing the output of threads spawned by the
          test thread. For long-lived threads, such as AWT event dispatch thread,
          I will also try to figure out something (I'll timestamp all print
          events, so it should be possible to figure out roughly which test
          printed things in the AWT thread).

          The output capturing is at the top of my TODO list, so I will have it
          implemented after a couple of productive afternoons when I have time to
          work on it. Soon after that I'll make the first alpha release.

          I've looked a bit at how to integrate Jumi with maven-surefire. Jumi and
          Surefire have some overlap, for example both of them take care of
          classloading and starting a JVM process. One option is to use Surefire's
          classloading and use only the core runner parts of Jumi. Otherwise Jumi
          would need information that what are the dependency JARs, what are the
          current project's classes, maybe also that which of them are test and
          production classes (Jumi will do caching of the dependencies' class
          loaders to speed up the tests). That would require changes to Surefire,
          or then a completely new Maven plugin.


          kristian@... wrote on 4.1.2012 8:51:
          > Sorry, I was doing just what I sometimes dislike of our own users ;)
          >
          > The real problem I want to solve is associating console output from a
          > running thread to a given test-case (for parallel runs). We have this
          > fairly well covered already and we manage to capture everything that
          > happens within a given test-case for that test-case.
          >
          > The real problem is that the RunListener API does not demarcate
          > test-classes properly, so I am unable to distinguish console output
          > happening in @AfterClass of one test with @BeforeClass on the test-class
          > on the same thread.
          >
          > This problem has been in the corner of my eye for quite some time and
          > I've been trying to find a decent angle to solve it from. I promise not
          > to do anything that is frowned upon, which is why I asked ;)
          >
          > Kristian
          >

          --
          Esko Luontola
          www.orfjackal.net
        • Dawid Weiss
          You may want to look into RandomizedRunner/JUnit4 task -- it has most of the things you mentioned (synchronized i/o events with reporting, capturing runaway
          Message 4 of 12 , Jan 6, 2012
          View Source
          • 0 Attachment
            You may want to look into RandomizedRunner/JUnit4 task -- it has most
            of the things you mentioned (synchronized i/o events with reporting,
            capturing runaway threads and the like).

            The code is here:
            https://github.com/carrotsearch/randomizedtesting/

            I have a talk about randomized testing which includes the above
            concepts at Lucene EuroCon Barcelona, the video is here:
            http://vimeo.com/32087114

            Dawid

            On Fri, Jan 6, 2012 at 1:29 PM, Esko Luontola <esko.luontola@...> wrote:
            >
            >
            >
            > I'm already working on something like that. I'm writing a new test
            > runner called Jumi http://jumi.fi/ which will have native support for
            > running tests in parallel in the same JVM, and it take care of capturing
            > the System.out/err of individual tests.
            >
            > Jumi will do the output capturing by replacing System.out/err with its
            > own implementation, which saves all print events (the same way it saves
            > test lifecycle events), synchronizes both out and err (I think all
            > current IDEs and build tools fail this:
            > https://github.com/orfjackal/misc-tools/blob/master/src/test/java/net/orfjackal/bugs/SystemOutErrBugTest.java),
            > and also takes care of capturing the output of threads spawned by the
            > test thread. For long-lived threads, such as AWT event dispatch thread,
            > I will also try to figure out something (I'll timestamp all print
            > events, so it should be possible to figure out roughly which test
            > printed things in the AWT thread).
            >
            > The output capturing is at the top of my TODO list, so I will have it
            > implemented after a couple of productive afternoons when I have time to
            > work on it. Soon after that I'll make the first alpha release.
            >
            > I've looked a bit at how to integrate Jumi with maven-surefire. Jumi and
            > Surefire have some overlap, for example both of them take care of
            > classloading and starting a JVM process. One option is to use Surefire's
            > classloading and use only the core runner parts of Jumi. Otherwise Jumi
            > would need information that what are the dependency JARs, what are the
            > current project's classes, maybe also that which of them are test and
            > production classes (Jumi will do caching of the dependencies' class
            > loaders to speed up the tests). That would require changes to Surefire,
            > or then a completely new Maven plugin.
            >
            > kristian@... wrote on 4.1.2012 8:51:
            >
            >
            > > Sorry, I was doing just what I sometimes dislike of our own users ;)
            > >
            > > The real problem I want to solve is associating console output from a
            > > running thread to a given test-case (for parallel runs). We have this
            > > fairly well covered already and we manage to capture everything that
            > > happens within a given test-case for that test-case.
            > >
            > > The real problem is that the RunListener API does not demarcate
            > > test-classes properly, so I am unable to distinguish console output
            > > happening in @AfterClass of one test with @BeforeClass on the test-class
            > > on the same thread.
            > >
            > > This problem has been in the corner of my eye for quite some time and
            > > I've been trying to find a decent angle to solve it from. I promise not
            > > to do anything that is frowned upon, which is why I asked ;)
            > >
            > > Kristian
            > >
            >
            > --
            > Esko Luontola
            > www.orfjackal.net
            >
            >
          Your message has been successfully submitted and would be delivered to recipients shortly.