Loading ...
Sorry, an error occurred while loading the content.
 

Tests starting before Logback initialisation complete when run in parallel

Expand Messages
  • Brett Delle Grazie
    Hi, I m using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35 on Windows 7 Surefire is configured with parallel=methods and threadCount=10 For
    Message 1 of 12 , Sep 3, 2012
      Hi,

      I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
      on Windows 7
      Surefire is configured with parallel=methods and threadCount=10
      For logging, I'm using SLF4J to Logback, initialised by default method
      (i.e. configuration file logback-test.xml is on the test classpath)

      I have an order of initialisation problem in that:
      (1) Some unit tests test classes that use commons-logging (and logback
      via jcl-slf4j) but are ending up with
      Null loggers.
      (2) jcl-slf4j is not initialised _until_ those classes are called.

      If I disable parallelism the code works correctly, implying an order
      of initialisation issue.

      According to the most recent FAQ on Junit.org (Section 5, #3) I could
      use @BeforeClass and @AfterClass at the class
      level but I'm talking about at the global level, something that needs
      to be executed even if only a single test is run.

      What is the correct method to add global, one-time, setup / tear-down
      hooks? In this case they would need to execute and
      complete (i.e. blocking) before any parallel tests are run.
      Is it simplest to override the Runner and initialise the logging in code there?

      Thanks,

      Best Regards,

      Brett
    • David Saff
      Brett, There s currently not a good single way to do what you want. There s current conversation on the github repo about how to do it in a future version,
      Message 2 of 12 , Sep 4, 2012
        Brett,

        There's currently not a good single way to do what you want. There's
        current conversation on the github repo about how to do it in a future
        version, but there's no great first-step answer.

        One possibility is to add a @BeforeClass or @ClassRule to each of your
        tests that would, in a thread-safe way, check if your initialization has
        already been performed, and if not, do it. I know that's not a great
        solution.

        David Saff


        On Mon, Sep 3, 2012 at 3:51 PM, Brett Delle Grazie <bdellegrazie@...>wrote:

        > **
        >
        >
        > Hi,
        >
        > I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
        > on Windows 7
        > Surefire is configured with parallel=methods and threadCount=10
        > For logging, I'm using SLF4J to Logback, initialised by default method
        > (i.e. configuration file logback-test.xml is on the test classpath)
        >
        > I have an order of initialisation problem in that:
        > (1) Some unit tests test classes that use commons-logging (and logback
        > via jcl-slf4j) but are ending up with
        > Null loggers.
        > (2) jcl-slf4j is not initialised _until_ those classes are called.
        >
        > If I disable parallelism the code works correctly, implying an order
        > of initialisation issue.
        >
        > According to the most recent FAQ on Junit.org (Section 5, #3) I could
        > use @BeforeClass and @AfterClass at the class
        > level but I'm talking about at the global level, something that needs
        > to be executed even if only a single test is run.
        >
        > What is the correct method to add global, one-time, setup / tear-down
        > hooks? In this case they would need to execute and
        > complete (i.e. blocking) before any parallel tests are run.
        > Is it simplest to override the Runner and initialise the logging in code
        > there?
        >
        > Thanks,
        >
        > Best Regards,
        >
        > Brett
        >
        >


        [Non-text portions of this message have been removed]
      • Brett Delle Grazie
        Hi David, ... Thanks David, I ll look into doing that. ... -- Best Regards, Brett Delle Grazie
        Message 3 of 12 , Sep 5, 2012
          Hi David,

          On 4 September 2012 19:24, David Saff <saff@...> wrote:
          > Brett,
          >
          > There's currently not a good single way to do what you want. There's
          > current conversation on the github repo about how to do it in a future
          > version, but there's no great first-step answer.
          >
          > One possibility is to add a @BeforeClass or @ClassRule to each of your
          > tests that would, in a thread-safe way, check if your initialization has
          > already been performed, and if not, do it. I know that's not a great
          > solution.

          Thanks David, I'll look into doing that.

          >
          > David Saff
          >
          >
          > On Mon, Sep 3, 2012 at 3:51 PM, Brett Delle Grazie <bdellegrazie@...>wrote:
          >
          >> **
          >>
          >>
          >> Hi,
          >>
          >> I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
          >> on Windows 7
          >> Surefire is configured with parallel=methods and threadCount=10
          >> For logging, I'm using SLF4J to Logback, initialised by default method
          >> (i.e. configuration file logback-test.xml is on the test classpath)
          >>
          >> I have an order of initialisation problem in that:
          >> (1) Some unit tests test classes that use commons-logging (and logback
          >> via jcl-slf4j) but are ending up with
          >> Null loggers.
          >> (2) jcl-slf4j is not initialised _until_ those classes are called.
          >>
          >> If I disable parallelism the code works correctly, implying an order
          >> of initialisation issue.
          >>
          >> According to the most recent FAQ on Junit.org (Section 5, #3) I could
          >> use @BeforeClass and @AfterClass at the class
          >> level but I'm talking about at the global level, something that needs
          >> to be executed even if only a single test is run.
          >>
          >> What is the correct method to add global, one-time, setup / tear-down
          >> hooks? In this case they would need to execute and
          >> complete (i.e. blocking) before any parallel tests are run.
          >> Is it simplest to override the Runner and initialise the logging in code
          >> there?
          >>
          >> Thanks,
          >>
          >> Best Regards,
          >>
          >> Brett
          >>
          >>
          >
          >
          > [Non-text portions of this message have been removed]
          >
          >
          >
          > ------------------------------------
          >
          > Yahoo! Groups Links
          >
          >
          >



          --
          Best Regards,

          Brett Delle Grazie
        • Mirko Friedenhagen
          Hello, wild guess: do you have both commons-logging as well as jcl-over-slf4j in your test class path? Regards Mirko ... [Non-text portions of this message
          Message 4 of 12 , Sep 5, 2012
            Hello,

            wild guess: do you have both commons-logging as well as jcl-over-slf4j in
            your test class path?

            Regards Mirko

            On Mon, Sep 3, 2012 at 9:51 PM, Brett Delle Grazie <bdellegrazie@...>wrote:

            > **
            >
            >
            > Hi,
            >
            > I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
            > on Windows 7
            > Surefire is configured with parallel=methods and threadCount=10
            > For logging, I'm using SLF4J to Logback, initialised by default method
            > (i.e. configuration file logback-test.xml is on the test classpath)
            >
            > I have an order of initialisation problem in that:
            > (1) Some unit tests test classes that use commons-logging (and logback
            > via jcl-slf4j) but are ending up with
            > Null loggers.
            > (2) jcl-slf4j is not initialised _until_ those classes are called.
            >
            > If I disable parallelism the code works correctly, implying an order
            > of initialisation issue.
            >
            > According to the most recent FAQ on Junit.org (Section 5, #3) I could
            > use @BeforeClass and @AfterClass at the class
            > level but I'm talking about at the global level, something that needs
            > to be executed even if only a single test is run.
            >
            > What is the correct method to add global, one-time, setup / tear-down
            > hooks? In this case they would need to execute and
            > complete (i.e. blocking) before any parallel tests are run.
            > Is it simplest to override the Runner and initialise the logging in code
            > there?
            >
            > Thanks,
            >
            > Best Regards,
            >
            > Brett
            >
            >


            [Non-text portions of this message have been removed]
          • Brett Delle Grazie
            ... Hi Mirko, No, commons-logging is explicitly excluded. The problem is definitely due to parallelism configuration. Disabling parallelism via none in maven
            Message 5 of 12 , Sep 5, 2012
              On 5 September 2012 15:41, Mirko Friedenhagen <mfriedenhagen@...> wrote:
              > Hello,
              >
              > wild guess: do you have both commons-logging as well as jcl-over-slf4j in
              > your test class path?

              Hi Mirko,
              No, commons-logging is explicitly excluded. The problem is definitely
              due to parallelism
              configuration. Disabling parallelism via 'none' in maven resolves the
              issue but obviously
              executes the tests slower.

              Essentially what happens is Logback is still executing its
              auto-initialisation while some
              tests start to write to the logs resulting in lost log information.

              As noted by another post in this thread, I'll have to serialise the
              initialisation explicitly.

              Thanks,

              Brett
              >
              > Regards Mirko
              >
              > On Mon, Sep 3, 2012 at 9:51 PM, Brett Delle Grazie <bdellegrazie@...>wrote:
              >
              >> **
              >>
              >>
              >> Hi,
              >>
              >> I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
              >> on Windows 7
              >> Surefire is configured with parallel=methods and threadCount=10
              >> For logging, I'm using SLF4J to Logback, initialised by default method
              >> (i.e. configuration file logback-test.xml is on the test classpath)
              >>
              >> I have an order of initialisation problem in that:
              >> (1) Some unit tests test classes that use commons-logging (and logback
              >> via jcl-slf4j) but are ending up with
              >> Null loggers.
              >> (2) jcl-slf4j is not initialised _until_ those classes are called.
              >>
              >> If I disable parallelism the code works correctly, implying an order
              >> of initialisation issue.
              >>
              >> According to the most recent FAQ on Junit.org (Section 5, #3) I could
              >> use @BeforeClass and @AfterClass at the class
              >> level but I'm talking about at the global level, something that needs
              >> to be executed even if only a single test is run.
              >>
              >> What is the correct method to add global, one-time, setup / tear-down
              >> hooks? In this case they would need to execute and
              >> complete (i.e. blocking) before any parallel tests are run.
              >> Is it simplest to override the Runner and initialise the logging in code
              >> there?
              >>
              >> Thanks,
              >>
              >> Best Regards,
              >>
              >> Brett
              >>
            • hkaipe
              Such global, one-time, setup / tear-down is outside the scope of JUnit, according to me. You should turn to your test-running tool on this one, i.e.
              Message 6 of 12 , Sep 6, 2012
                Such "global, one-time, setup / tear-down" is outside the scope of JUnit, according to me. You should turn to your test-running tool on this one, i.e. Surefire.

                Surefire is your tool to organize which tests to run when and how. JUnit is glue between Surefire and the JUnit test-classes.

                About your problem I think you have put yourself in a corner by trying to run all tests in parallel in the same JVM. Even if you could use "global, one-time, setup / tear-down" you still put very strict limits on what your test-classes can safely do and I simply don't think it is worth it.

                I strongly advice you to rethink here and configure Surefire with forkMode="perthread", so that parallel test-executions take place in separate JVM-processes.

                cheers /Henrik

                --- In junit@yahoogroups.com, Brett Delle Grazie <bdellegrazie@...> wrote:
                >
                > Hi,
                >
                > I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
                > on Windows 7
                > Surefire is configured with parallel=methods and threadCount=10
                > For logging, I'm using SLF4J to Logback, initialised by default method
                > (i.e. configuration file logback-test.xml is on the test classpath)
                >
                > I have an order of initialisation problem in that:
                > (1) Some unit tests test classes that use commons-logging (and logback
                > via jcl-slf4j) but are ending up with
                > Null loggers.
                > (2) jcl-slf4j is not initialised _until_ those classes are called.
                >
                > If I disable parallelism the code works correctly, implying an order
                > of initialisation issue.
                >
                > According to the most recent FAQ on Junit.org (Section 5, #3) I could
                > use @BeforeClass and @AfterClass at the class
                > level but I'm talking about at the global level, something that needs
                > to be executed even if only a single test is run.
                >
                > What is the correct method to add global, one-time, setup / tear-down
                > hooks? In this case they would need to execute and
                > complete (i.e. blocking) before any parallel tests are run.
                > Is it simplest to override the Runner and initialise the logging in code there?
                >
                > Thanks,
                >
                > Best Regards,
                >
                > Brett
                >
              • Cédric Beust ♔
                ... Right, which is why I think a @BeforeSuite annotation belongs to the testing framework, not the build tool. How would your build tool create a connection
                Message 7 of 12 , Sep 6, 2012
                  On Thu, Sep 6, 2012 at 10:38 AM, hkaipe <hkaipe@...> wrote:

                  > Surefire is your tool to organize which tests to run when and how.


                  Right, which is why I think a @BeforeSuite annotation belongs to the
                  testing framework, not the build tool. How would your build tool create a
                  connection pool or an executor for your tests to use?

                  --
                  Cédric


                  [Non-text portions of this message have been removed]
                • Kristian Rosenvold
                  People *have* been known to make a microscopic Junit run-listener they attach to surefire to do this. I know this might give me flames ;) Check docs at
                  Message 8 of 12 , Sep 6, 2012
                    People *have* been known to make a microscopic Junit run-listener they
                    attach to surefire to do this. I know this might give me flames ;)

                    Check docs at
                    http://maven.apache.org/plugins/maven-surefire-plugin/examples/junit.html
                    and small sample project at
                    https://svn.apache.org/repos/asf/maven/surefire/trunk/surefire-integration-tests/target/JUnit4RunListenerIT/junit4-runlistener
                    for a sample on how to do this.

                    Kristian


                    2012/9/6 hkaipe <hkaipe@...>

                    > **
                    >
                    >
                    > Such "global, one-time, setup / tear-down" is outside the scope of JUnit,
                    > according to me. You should turn to your test-running tool on this one,
                    > i.e. Surefire.
                    >
                    > Surefire is your tool to organize which tests to run when and how. JUnit
                    > is glue between Surefire and the JUnit test-classes.
                    >
                    > About your problem I think you have put yourself in a corner by trying to
                    > run all tests in parallel in the same JVM. Even if you could use "global,
                    > one-time, setup / tear-down" you still put very strict limits on what your
                    > test-classes can safely do and I simply don't think it is worth it.
                    >
                    > I strongly advice you to rethink here and configure Surefire with
                    > forkMode="perthread", so that parallel test-executions take place in
                    > separate JVM-processes.
                    >
                    > cheers /Henrik
                    >
                    > --- In junit@yahoogroups.com, Brett Delle Grazie <bdellegrazie@...> wrote:
                    > >
                    > > Hi,
                    > >
                    > > I'm using JUnit 4.10 with Maven 3.0.4, Surefire 2.12.3, JDK 1.6.0_35
                    > > on Windows 7
                    > > Surefire is configured with parallel=methods and threadCount=10
                    > > For logging, I'm using SLF4J to Logback, initialised by default method
                    > > (i.e. configuration file logback-test.xml is on the test classpath)
                    > >
                    > > I have an order of initialisation problem in that:
                    > > (1) Some unit tests test classes that use commons-logging (and logback
                    > > via jcl-slf4j) but are ending up with
                    > > Null loggers.
                    > > (2) jcl-slf4j is not initialised _until_ those classes are called.
                    > >
                    > > If I disable parallelism the code works correctly, implying an order
                    > > of initialisation issue.
                    > >
                    > > According to the most recent FAQ on Junit.org (Section 5, #3) I could
                    > > use @BeforeClass and @AfterClass at the class
                    > > level but I'm talking about at the global level, something that needs
                    > > to be executed even if only a single test is run.
                    > >
                    > > What is the correct method to add global, one-time, setup / tear-down
                    > > hooks? In this case they would need to execute and
                    > > complete (i.e. blocking) before any parallel tests are run.
                    > > Is it simplest to override the Runner and initialise the logging in code
                    > there?
                    > >
                    > > Thanks,
                    > >
                    > > Best Regards,
                    > >
                    > > Brett
                    > >
                    >
                    >
                    >


                    [Non-text portions of this message have been removed]
                  • Malte Finsterwalder
                    This topic comes up again and again. Often not with the multithreading-aspect attached. So it s not uncommon, that people want to do one time initialization
                    Message 9 of 12 , Sep 7, 2012
                      This topic comes up again and again. Often not with the
                      multithreading-aspect attached.
                      So it's not uncommon, that people want to do one time initialization
                      and cleanup.
                      I had this problem two more than once.
                      One time initialization can be done more or less.
                      One time cleanup can't be done, really.

                      I would really like to see this in JUnit, since I execute my JUnit
                      tests in two different IDEs and in maven. So Changing the JUnit Runner
                      is really not a choice.

                      We already have RunListeners, which would be nice in theory, but does
                      not work well in IDEs.
                      So if I could somehow attach a RunListener, even though I don't call
                      the Runner myself, that would be really cool!

                      Greetings,
                      Malte
                    • Esko Luontola
                      ... Well, I m working on a new test runner (http://jumi.fi/) and one-time initialization and cleanup is definitely on the roadmap. This far I ve had to use
                      Message 10 of 12 , Sep 7, 2012
                        Malte Finsterwalder wrote on 7.9.2012 23:38:
                        > This topic comes up again and again. Often not with the
                        > multithreading-aspect attached.
                        > So it's not uncommon, that people want to do one time initialization
                        > and cleanup.
                        > I had this problem two more than once.
                        > One time initialization can be done more or less.
                        > One time cleanup can't be done, really.
                        >
                        > I would really like to see this in JUnit, since I execute my JUnit
                        > tests in two different IDEs and in maven. So Changing the JUnit Runner
                        > is really not a choice.

                        Well, I'm working on a new test runner (http://jumi.fi/) and one-time
                        initialization and cleanup is definitely on the roadmap. This far I've
                        had to use hacks like shutdown hooks (java.lang.Runtime.addShutdownHook)
                        and I too want proper support from testing frameworks.

                        One of the motivations of making Jumi is to have the tests executed the
                        same way regardless of whether an IDE or a build tool launches them. It
                        has a slightly larger scope than JUnit's test runner (including process
                        forking and test discovery) so adding this kind of hooks there may be a
                        bit easier than to JUnit. Though you'll need to wait a while before it
                        gets integration support from all the tools - the first release is to be
                        expected within a couple of months.

                        --
                        Esko Luontola
                        www.orfjackal.net
                      • hkaipe
                        ... I can see value in a @BeforeSuite-annotation but in general I think test-suites should be administered by the build-tool. - In this case SureFire, which
                        Message 11 of 12 , Sep 9, 2012
                          --- In junit@yahoogroups.com, Cédric Beust â™" <cedric@...> wrote:
                          >
                          > On Thu, Sep 6, 2012 at 10:38 AM, hkaipe <hkaipe@...> wrote:
                          >
                          > > Surefire is your tool to organize which tests to run when and how.
                          >
                          >
                          > Right, which is why I think a @BeforeSuite annotation belongs to the
                          > testing framework, not the build tool. How would your build tool create a
                          > connection pool or an executor for your tests to use?
                          >
                          > --
                          > Cédric

                          I can see value in a @BeforeSuite-annotation but in general I think test-suites should be administered by the build-tool. - In this case SureFire, which perhaps should support some suite/category-solution that can be applied to many test-frameworks, not only JUnit.

                          But Brett's particular problem is not limited to a test-suite or testing framework. It goes beyond testing. - He can accidently encounter it whenever he starts up a multi-threaded JVM, therewith making JVM-startup his problem domain. Therefore I think that using a javaagent would be a good pattern and - yes - my build tool can do that ... - In the Surefire-configuration you specify an "argLine" for the JVM:

                          <plugin>
                          <artifactId>maven-surefire-plugin</artifactId>
                          <configuration>
                          <argLine> -javaagent:${localRepository}/path/to/myagent/1.0-SNAPSHOT/myagent-1.0-SNAPSHOT.jar </argLine>
                          </configuration>
                          </plugin>

                          In the "Premain-Class" of "myagent" we can define JVM-wide setup and/or tearDown - e.g.

                          public class GlobalTearDown extends Thread
                          {
                          public static void premain(String args)
                          {
                          System.out.println("HELLO FROM premain");
                          Runtime.getRuntime().addShutdownHook(new GlobalTearDown());
                          }

                          public void run()
                          {
                          System.out.println("TEAR-DOWN EXECUTES HERE");
                          }
                          }

                          Output from Maven/Surefire when "GlobalTearDown" is used as "Premain-Class":
                          ---------------------------------------------------------------------
                          [surefire:test]
                          Surefire report directory: /home/me/projects/experiments/target/surefire-reports
                          HELLO FROM premain

                          -------------------------------------------------------
                          T E S T S
                          -------------------------------------------------------
                          Running org.callbackparams.experiments.DummyTest
                          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec

                          Results :

                          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

                          TEAR-DOWN EXECUTES HERE
                          ---------------------------------------------------------------------

                          I don't know how the "Premain-Class" should look to solve Brett's problem but I'm quite sure the problem can be elegantly solved with this pattern. - And his solution can perhaps be easily reused in other projects as well as outside of the testing activities.

                          /Henrik
                        • Brett Delle Grazie
                          ... Except when using an IDE, there I just want to: (1) select an individual test and have that run in a Debugger. Or (2) select a suite or class hierarchy and
                          Message 12 of 12 , Sep 10, 2012
                            On 9 September 2012 13:38, hkaipe <hkaipe@...> wrote:
                            > --- In junit@yahoogroups.com, Cédric Beust â™" <cedric@...> wrote:
                            >>
                            >> On Thu, Sep 6, 2012 at 10:38 AM, hkaipe <hkaipe@...> wrote:
                            >>
                            >> > Surefire is your tool to organize which tests to run when and how.
                            >>

                            Except when using an IDE, there I just want to:
                            (1) select an individual test and have that run in a Debugger. Or
                            (2) select a suite or class hierarchy and have the output appear in
                            the JUnit window (and log window if appropriate) confirming things are
                            still working.

                            >>
                            >> Right, which is why I think a @BeforeSuite annotation belongs to the
                            >> testing framework, not the build tool. How would your build tool create a
                            >> connection pool or an executor for your tests to use?

                            Agree completely here, with the caveat that _most_ situations would
                            probably be catered for by @BeforeClass.

                            Q: In multi-threaded mode: are multiple threads used to run the
                            @BeforeClass code?

                            >>
                            >> --
                            >> Cédric
                            >
                            > I can see value in a @BeforeSuite-annotation but in general I think test-suites should be administered by the build-tool. - In this case SureFire, which perhaps should support some suite/category-solution that can be applied to many test-frameworks, not only JUnit.

                            I agree partially here. Tests should easily selectable within an IDE,
                            in particular I want to be able to debug a specific test or run all
                            tests with the same convenience that I can execute the debugger
                            against the main project.
                            Using Maven + Surefire results in added complexity that isn't really necessary.
                            This fails with the IDE use case - IDE users quite often want to
                            execute a suite or all tests or a particular set of tests simply by
                            grouping... then they want to debug within that set.

                            >
                            > But Brett's particular problem is not limited to a test-suite or testing framework. It goes beyond testing. - He can accidently encounter it whenever he starts up a multi-threaded JVM, therewith making JVM-startup his problem domain. Therefore I think that using a javaagent would be a good pattern and - yes - my build tool can do that ... - In the Surefire-configuration you specify an "argLine" for the JVM:

                            I disagree here.
                            In applications (web or console based) I can already control
                            programmatically how the logging apparatus is instantiated, meaning
                            this isn't an issue
                            for me in the normal case. When using JUnit in parallel test mode I
                            can't control this at all

                            >
                            > <plugin>
                            > <artifactId>maven-surefire-plugin</artifactId>
                            > <configuration>
                            > <argLine> -javaagent:${localRepository}/path/to/myagent/1.0-SNAPSHOT/myagent-1.0-SNAPSHOT.jar </argLine>
                            > </configuration>
                            > </plugin>
                            >
                            > In the "Premain-Class" of "myagent" we can define JVM-wide setup and/or tearDown - e.g.
                            >
                            > public class GlobalTearDown extends Thread
                            > {
                            > public static void premain(String args)
                            > {
                            > System.out.println("HELLO FROM premain");
                            > Runtime.getRuntime().addShutdownHook(new GlobalTearDown());
                            > }
                            >
                            > public void run()
                            > {
                            > System.out.println("TEAR-DOWN EXECUTES HERE");
                            > }
                            > }
                            >
                            > Output from Maven/Surefire when "GlobalTearDown" is used as "Premain-Class":
                            > ---------------------------------------------------------------------
                            > [surefire:test]
                            > Surefire report directory: /home/me/projects/experiments/target/surefire-reports
                            > HELLO FROM premain
                            >
                            > -------------------------------------------------------
                            > T E S T S
                            > -------------------------------------------------------
                            > Running org.callbackparams.experiments.DummyTest
                            > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec
                            >
                            > Results :
                            >
                            > Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
                            >
                            > TEAR-DOWN EXECUTES HERE
                            > ---------------------------------------------------------------------
                            >
                            > I don't know how the "Premain-Class" should look to solve Brett's problem but I'm quite sure the problem can be elegantly solved with this pattern. - And his solution can perhaps be easily reused in other projects as well as outside of the testing activities.
                            >
                            > /Henrik
                            >

                            I appreciate you've gone to great lengths to propose a general 'fits
                            all' category of solution but I am concerned about the unintended
                            consequences such
                            behaviour might introduce (there might be none and I fully admit that
                            I don't rightly know). I do know that this type of shutdown hook
                            processing is the
                            wrong solution for web applications (they should hook into the
                            container life-cycle for such tasks).

                            Essentially we're talking about controlled initialisation and shutdown
                            of an application using whatever is the appropriate life-cycle. In
                            this context the
                            life-cycle is the initialisation and shutdown of JUnit or whatever
                            testing framework being used.

                            Containers / Frameworks that support multi-threading usually have a
                            single-threaded initialisation and shutdown phases to enforce
                            deterministic behaviour
                            during these critical phases before going into full mutli-threaded mode.

                            We are talking about a very specific case of behaviour which is
                            ultimately caused by parallel test execution without a guaranteed
                            deterministic stable
                            initialisation phase. It boils down to the simple fact that there is a
                            use case for something similar to:
                            @BeforeAll / @AfterAll (or any appropriate naming)
                            These should be executed serially rather than multi-threaded and
                            considered 'reserved' only for special global setup / tear-down
                            operations that must
                            be executed under single-threading conditions.

                            What's more, this should work out of the box with any IDE.

                            As David et. al. have indicated already this is technically possible by using:
                            (1) A common base class and locking
                            OR
                            (2) Annotating all classes with a special rule and using locking

                            The problem with this solution is its logically in reverse. I would
                            like the startup / teardown behaviour applied automatically rather
                            than me
                            having to decorate classes accordingly or use a specific base class.
                            Use of a special Runner is possible I guess but would break in a IDE
                            (which typically uses its own runner).

                            Essentially I guess this is a necessary start-up 'aspect' - which I
                            suspect is, in the main, the idea you're proposing by using a Java
                            Agent

                            Irrespective however, the discussion is most interesting.

                            --
                            Best Regards,

                            Brett Delle Grazie
                          Your message has been successfully submitted and would be delivered to recipients shortly.