JUnit4 Threaded Tests

In an effort to reduce the complexity and improve efficiency of testing, the FS team (Financial Services) is converting all of our unit and integration tests to JUnit4. This conversion is necessary to take advantage of the new features in JUnit4, specifically the Runners. JUnit4 provides the ability to specify a customized org.junit.runner.Runner for each Test through the use of an Annotation.

FSQueuedTestClassRunner

We in FS have always been a firm believer in multi-threading our tests to take advantage of our multi-core machines. To accomplish this we had built a customized ActiveTestSuite that detected the junit tests from a list of eclipse projects. this worked great as long as you knew ahead of time the proper way to group the tests to properly balance the load. Too many threads, the JVM will start to thrash, too little threads or unbalanced threads are inefficient.

Using JUnit4 we created a custom test Runner called “FSQueuedTestClassRunner” that removes the need to assign the test class to a particular thread. This runner interacts with our “FSRunnerQueue” which queues the test for a later run. The FSRunnerQueue takes advantage of the java.util.concurrent package and starts a number of executors based on your number of processors. These executors pull tests off of the queue as they become available. To hookup the Runner to a JUnit test is a straightforward annotation.

@RunWith(FSQueuedTestClassRunner.class)
public class StringTokenHelperTest {
...
}

This annotation works for all of our integration tests also.

Converting all the tests

Using some sed scripts we converted >35 eclipse projects worth of tests (both unit and integration ~5000) into the new JUnit4 format with annotations. We basically added annotations to the classes and included the necessary imports.

@RunWith(FSQueuedTestClassRunner.class)
public class StringTokenHelperTest {
...
@Before
public void setUp() throws Exception {
...
}
@After
public void tearDown() throws Exception {
...
}
@Test
public void testApplyTokens() throws Exception {
...
}
}

With some simple import re-organizations and some small compile fixes, in the matter of a day had all the tests converted and running using the new Runner. Now using eclipse we get the benefit of multi-threaded tests with all test runs.

The new FS Test Standard

To run all the tests across multiple eclipse projects we created a custom Runner “ClasspathSuiteEnforceFSConventions”. This Runner has a dual purpose

  • Discover all the eclipse projects on the classpath and discover all the JUnit tests that exist
  • Ensure that each test that we find follows our FS test conventions. Essentially prevent any more JUnit3 style tests.
    • @RunWith(FSQueuedTestClassRunner.class) annotation on class
    • @Test annotation on all public void test…() methods
    • @Before on all public void setUp() methods
    • @After on all public void tearDown() methods

Each of our applications has a Build project in eclipse that has project references to all of the required eclipse projects. This is where we create or test runner class. Here is an example of such a class. This one runs all unit tests avoiding all of our integration tests or tests in out “.webtest.” package.

@RunWith(ClasspathSuiteEnforceFSConventions.class)
@ClassnameFilters({"!.*webtest\\..*"})
public class AllProjectsTestSuite  {
}

Performance improvements

With one of our applications we experienced a reduction from 4.5 minutes to 95 seconds for all the unit tests across multiple projects. This savings for each test run multiplied by each developer across a week could add up to a substantial savings in time.

Kevin

It's only fair to share...
Share on FacebookGoogle+Tweet about this on TwitterShare on LinkedIn

Leave a Reply