Test Suites

[Stefan Bell]

Do you setup test suite for your tests? My company has ended up doing it by package. What I would like to setup would be one huge test suite that would perform like ant scripts (which we have setup but not many of our programmers have access too) where when they make a change that they think may cause problems elsewhere, the person could run this locally. Also, run one suite across projects? Possible or just dreaming.

[Pradeep bhatt]

What is the test suite BTW?

[Greg Ostravich]

I believe it's the collection of all tests.
It's what can be called from the GUI or the textual interface to run all your tests.
Here's an example of my test suite.
You'll see the main that calls the tests first, followed by the suite.
Note that all the individual unit tests are within each class of the suite (i.e. ClasExpMonthsTest.class contains all the individual unit tests for that class) and JUnit uses introspection to find all the tests.

  
package ADS_Upload;
import junit.framework.*;
/**
 * TestSuite that runs all the sample tests
 * Note: Can use 'fail()' to force failure
 */
public class AllTests 
{
public static void main (String[] args) {
  junit.textui.TestRunner.run (suite());// text version
  // junit.awtui.TestRunner.run ( suite() );// awt version
  // junit.swingui.TestRunner.run ( suite() ); // swing version
}
public static Test suite ( ) 
{
  TestSuite suite= new TestSuite("All JUnit Tests");
  suite.addTest(new TestSuite(ClasExpMonthsTest.class));
  suite.addTest(new TestSuite(StringToolsTest.class));
  suite.addTest(new TestSuite(FmtRulesTest.class));
  suite.addTest(new TestSuite(FieldDataTest.class));
  suite.addTest(new TestSuite(FieldMapTest.class));
  suite.addTest(new TestSuite(RecordMapTest.class));
  suite.addTest(new TestSuite(RecordDataTest.class));
  suite.addTest(new TestSuite(ParseRuleTest.class));
  suite.addTest(new TestSuite(ParseMapTest.class));
  return suite;
} // suite
} // AllTests

[Stefan Bell]

Originally posted by Greg Ostravich:
I believe it's the collection of all tests.
It's what can be called from the GUI or the textual interface to run all your tests.

Exactly, but have tried to combine tests from other projects. I think maybe having a testsuite that call all the project test suites. I am still trying to come up with the best design to be able to do it so if anybody has tried to do this please let me know. Thanks...

[Pradeep bhatt]

Do we use junit.textui.TestRunner.run (TestClass.Class) for testing all the test in the TestClass and junit.textui.TestRunner.run (suite()); for testing all the classes? :roll:

Thanx

[Lasse Koskela]

Originally posted by Pradeep Bhat:
Do we use junit.textui.TestRunner.run (TestClass.Class) for testing all the test in the TestClass and junit.textui.TestRunner.run (suite()); for testing all the classes?

Yes, the run(Class) method uses the Reflection API to pick up all JUnit test methods from the given class and executes those tests.

However, what the run(TestSuite) method executes depends on what the given TestSuite object returns from its static suite() method. A typical implementation for a TestSuite is to manually construct a list of all children of TestCase in the package (see Greg's example class). It is also a convention to name this kind of TestSuites as "AllTests" similarly to how you name all your TestCases as "_________Test".

[Jeanne Boyarsky]

We have a test suite for each project (that uses reflection to pick up all the tests). A project is a group of packages in Eclipse/WSAD. We also have a master test suite for the project as a whole (a project from a business point of view) that runs all the test suites.

[Pradeep bhatt]

Lasse and Jeanne.

[Lasse Koskela]

Originally posted by Jeanne Boyarsky:
We have a test suite for each project (that uses reflection to pick up all the tests).

May I ask how have you implemented this reflection-based test suite? Is it something you've written yourselves?

[Jeanne Boyarsky]

Lasse,
Well, I personally cut and pasted it from someone else at our company. But basically, it gets all the classes using the test collector for the junit api. It then filters by package name (so you don't run the tests in other projects or jar files if you don't want to.) It also checks to see if the class is abstract and then skips it.

[Lasse Koskela]

Originally posted by Jeanne Boyarsky:
basically, it gets all the classes using the test collector for the junit api.

Do you mean junit.runner.ClassPathTestCollector? Sounds cool.

[Jeanne Boyarsky]

Lasse,
That's the one! I like it because you don't have to remember to add classes to the "master" suite.

[Lasse Koskela]

Yeah, me too. Thanks for introducing me to this little gem

[lakshmi bvr]

It is a collection of all test cases,test data and test environment including all the modules for entire project, which is maintained by any sofware configuaration tool like VSS.

[Lasse Koskela]

Originally posted by lakshmi bvr:
It is a collection of all test cases,test data and test environment including all the modules for entire project, which is maintained by any sofware configuaration tool like VSS.

Again, it depends on the context. A TestSuite in association with JUnit is a collection of Test instances and nothing more.

[Jeff Langr]

Here's something I wrote for Agile Java. The design is driven by tests (not shown) and the fact that JUnit is a legacy mess. createClass is called twice, but you could optimize by caching the class if necessary.

package sis.testing;
 
import java.util.*;
import junit.runner.*;
import junit.framework.*;
import java.lang.reflect.*;
 
public class SuiteBuilder {
   public TestSuite suite() {
      TestSuite suite = new TestSuite();
      for (String className: gatherTestClassNames()) {
         Class klass = createClass(className);
         suite.addTestSuite(klass);
      }
      return suite;
   }
 
   List<String> gatherTestClassNames() {
      TestCollector collector = new ClassPathTestCollector() {
         public boolean isTestClass(String classFileName) {
            if (!super.isTestClass(classFileName))
               return false;
            String className = classNameFromFile(classFileName);
            Class klass = createClass(className);
            return
               TestCase.class.isAssignableFrom(klass) &&
               isConcrete(klass);
         }
      };
      return Collections.list(collector.collectTests());
   }
 
   private boolean isConcrete(Class klass) {
      if (klass.isInterface())
         return false;
      int modifiers = klass.getModifiers();
      return !Modifier.isAbstract(modifiers);
   }
 
   private Class createClass(String name) {
      try {
         return Class.forName(name);
      }
      catch (ClassNotFoundException e) {
         return null;
      }
   }
}

At a client site, Jeff Bay and I paired on a more sophisticated version that covered pattern-based package exclusions (e.g. ignore anything in *.performance) and that read a list of classes to be explicitly ignored. The problem with ignoring tests is just that--often they are forgotten completely!

I look forward to a rewrite of JUnit that takes advantage of the new annotation types in Java 1.5, a la NUnit. I modified JUnit to support an @Ignore tag and can show in the swingui a tab of all test methods marked with @Ignore.

-Jeff-
[ June 02, 2004: Message edited by: Jeff Langr ]

[Chris Mathews]

Originally posted by Jeff Langr:
I look forward to a rewrite of JUnit that takes advantage of the new annotation types in Java 1.5, a la NUnit. I modified JUnit to support an @Ignore tag and can show in the swingui a tab of all test methods marked with @Ignore.

Actually, Cedric Beust has already wrote one called TestNG and it is compatible with JUnit.
[ June 02, 2004: Message edited by: Chris Mathews ]

[Jeff Langr]

Excellent, thanks for posting the link!