Is it a bad practice for all test classes to extend a particular class ?

[Tom Joe]

So far, I have only seen custom Java frameworks for UI or API testing where all test classes extend a particular class which we can call "base class". This base class usually runs some setup & cleanup code before each test class is run. In some cases, the base class is also used to store test data like usernames, passwords, authentication tokens etc. and provide utility methods.

Are there any serious problems in this approach ? If yes, then are there any good alternatives to this approach ?

[Jeanne Boyarsky]

Tom,
It's not a problem. I see more commonly where a bunch of classes extend the same superclass, but not necessarily all of them.

For example, in this forum software we have an integration test superclass for all integration tests. We also have a mock action superclass for the front end action unit tets.

[Junilu Lacar]

Standard consultant's reply: It depends.

Most people dont know how to design for inheritance and they make the base class the kitchen sink for utility and setup/config concerns. This is a horrific antipattern that results in entire hierarchies of untestable code. The smell that points to this is that you can't unit test anything but have to do end-to-end testing for every little change you want to make in the code base, if you're doing testing at all.

If I had a nickel for every time I've seen this anti pattern, well, I'd have maybe a few bucks. That's still a lot though considering I've only been a developer for thirty years.

On the other hand, I'd probably have a little less than a few bucks for all the times I've seen base classes designed for inheritance well. That kind of code is found in many popular and successful frameworks.

If you're designing your own, prefer composition and delegation to inheritance.

[daniel jaimini]

Inheriting a common base class in all subclass practice is followed by many high-end developers, so it won't be problematic at all

[Junilu Lacar]

daniel jaimini wrote:Inheriting a common base class in all subclass practice is followed by many high-end developers, so it won't be problematic at all

Maybe it's just me but I don't share the same kind of optimism. I've been around for a while and I've seen the intersection of the Pareto Principle and the Dunning-Kruger Effect.

At most, only 20% of the developer population are "high-end" developers. Some would peg it at less than that even. My personal experience tells me it is in the single digits. If we are to believe Uncle Bob Martin's claim that we are a perpetually inexperienced profession, then at least 50% of developers have less than 5 years experience and don't know how to design for inheritance properly.

The Dunning-Kruger effect tells us that the 80% who are not actually high-end developers, think they are.

[Junilu Lacar]

I may seem overly pessimistic with my responses but I'm speaking from first-hand experience.

My last client (my day job is consulting as an Agile technical coach) had such a base test class as I've alluded to in my first response. That base test class had methods that took care of setting up database connections, message queue connections, and other infrastructure concerns. Then they instituted a policy that required developers to extend this class for all their tests, including unit tests.

When I tried to coach them on TDD (Test-Driven Development), we had to wait literally for 5 minutes for each compile/build/test to finish because the tests were all end-to-end tests and did all kinds of infrastructure configuration and setup. You can't do TDD if your red-green-refactor cycle time is at least 20 minutes. And that is just one cycle.

The whole test suite was like that. They had builds that ran forever. Ok, they ran overnight, so developers would leave for the day and come back next morning to find that the tests failed. The tests were flaky, failing inconsistently because of race conditions the tests themselves unintentionally created because of that infernal base test class.

So you see, I'm not just Chicken Little running around telling you that the sky is falling. If you are not careful and you don't entirely know what you're doing (why would you ask the question you did in the first place?), then it is highly likely that the weight of a poorly-designed base test class will eventually make your sky fall down on you.

[Junilu Lacar]

I also have first-hand experience with the different skill levels developers have. I once had to fill three developer positions on my team and it took me going through 60 people to find the right ones. Three out of sixty is 5% and those were just the ones I could easily coach to become really good developers. I see about the same ratio at most places I've been and I've been at a lot of places.

[Tim Holloway]

It's often very useful to create a test base class (or classes) and abstract the actual test classes from them. For example, rather than hard-coding a service locator into each separate test, I might make (actually have made) a base test class that provides an abstract internal method for the tests to invoke.

I think what Junilu may have been experiencing is a common nightmare where people try to create the entire Universe each and every time every test runs. This is a common peril in particular of dbUnit testing where you need to construct and destroy database tables as part of the test to ensure that the exact same data is being used on each run of the test.

So basically, if you're going to use common code - no matter where you put it - make sure you only invoke code that the particular test actually needs (and try to minimize that).

[Junilu Lacar]

There are good reasons for following the advice to prefer composition over inheritance. At the same time, inheritance can give you a lot of benefits if you use it judiciously. All I'm saying is be careful when you use inheritance for test classes. If you are motivated to use inheritance because there is a legitimate "TestSubclass is a specialization of TestBaseClass" relationship, then go for it. Otherwise, if your only reason for using inheritance is to provide commonly used infrastructure or configuration, then you might want to reconsider. As a matter of fact, this also applies to production code.

[Stephan van Hulst]

I avoid inheritance, but I found it useful for test cases if your case tests a class that is also a sublass of another class that has a test case. Your test case then inherits the tests for the superclass and it will tell you that your class follows the Liskov Substitution Principle for a minimal amount of work.

[Junilu Lacar]

Stephan cites one valid use case that I have seen is a valid use of inheritance for test cases. This also applies to contract testing, where you have an interface and you want to test various implementations of it. Again, LSP is the motivation. How many people do you know who can properly explain what that principle even means, much less properly design classes that adhere to it? My experience tells me there are more people who can't than can which is why I would advise extreme caution and an overabundance of thoughtful consideration before proceeding to go against "prefer composition over inheritance."

[Tom Joe]

Junilu Lacar wrote:I may seem overly pessimistic with my responses but I'm speaking from first-hand experience.

My last client (my day job is consulting as an Agile technical coach) had such a base test class as I've alluded to in my first response. That base test class had methods that took care of setting up database connections, message queue connections, and other infrastructure concerns. Then they instituted a policy that required developers to extend this class for all their tests, including unit tests.

When I tried to coach them on TDD (Test-Driven Development), we had to wait literally for 5 minutes for each compile/build/test to finish because the tests were all end-to-end tests and did all kinds of infrastructure configuration and setup. You can't do TDD if your red-green-refactor cycle time is at least 20 minutes. And that is just one cycle.

The whole test suite was like that. They had builds that ran forever. Ok, they ran overnight, so developers would leave for the day and come back next morning to find that the tests failed. The tests were flaky, failing inconsistently because of race conditions the tests themselves unintentionally created because of that infernal base test class.

So you see, I'm not just Chicken Little running around telling you that the sky is falling. If you are not careful and you don't entirely know what you're doing (why would you ask the question you did in the first place?), then it is highly likely that the weight of a poorly-designed base test class will eventually make your sky fall down on you.

Seems like a problem with the company policy rather than with using inheritance in tests. Could they have solved the problem by simply having a different base class for unit tests ? If yes, then would inheritance still have been a problem for them ?

[Tom Joe]

Junilu Lacar wrote:Stephan cites one valid use case that I have seen is a valid use of inheritance for test cases. This also applies to contract testing, where you have an interface and you want to test various implementations of it. Again, LSP is the motivation. How many people do you know who can properly explain what that principle even means, much less properly design classes that adhere to it? My experience tells me there are more people who can't than can which is why I would advise extreme caution and an overabundance of thoughtful consideration before proceeding to go against "prefer composition over inheritance."

For test classes, could you please share an example of how composition could be used instead of inheritance and still provide the same functionality as inheritance ?

[Stephan van Hulst]

This works for ALL classes, not just test classes:

final class Derived extends Base {

  @Override
  public void doSomething() {
    super.doSomething();
    doSomethingElse();
  }

  ...

}

final class Composed implements SomeInterface {

  private final Base component;

  Composed(Base component) {
    if (component == null)
      throw new IllegalArgumentException();

    this.component = component;
  }
  
  @Override
  public void doSomething() {
    component.doSomething();
    doSomethingElse();
  }

  ...

}

[Junilu Lacar]

Tom Joe wrote:Seems like a problem with the company policy rather than with using inheritance in tests. Could they have solved the problem by simply having a different base class for unit tests ? If yes, then would inheritance still have been a problem for them ?

No, the problem was with engineering and design. Inheritance was used purely as a convenience, so they could set up common infrastructure concerns like connections to databases and message queues. It had nothing to do with one set of tests being common to specializations of a class. Tests should be given the same care and feeding that production code gets. This includes any kind of design considerations. I have no objection with using base test classes, as long as there is a valid reason to model a Generalization-Specialization relationship in the tests that reflect the same kind of relationships in the production classes they apply to.  

However, I do have an objection to using inheritance as a convenience mechanism. I think that using it to deal with common infrastructure concerns is a poor design choice. The perceived benefit you gain from "ease of use and configuration" is quickly offset in many cases by the loss of testability at a unit level. In the cases that I have seen, the tests are often designed and implemented poorly and they become tied to specific infrastructure concerns. This results in the loss of flexibility that allows you to mock or stub out many of these kind of dependencies that should be abstracted away in tests.

Once you pull actual infrastructure dependencies into tests, they become slower. Developers tend not to run slower tests often. This starts a vicious cycle and developers often don't recognize it. They often become inured of the pain it's causing them, just as the team I was coaching had become.  It often takes someone from outside the team to tell them that it's too painful, that they can do better. At that point, however, the code rot is too extensive and mitigating it is more effort than just branching off and starting on a different path. Where policy might come into play is when people see this "deviation" and say "No, you can't do that because we need to be consistent." Seems like many engineering leaders put more stock in consistency than speed and quality. Seems like they'd rather stick with a painful way of doing things that "for the most part" works rather than try something new but different that could make the pain go away because "No, that would be inconsistent."  

In other words, they'd rather do things poorly but consistently rather than try to do some things better and live with some inconsistency for a while.

[Bill Stackhouse]

I have written a JUnit framework and it has helped to standardize my tests. I use Eclipse as my IDE so I have a template to add prefix lines of code to each test class.

@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class LevelFilterPanelTest {
    @ClassRule
    public static RuleGeneralClass ruleClass = new RuleGeneralClass(LevelFilterPanelTest.class);
    @Rule
    public RuleGeneralTest         ruleTest  = new RuleGeneralTest(ruleClass, this);

ruleClass logs the start/end of the class and creates a directory using the class name that lasts the life of the test class. Annotations allow overriding the location (tmp, desktop, etc.). If a collection of test classes need special setup/teardown, it is easy to extends either RuleGeneralClass or RuleGeneralTest, or use before/after annotated methods.

ruleTest logs the start of each test and creates a directory within the ruleClass direcotry using the test name lasting the life of one test. If a test fails it logs the failure with class:method:line number of the assertion failure. The (QA) in the listing below indicates it is using QA preference files.

2018-09-29 14:18:01 TEST   -- Starting LevelFilterPanelTest (QA)  
2018-09-29 14:18:02 TEST   ---- Start test acceptLogRecord()  
2018-09-29 14:18:03 TEST   ---- Start test acceptOwnerNotSelected()  
2018-09-29 14:18:03 TEST   ---- Start test acceptOwnerSelected()  
2018-09-29 14:18:03 TEST   ---- FAILURE Test:acceptOwnerSelected, Error:AssertionError(), at LevelFilterPanelTest:acceptOwnerSelected:56  
2018-09-29 14:18:03 TEST   -- Ending LevelFilterPanelTest duration 01.317 tests 3 expected 0 ignored 0

I have a test utility class so I can call private methods, set private fields, call private constructors, reset annotated static variables, select production/test/qa preference files, etc. This is all part of a structured logging system with a log viewer with filters (time, application, level, caller of log system, etc.) so tests can write out plenty of detail without cluttering the console output. It's not something I can share, but I wanted to provide some ideas of things a test framework should help with.