Hi. I have a question about integration tests.
Our codebase has a lot (really, a lot)
unit tests, that mock dependencies, perform negative tests (like testing how a service will behave on invalid input or an exception being thrown somewhere deep and so on), test various permutations of input - we take good care to tests as much as we can. The result is obvious - we are very confident about our units that they work by themselves, adhere to their contracts and so on - the little classes that will eventually be assembled into modules / application. As mentioned, we use mocks wherever possible to make sure we test a single unit at a time.
Now, time has come for integration tests. Our application has a service module, that uses persistence module and many others. It has also a facade module that is coded as EJBs that use the services internally (injected via CDI), to have JTA support, security, serialization guarantees... The whole application is a
Java EE 6 application, we are using GlassFish 3.1 (snapshots) for the embedded
EJB support - this allows us to have tests that look up EJB, invoke methods on them, have dependency injection performed by CDI, have JTA transactions where we need them and so on. We also use a real database (PostgreSQL, the target for the initial release), and we use DbUnit to fill it with data whenever it is important (pretty much everywhere). The whole infrastructure is very complex but very powerful, we are using our own framework build upon
TestNG, we can test whatever we like, and we have dependency injection into test instances as well.
The question is - how much to test in integration tests? For example take ServiceA that was tested for invalid input, for underlying storage exceptions being thrown, for various calls with valid (but differing) input so that we know that we handle what is necessary - this was all done in unit tests. Do we have to repeat all the tests in 'kind of real' environment now (kind of as it is not real deployment but EJB embedded, but this is as close as we can get), or is it enough just to include a few test that test 'valid paths' in the application, and not test everything again? As far as I know, we need integration tests to see if the module assembly is correct (no unsatisfied dependencies), that the parts can work together and so on, but not necessarily test every single permutation of input (valid and invalid) for each service - this would basically duplicate unit tests for this specific behaviour.
What do you, experienced test specialists, have to say?
Raf