The company I work at is big on ATDD, and my team currently uses Gherkin and Cucumber as the primary testing framework. For the most part, it works great, and we are able to everything we need to do. Generally, the testing we are doing consists of submitting an application, waiting for it to approve (~12 seconds), getting JSON responses from a URL, and then parsing values out of that JSON and ensuring they match the expected values, in that order.
Cucumber and our testing framework has been great for this functionally, but for large (100+ test case) implementations, this takes a LONG time, and I'm wondering if there is a framework that would allow us to run test cases in parallel instead. There's no reason to run these test cases sequentially; the apps could all be dropped simultaneously, and the infrastructure behind the URLs is robust enough to be hit more than once every 30 seconds (I think... :P). I'm kind of green at testing and automations, do ya'll have any suggestions? Much appreciated!
Hey Jeanne, thanks for the response! I am using Maven - I've actually seen the plugin before, but I'm hoping to get even more granular. Running features in parallel has already cut down on a lot of time, but since the examples within a Scenario Outline still run sequentially, I was hoping to run the examples in parallel. Have you heard of anything with that kind of functionality?
I've never had to do this but I imagine if I did, I'd just create any number of profiles in the Maven, then each profile can include a subset of the integration tests to run. In my CI system, I'd define as many jobs as I have integration test profiles. I'd basically let my CI system take care of starting up and managing multiple processes that are each running a subset of the integration tests. I hope that makes sense. Again, the disclaimer here is that I haven't actually tried this approach; this is just an idea and one that I'd probably explore first if I were faced with this kind of need.
The other nice thing about letting a CI system like Jenkins manage the multiple processes to run the integration tests is that each job can be assigned to a different slave node. That will help you spread the work around to different hosts and not have to load up any one host with too many active processes that are all vying for shared system resources.