Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Running test cases in parallel  RSS feed

 
Ryan Kostrzewa
Ranch Hand
Posts: 31
1
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The company I work at is big on ATDD, and my team currently uses Gherkin and Cucumber as the primary testing framework. For the most part, it works great, and we are able to everything we need to do. Generally, the testing we are doing consists of submitting an application, waiting for it to approve (~12 seconds), getting JSON responses from a URL, and then parsing values out of that JSON and ensuring they match the expected values, in that order.

Cucumber and our testing framework has been great for this functionally, but for large (100+ test case) implementations, this takes a LONG time, and I'm wondering if there is a framework that would allow us to run test cases in parallel instead. There's no reason to run these test cases sequentially; the apps could all be dropped simultaneously, and the infrastructure behind the URLs is robust enough to be hit more than once every 30 seconds (I think... :P). I'm kind of green at testing and automations, do ya'll have any suggestions? Much appreciated!
 
Jeanne Boyarsky
author & internet detective
Sheriff
Posts: 37230
519
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ryan,
Yes. Running this type of tests in parallel saves a lot of time. The exact mechanism varies. If you are using a build tool like Maven, you might be able to use it. Otherwise, take a look at this library for running Cucumber tests in parallel.
 
Ryan Kostrzewa
Ranch Hand
Posts: 31
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey Jeanne, thanks for the response! I am using Maven - I've actually seen the plugin before, but I'm hoping to get even more granular. Running features in parallel has already cut down on a lot of time, but since the examples within a Scenario Outline still run sequentially, I was hoping to run the examples in parallel. Have you heard of anything with that kind of functionality?

-Ryan
 
Jeanne Boyarsky
author & internet detective
Sheriff
Posts: 37230
519
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No. Worst case you can have the same scenario outline twice with different examples. That would let you have two parallel streams.
 
Junilu Lacar
Sheriff
Posts: 11139
160
Android Debian Eclipse IDE IntelliJ IDE Java Linux Mac Spring Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I've never had to do this but I imagine if I did, I'd just create any number of profiles in the Maven, then each profile can include a subset of the integration tests to run. In my CI system, I'd define as many jobs as I have integration test profiles. I'd basically let my CI system take care of starting up and managing multiple processes that are each running a subset of the integration tests.  I hope that makes sense. Again, the disclaimer here is that I haven't actually tried this approach; this is just an idea and one that I'd probably explore first if I were faced with this kind of need.
 
Junilu Lacar
Sheriff
Posts: 11139
160
Android Debian Eclipse IDE IntelliJ IDE Java Linux Mac Spring Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The other nice thing about letting a CI system like Jenkins manage the multiple processes to run the integration tests is that each job can be assigned to a different slave node. That will help you spread the work around to different hosts and not have to load up any one host with too many active processes that are all  vying for shared system resources.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!