• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

BDD in Action: Question about different frameworks

 
Burk Hufnagel
Ranch Hand
Posts: 814
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
John,
You've mentioned several different BDD frameworks for Java developers, and I was wondering is you could say a few words about them. I've used Spock, but have no experience with JBehave or Thucydides, and I was hoping you could compare them briefly - or at least tell me if there's one you prefer and why.

I also noticed you haven't mentioned easyB and was wondering if it's because you don't have enough experience to talk about it, or if you just don't recommend it.

Thanks,
Burk
 
John Smart
Author
Ranch Hand
Posts: 43
5
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
OK, here goes. There are two levels of BDD as you know: one for high-level requirements and automated acceptance criteria, and one more focused on coding. While you could use the same tooling for both, the focus of tools tends to be better suited for one of these two approaches. Here a few of the main ones:

1) High-level BDD tools: Usually separate the specifications (e.g. .story or .feature files for JBehave, Cucumber etc, or HTML pages for Concordion):
- JBehave is the evolution of Dan North's original BDD tool: it uses the classic BDD "Given..When..Then" notation. Specifications are in .story files, and the step implementations are in Java classes. This approach work very well as a collaboration and communication tool to define the specs, but would usually be too much overhead for low-level coding.
- Cucumber is a widely-used tool from the Ruby world, which now has a JVM version that works in many languages. It uses a very similar "Given..When..Then" notation. Ports of Cucumber exist for .NET (SpecFlow), Python (Behave and Lettuce), PHP (Behat) and other languages. These tools use the same approach as JBehave, and the same comments apply.
- In Concordion you write the specifications in the form of tagged HTML, and implement the actual tests in Java. I've mentioned Concordion in another post: I find the HTML page side of Concordion a bit hard to scale for larger projects, when you need to be able to slice and dice your requirements (e.g. have views by feature, by release, by iteration, etc...)
- Fitnesse is an older tool where you write specifications in a Wiki and implement the steps in Java. Haven't used it much myself, but I find places that use it tend to end up expressing everything in tabular format, which I find not appropriate for all sorts of requirements.
- Easyb is a Groovy-based BDD DSL where the "Given-When-Then" specs are embedded in a Groovy script alongside the implementation. This has advantages and disadvantages. Easyb can be used for both high-level and coding level BDD, but is harder for non-developers to work with.

Thucydides (http://www.thucydides.info) is a bit different: it is a reporting tool that helps you organize your automated acceptance tests, or just helps generate better living documentation. It's goal is to work with all of the main BDD tools: currently it has deep integration and reporting capabilities with JBehave, JUnit and easyb, and can generate living documentation for tools like Cucumber, Specflow or Behave.

2) TDD-style (coding-level) BDD tools are unit-testing tools with a BDD flavor: they are more focused on developer productivity and technical documentation than communicating with the business. Some use a "given-when-then" structures and support tabular data, whereas others use a more primitive style where you say things like 'it "should do this".
- Spock: very nice light-weight BDD unit testing in Groovy for Groovy or Java code
- Spec2: code-level BDD in Scala
- RSpec: low-level BDD in Ruby, using a simpler structure
- Jasmine: BDD for Javascript (more like rspec than spock)
- Kiwi: BDD for iOS (uses an rspec style)

There are a few native Java BDD tools (e.g. JDave), but I find they really struggle with the limits of the Java syntax, and contain a lot of boiler-plate code. Note you can also do BDD-style tests in JUnit or NUnit just by using expressive naming conventions and structuring your tests appropriately. The other tools just provide stronger built-in support for this.

Personally at the moment I use JBehave and Thucydides at the requirements level and Spock at the coding level for JVM projects, Specflow and Thucydides for .Net, Jasmine (and Karma) when I'm working with AngularJS. I'd be happy to use Cucumber as well but I haven't got round to getting the Thucydides/Cucumber integration done yet.
 
Burk Hufnagel
Ranch Hand
Posts: 814
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Wow. Thanks for the info dump on frameworks. Way more than I had expected.

Sounds like I need to get some experience with Thucydides.

Wondering about your statement that the Given-When-Then format is too much overhead for low level coding. For me, it makes the context very clear. I don't get how the 'it should' version does that. Can you help me understand?

Thanks again, John. This has been a very illuminating week.

With appreciation,
Burk
 
John Smart
Author
Ranch Hand
Posts: 43
5
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I didn't mean that the Given-When-Then is overhead for unit tests - in fact it works really nicely with Spock. When you use tools like JBehave or Cucumber, the scenarios (Given-When-Thens) are in plain text files, and the test implementations is in source code form. Putting the requirements in text (.story or .feature) files is more overhead, as you need to keep the specification texts and the implementation code in sync, but the overhead is justified for BDD acceptance criteria because it makes them much easier to use as a communication tool. For unit tests, something like Spock is better.
 
Burk Hufnagel
Ranch Hand
Posts: 814
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Ah. That makes a lot more sense.

It seems odd though to have the scenarios in a separate plain text file - I'd be afraid that the scenarios and the implementations would get out of synch. Is there something in the framework that helps prevent that problem? Or is it up to the developers to make sure everything stays synched up?
 
John Smart
Author
Ranch Hand
Posts: 43
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
If they get out of sync, it's easy to spot: the tests don't run :-). For BAs and testers, the separation keeps the requirements clean and isolated from the implementation, which makes them easier to work with collaboratively. But you do have to be well organized with larger projects to avoid stepping on everyones toes. I'll include some techniques to do this in JBehave in the book somewhere.
 
Burk Hufnagel
Ranch Hand
Posts: 814
3
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
John Smart wrote:If they get out of sync, it's easy to spot: the tests don't run :-).

I don't understand. If the scenarios are in a separate, plain text file, then as long as the tests match the system's behavior shouldn't they pass? How does the content of the scenarios file impact the tests?
 
Leo Gallucci
Greenhorn
Posts: 1
Chrome Ruby Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
John Smart wrote:I'd be happy to use Cucumber as well but I haven't got round to getting the Thucydides/Cucumber integration done yet.


Hi John, did you found a way to connect Cucumber with Thucydides in an language agnostic way? or more specifically with Ruby Cucumber bindings?
I opened an issue https://github.com/serenity-bdd/serenity-cucumber/issues/11 in Thucydides repo (now renamed to Serenity) to track this down.
 
Stephane Roussel
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey all,

This thread seems to be dead... but a question is still in the air. John, you talked about using Thucydides (aka Serenity) with tools like Specflow in C#. I'm trying to make this working right now and I do not see how Thucydides can be setup to be used with SpecFlow. Could you please elaborate on how have you make it work ? Does anyone has already figured out how to properly setting up Thucydides with Specflow ?

Thank you
 
Junilu Lacar
Bartender
Posts: 7607
54
Android Eclipse IDE IntelliJ IDE Java Linux Mac Scala Spring Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Stephane,

Welcome to the Ranch!

You're right about the thread being dead and it's been a while since John has posted so you probably won't get an answer from him anytime soon. Sorry I can't help you on this one though, I'm not familiar with any of the technologies you mentioned.
 
Stephane Roussel
Greenhorn
Posts: 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi Junilu,

Thank you for your reply. I thought someone else might have the answer.

Anyway,
Thank you
 
Junilu Lacar
Bartender
Posts: 7607
54
Android Eclipse IDE IntelliJ IDE Java Linux Mac Scala Spring Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
No problem Stephane, but as you might have noticed, this is a pretty quiet forum with very little traffic other than those occasions when we're giving away free stuff. You never know though, someone might come along who has some helpful information. Just don't hold your breath on it.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic