• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Knute Snortum
  • Paul Clapham
  • Tim Cooke
Sheriffs:
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Ron McLeod
  • Piet Souris
  • Frits Walraven
Bartenders:
  • Ganesh Patekar
  • Tim Holloway
  • salvin francis

Testing the test code  RSS feed

 
Ranch Hand
Posts: 8934
Firefox Browser Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi author,

How do we make sure that that the test code we write is free from bugs? :roll:
 
author & internet detective
Posts: 39283
727
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Pradeep,
The real code and test code test each other. If the real code isn't working properly, the test code flags this by failing. If the test code isn't working, it won't test the real code accurately (for the most part.) This system works well if you code/test in small chunks.

It is up to the author to make sure you don't write assertTrue(true);. And a code coverage tool can make sure you are indeed testing the code.
 
Ranch Hand
Posts: 328
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You really hit the spot here, Pradeep. This topic has been bugging me for last 2 weeks.

Code which does the work, code that tests the code that does the work, code that tests the code that tests the code that does the work, etc.

And of course mock objects. They need to be tested as well. Do they mock real objects' behaviour close enough? How correctly do they do that?

How about the code which sets up environment for a test case? It could easily be larger and more complex than the code to be tested.

"Big fleas have little fleas
Upon their back to bite 'em
And little fleas have lesser fleas
And so ad infinitum."(C)

The real code and test code test each other.

They sure do. But how do you know which one of them fails, if none of them are working properly?

If the real code isn't working properly, the test code flags this by failing.

...only if the test code is working properly. What if it does not?

If the test code isn't working, it won't test the real code accurately (for the most part.)

That's right. So, how accurately "the real code and test code test each other" if both are not working properly?

This system works well if you code/test in small chunks.

...and not always well even then.
 
author
Posts: 87
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I can't make sure that you write defect-free software.

I can, however, tell you how I deliver low-defect software. (A handful of defects per year.)

1. Think of a behavior your software needs that it doesn't have.
2. Write a test that verifies the presence of that behavior.
3. Run the test and watch it fail.
4. Write the code to pass that test, and all the other tests you already have.
5. Look for ways to improve your code, mostly by removing duplication and making sure everything has a good name.
6. Return to step 1.

I just do this a few dozen times a day every day until the project is over.
 
Jeanne Boyarsky
author & internet detective
Posts: 39283
727
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dmitry,

And of course mock objects. They need to be tested as well. Do they mock real objects' behaviour close enough? How correctly do they do that?


Mock objects shouldn't have any behavior complex enough to test. They are usually 90% getters/setters. And if you use a framework like easymock, there isn't a mock objects class (just an interface), so you don't have to worry about it at all.

How about the code which sets up environment for a test case? It could easily be larger and more complex than the code to be tested.


Yes, but it's usually reusable. Once you do it once (and know it is working), you are done.

The real code and test code test each other. They sure do. But how do you know which one of them fails, if none of them are working properly?


If you do TDD: Add a test, implement code. If code doesn't seem right or can't pass, the test is wrong. Otherwise it is the code. Once you see the green bar, any future problems are with the code (unless you explicitly change the test.)

If the test code isn't working, it won't test the real code accurately (for the most part.)


This is only a problem if you make the exact same mistake in the real code and the test. (or code the test in a way that it doesn't do anything)
 
author and iconoclast
Posts: 24203
43
Chrome Eclipse IDE Mac OS X
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Dmitry Melnik:

They sure do. But how do you know which one of them fails, if none of them are working properly?



You don't. But if you have make an error in 10% of the lines of code you write, then if the errors are randomly distributed, then writing a unit test reduces the chance of there being an error in both the test and the production code to 1% -- a factor of 10 improvement. The impossibility of attaining perfection is a lousy excuse for not trying.

Regarding the test scaffolding being more complex than the production code: then make it simpler. One of the things that TDD does for you -- if you're doing it right -- is that it forces you to write code that's easy to test. If you write the code first, well, then it might be hard to test unless you think about how to test it (in which case, why not just write the test first?)
 
blacksmith
Posts: 1332
2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Dmitry Melnik:

If the real code isn't working properly, the test code flags this by failing.

...only if the test code is working properly. What if it does not?


I think that in practice, it's quite rare for the test code and the real code to fail in exactly the same way. I do think there's an argument here for having the test code and the real code written by different people, so they won't make the same algorithmic and design mistakes - but if one is aware of that problem, one can be careful to use different algorithms in the two. For example, I feel free to hard code the results of hand calculations in the test code.

Testing isn't a silver bullet, though. That's why other techniques for reducing bugs, such as code reviews or pairing, and commenting the code, are also important.
 
J. B. Rainsberger
author
Posts: 87
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

You really hit the spot here, Pradeep. This topic has been bugging me for last 2 weeks.



Maybe I can help you.

Code which does the work, code that tests the code that does the work, code that tests the code that tests the code that does the work, etc.



If we make the tests super-simple, then they're much less likely to be wrong. Of course, this is true of production code, too. Fortunately, there are ways to make the tests super-simple, and we're writing books about that and sharing our experiences.

I have never needed to test the tests, and I ship good software, so it can be done.

And of course mock objects. They need to be tested as well. Do they mock real objects' behaviour close enough? How correctly do they do that?



Programmer testing is not a substitute for thinking; rather, it is a way of codifying your thoughts in an executable manner. Programmer testing is like taking your ideas about how the code should work and giving them to the computer to verify for you, all day, every day.

That said, we use Learning Tests to discover how expensive external resources work, then we use that information to program our mock objects. We run the Learning Tests less frequently, and if they ever fail, then we know that our third-party software has changed and we need to change our mocks. In practice, this is rarely a problem.

We all know that we should write acceptance tests for the third-party software we use, so we can detect those parts of it that change that we rely on. Few of us do. (I never did.) Learning Tests provide some of that value.

How about the code which sets up environment for a test case? It could easily be larger and more complex than the code to be tested.



It could, but it doesn't have to be; and when it is, there is usually an equivalent, simpler design that allows for simpler fixtures.

They sure do. But how do you know which one of them fails, if none of them are working properly?



The tests are really simple, so if you're not sure whether a test it write, throw it away and write it again.

If the real code isn't working properly, the test code flags this by failing.

...only if the test code is working properly. What if it does not?



See above.

This system works well if you code/test in small chunks.

...and not always well even then.



Programmer testing is not a magic wand; it is a tool for increasing confidence in the correctness of code and for creating an environment in which change is less expensive. Is it perfect? Of course not. What is?

Just because it ain't perfect don't make it useless.
 
Jeanne Boyarsky
author & internet detective
Posts: 39283
727
Eclipse IDE Java VI Editor
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Warren Dew:
I do think there's an argument here for having the test code and the real code written by different people, so they won't make the same algorithmic and design mistakes


But then there is the disadvantage of the test code writer to not have thought of all the things the real code writer thought of. Writing the real code forces you to think of the detailed conditions.
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Jeanne Boyarsky:
But then there is the disadvantage of the test code writer to not have thought of all the things the real code writer thought of. Writing the real code forces you to think of the detailed conditions.

...which leads us to seriously consider pairing
 
Always! Wait. Never. Shut up. Look at this tiny ad.
ScroogeXHTML - small and flexible RTF to HTML converter library
https://coderanch.com/t/710903/ScroogeXHTML-RTF-HTML-XHTML-converter
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!