• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

randomness in test data

 
Ranch Hand
Posts: 308
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
hi,

i am having a unit test, which uses a list of objects as test data.

i am testing that this list forces a method to be called a certain amount of times depending on the list's size and objects type inside the list. i am using a mocks to isolate my class.

because i want to have random objects in this list i am calling a build method which generates a certain number of fixed objects whose type is depending on a int return value of a random function.

now my question:
is it alright if some randomness is included in my test data or should it be avoided at any time?

pros:
-i use random (in my case realistic) test data.
cons:
-a failing test is very difficult (or even impossible) to reproduce.
-unit tests shouldn't be dynamic, running them once or thousand times must show the same results, which is not 100% sure if test data can change during different test runs

of course i would use only random data in very rare cicumstances! but i thought it my case useful, because it is very time consuming and annoying to generate a good mixed and big list in my test suite.

maybe the solution to my problem could be a little code generater and copy/paste it into my tests...

thanks for your thoughts.
 
Sheriff
Posts: 7001
6
Eclipse IDE Python C++ Debian Java Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
maybe the solution to my problem could be a little code generater and copy/paste it into my tests...

That's probably what I would do in this case.

Before that, though, I would investigate to see if I could repeatably "seed" the random number generator to produce the same pseudo-random sequence each time. If necessary, you could then fairly easily repeat the test for different seeds.
 
author & internet detective
Posts: 41878
909
Eclipse IDE VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Manuel,
For unit tests, I prefer to use the same data each time as it results in repeatable tests. For integration tests, I don't mind some randomness.
 
manuel aldana
Ranch Hand
Posts: 308
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
thanks, i got your points.
 
Ranch Hand
Posts: 38
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
manuel aldana,

My suggestion is when you have no choice but use randomness in your unit testing, you can write extra code to log the list information when a failure occurs, which may make it easy to reproduce the failure.
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic