Win a copy of Head First Android this week in the Android forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Tim Cooke
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
Sheriffs:
  • Jeanne Boyarsky
  • Rob Spoor
  • Bear Bibeault
Saloon Keepers:
  • Jesse Silverman
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
Bartenders:
  • Piet Souris
  • Al Hobbs
  • salvin francis

Why automated tests (web apps/black box)

 
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hello people,
I want to test the functionality of my web app (actually a mobile app). I read some articles about HTTPUnit, and how it can be used instead of a browser to send ur requests from ur program, and checks the replies (it even comes with helpful functionality like parsing HTML tables and so on).
Now, I assume automated tests are supposed to be helping us developers from making repetetive long tasks, so instead of sitting and typing on the brwoser the input and manually checking the output, let the computer do it.
However (My question) how can I chose the Input/output pairs that will make sure most of my webb app is functionning well, just check all "the pages" that the cleint might request? The application is database driven, and there are endless no of states that it could be in, so to make many input /output sequence, I might just make an other program that is getting the data from t�he database, and checking it aainst the reply returned (and parsed) from the HTTP request via my HTTPUnit program??? Now how do u make sure this new program is well functionning? If u r not supposed to make sucha program that is able to generate many input /output pairs, and u find them manually (BTW what is the approach in here) then why would u need to make a test program at all, u can just make them manually (it is shorter for u to sit and type the input and expect the output, then to make a program that parses ur own data format and checks the output???

Please help, what did I miss? writting automated functional tests with a big number of pairs (input and output) seems to be an extra overhead for me? (apart from "documenting" or haveing a prove that the application has been tested )
Thank you very much :roll:
 
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Surely even the smallest application has a huge number of different combinations to test -- you just need to select the most important cases to start with.

Even though applications have vast numbers of possible combinations for input/output, there is usually a way to classify the inputs and outputs into a smaller number of "input classes", so to speak. E.g. "positive non-zero integers" instead of {1, 2, 3, 4, 5, ...}. Once you think in terms of these classes, you'll have a lot less to test. From the "positive non-zero integers" class, for example, you might pick values {1, 2, 100000000, Integer.MAX_VALUE - 1, Integer.MAX_VALUE} for your tests. If those work ok, the others in that class should work as well.
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Regarding your concerns about the output being dependent on what's in the database... Yes, it's a common approach to test this kind of output by verifying the output against what you read from the database directly. dbUnit is a helpful tool in this "domain". Also, you might consider populating the database through the same UI you're testing and then checking the output against what you just put in.
 
Tonny Tssagovic
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thank you for your help Lasse
All the others input is very much welcomed as well, I think having a longer discussion /more points of view will help me (and others hopefully) to understand these issues

But what does this automated testing bring that hand testing would not (it seems for me that checking the reply with code would take at least as long as checking it by just looking at it (yeah maybe some hidden/space charecters or some)
Thank you in advance!
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Tonny Tssagovic:
But what does this automated testing bring that hand testing would not (it seems for me that checking the reply with code would take at least as long as checking it by just looking at it)


What happens when your application's features grow from 100 to 200? Do you hire twice as much testers? Do the existing testers become twice as fast? The point I'm trying to make with this dumbass question is that automated tests are automated. "Automated" means that the tests actually get executed and you'll have a safety net for regression.

Automated testing can't replace "visual testing" completely in projects where a GUI is involved. The automated tests should focus on the semantics while the visual testing should focus on aesthetics, user experience and visual guidelines, if applicable.
 
Tonny Tssagovic
Ranch Hand
Posts: 226
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for the reply
I think I did not make myself clear

Originally posted by Lasse Koskela:

The point I'm trying to make with this dumbass question is that automated tests are automated.



Yeah, but what I mean is that if we have 100 different options to test (with all different kind of replies) then u still need to make 100 tests, and 100 different "parsers and checkers" of your reply. and u still need to double the number of testers, as the "automated" test is not much of automated, it can check if 10000 pages conform to the same schema/dtd/format, but can not check if 100 different pages are all ok (conform to the protocol).


Automated testing can't replace "visual testing" completely in projects where a GUI is involved. .


Yeah, actually I am just checking the data, and it is no HTML reply but our own format to be parsed and presented by our client app. Actually I tested the client application and the final GUI looks fine, but I can not make sure my client application is actually going through all the possible situations. BTW this actually raises a question, in case I am responsible of the client, I can make sure that my client is sending some parameters and that they are in the right format. My server application is not checking these, and thus I should not make tests when these parameters are passed in the wrong format?

So as far as I understand, there is not much of automated testing in here (well it always takes more time to check the reply with ur eyes then with ur code) and in this case HTTPUnit is not really suitable to test. (it would be if it were checking 10000 pages with all the same "structure" maybe).
 
Lasse Koskela
author
Posts: 11962
5
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hmm. Is there any patterns in those 100 inputs/outputs? If there isn't, then I guess your only options are to either pick a subset of those inputs you consider most important, or to test each and every one of them.

Oh, and remember to refactor out the duplication you manage to create when adding new tests (this is where the presence of patterns really helps).
 
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi ,

It's very good question . u can do one thing for testing of data base driven use any automation tool which is supporting database test like winrunner and give the Negative testing input parameters also, if ur application is working fine with ngative testign paramates , then ur application is giving wrong for the perticular naegative testing parametes.so u can find out where and at what conditions ur application is failing and u can update ur code and u can imporve quality of application.

i hope u can follow like this.
 
Ranch Hand
Posts: 145
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Tonny Tssagovic:
Thanks for the reply
Yeah, but what I mean is that if we have 100 different options to test (with all different kind of replies) then u still need to make 100 tests, and 100 different "parsers and checkers" of your reply. and u still need to double the number of testers, as the "automated" test is not much of automated, it can check if 10000 pages conform to the same schema/dtd/format, but can not check if 100 different pages are all ok (conform to the protocol).



Hi Tonny,

Let me ask you this - If there IS a problem with your code and you have to rerun 75 of those 100 tests, and out of those 75 only 30 pass, in the next test run only 35 pass (out of remaining 45), in the next test 8 out of the remaining 10 pass, and finally the last 2 pass.

Instead of having 100 tests, which are as quick to test manually as well as writing the automated tests - you end up with (..quick mental arithmetic...) 232 tests! You would have more than twice as many manual tests to set up and run and check the replies / database entries. If you automated the tests, you can now rerun your automated tests at the press of a button.

Also if you add in new features into your software, you test them and the new features pass their tests. What if the new features had an adverse affect on another part of the program? With manual testing you would not know about the problem and you will have your customer complaining about buggy software. If you have automated tests then you can rerun ALL tests before releasing the code and you would have trapped that bug.

Being able to rerun the test is the biggest driver for using automated testing.

Regards,

Fintan
[ June 04, 2004: Message edited by: Fintan Conway ]
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Fintan Conway:

If you automated the tests, you can now rerun your automated tests at the press of a button.



Or you can integrate running the tests into the automated build process.

We do have an instance of Cruise Control runnning on our build server. Every two hours, it checks out the current sources, builds the system and runs the automated tests. If something fails, the developers get notified via email.

This kind of feedback is *very* valuable!
 
You showed up just in time for the waffles! And this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
reply
    Bookmark Topic Watch Topic
  • New Topic