This week's book giveaway is in the HTML Pages with CSS and JavaScript forum.
We're giving away four copies of Testing JavaScript Applications and have Lucas da Costa on-line!
See this thread for details.
Win a copy of Testing JavaScript Applications this week in the HTML Pages with CSS and JavaScript forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Bear Bibeault
  • Ron McLeod
  • Jeanne Boyarsky
  • Paul Clapham
Sheriffs:
  • Tim Cooke
  • Liutauras Vilda
  • Junilu Lacar
Saloon Keepers:
  • Tim Moores
  • Stephan van Hulst
  • Tim Holloway
  • fred rosenberger
  • salvin francis
Bartenders:
  • Piet Souris
  • Frits Walraven
  • Carey Brown

Need to Unit test an app that submits a form to a webpage

 
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hey All,
I'm developing an app (a bot really) that retrieves information from webpages by submitting forms to the page. I'm wondering how can I unit test this feature.
Just to make it clear, I do not wish to unit test the servlet/jsp that the form is being submitted to. Rather I am trying to unit test the bot that will do the submitting. Any help would be appreciated.
Sincerely,
Jon McDonald
 
Greenhorn
Posts: 15
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Some issues/hints...
* Network code can be vulnerable to thread timing issues which can make your tests fail intermittently every five or fifty or five hundred times.
* If you don't control the target website, there tends to be a bit of trial and error in finding the correct url and html formats... tests are a good way of being systematic about this trial and error... if a page breaks your bot in trials, you add it to your tests and fix your bot until it passes the tests.
* There's a question of how heavy do you have to make the test framework... if scale wasn't an issue, we might test our cnn.com headline scraping bots by simulating the whole cnn.com website in our test harness.
* Your test code might be more robust and manageable if you test the following areas of functionality separately (assuming that your bot has a structure along these lines...)
- mapping of information request to url
- the http client itself (mapping of url to html page)
- mapping of html page to list of headlines or whatever (html parsing)
 
author
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Jon McDonald:
I'm developing an app (a bot really) that retrieves information from webpages by submitting forms to the page. I'm wondering how can I unit test this feature.


Basically (and generally), divide and conquer: Identify the different steps you need to do and test them separately.
For example, you probably need to parse HTML pages for the information you want. You can develop and test this parser without any thought on sending forms over the net etc.
Did that help?
 
Jon McDonald
Ranch Hand
Posts: 167
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the info guys!
I think I'll go the divide and conquer route. It should also make for more reusable code that way.
Jon
 
Ilja Preuss
author
Posts: 14112
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Originally posted by Jon McDonald:
It should also make for more reusable code that way.


Yes!
In fact, that is one of the reasons why many developers nowadays find that taking testing into account from early on (often even before writing the actual production code) makes for better designs: for code to be testable, it needs to be well decoupled.
See http://c2.com/cgi/wiki?TestDrivenDevelopment
 
You didn't tell me he was so big. Unlike this tiny ad:
Building a Better World in your Backyard by Paul Wheaton and Shawn Klassen-Koop
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
    Bookmark Topic Watch Topic
  • New Topic