Originally posted by Ernest Friedman-Hill:
One thing that's interesting to me about your book's writeup is that while the title is "Java Testing and Design," the "Java" part apparently refers more to the testing technology than to the tested technology: you show how to test not only J2EE apps but .NET apps and perhaps other technologies as well.
My question: tell us what's similar, and what's different, about client-side testing of J2EE Web apps and .NET Web apps? Does the back end technology matter at all to you when your testing hat is on? Should it?
Originally posted by Ali Pope:
In one way, I think you should be able to ignore what technology is used to implement an application so that you can test the functionality of that application at face value. Regardless of what was used to create an application, it should still perform the same functions and those functions certainly need to be tested. That would seem to fall right in-line with black-box testing.
On the other hand, I think it's only fitting that you do pay attention to the technology that was used to write the application so that you can ensure that every execution path is tested (or at least try to). That would line up nicely with white-box testing. It would be great if the black-box tests covered all possible execution paths but, unless you've got a lot of black-box tests lined up, you're probably going to need to use some white-box tests
Java Testing and Design puts forth User Goal Oriented Testing (UGOT.) UGOT is a testing methodology that models the behavior of archetypal users of a service. In the book I show how to understand these behaviors and how to turn them into test agent code. The resulting code does a functional test of a service. The functional test is intelligent since the test requires multiple calls to the service to accomplish the user's goals. Rather than testing every function of the service, the UGOT method tests the service against the goals of the archetypal users. This has worked really well with GM, Sun, and others. The book covers three case studies.
At that point, you could use the archtypes all the way through the development process to design the system in the beginnging and test the system in the e
1) Agile stresses testing-first where a unit/functional test is built before and engineer writes the application software. UGOT says "test first" is a good idea but it is optional. UGOT is not nearly as militant as Agile sometimes comes across.
2) UGOT provides a decision maker with actionable knowledge to decide when a service is ready for production. UGOT provides:
- Functional test to check for regression and successful operation.
- Scalability testing for capacity planning. Scalability is stated as a function of throughput measured at the consumer/client in requests-per-second as the number of concurrent consumers/clients increases. This shows that the system has enough capacity to serve forecasted levels of users. (See http://www.pushtotest.com/Docs/howto/onscalability2.html to see how I have been plotting this for our customers. I'm open to feedback/criticism.)
- Performance testing to make sure the service meets user expectations for response times. A 3 tall mocha latte's a day person isn't going to wait more than 5 seconds for their email client to check for new messages.
3) Bret described "coaching tests, a way of thinking about Acceptance tests" that turn user stories into tests. UGOT identifies archetypal users by their behavior when using the service. By understanding the archetypal users behavior, we can then write test agent code that drives the service, as the archetypal user will.
Java Testing and Design shows the reason why building archetypes is important to a business, how to go about doing it, and then how to repurpose them between developers, QA technicians, and IT managers.
When you make a request to the service, it's functionality and speed-to-perform-the-request depends on what other requests are being served concurrently. Black-box testing is fine to achieve a simple regression test to make sure the box still performs functions correctly, but you cannot take the response and extrapolate that the functional will be correct for subsequent requests.
Java Testing and Design puts forth User Goal Oriented Testing (UGOT.) UGOT is a testing methodology that models the behavior of archetypal users of a service.