Myers tom badgett corey sandlerjohn wiley sons

www.it-ebooks.info

SOFTWARE

TESTING

THE ART OF SOFTWARE TESTING

Third Edition

www.it-ebooks.info

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

ISBN 978-1-118-03196-4 (cloth); ISBN 978-1-118-13313-2 (ebk); ISBN 978-1-118-13314-9 (ebk); ISBN 978-1-118-13315-6 (ebk)

1. Computer software—Testing. 2. Debugging in computer science. I. Sandler, Corey, 1950- II. Badgett, Tom. III. Title.

Contents

www.it-ebooks.info

During that same time, the authors of this edition (the third) of The Art of Software Testing published, collectively, more than 200 books, most of them on computer software topics. Some of these titles sold very well and, like this one, have gone through multiple versions. Corey Sandler’s Fix Your Own PC, for example, is in its eighth edition as this book goes to press; and Tom Badgett’s books on Microsoft PowerPoint and other Office titles have gone through four or more editions. However, unlike Myers’s book, none of these remained current for more than a few years.

What is the difference? The newer books covered more transient topics—operating systems, applications software, security, communica-tions technology, and hardware configurations. Rapid changes in computer hardware and software technology during the 1980s and 1990s necessi-tated frequent changes and updates to these topics.

A rule of thumb that in a typical programming project approximately t the time this book was first published, in 1979, it was a well-known

50 percent of the elapsed time and more than 50 percent of the total cost were expended in testing the program or system being developed.

www.it-ebooks.info

We direct this book at three major audiences. First, the professional programmer. Although we hope that not everything in this book will be new information to this audience, we believe it will add to the profes-sional’s knowledge of testing techniques. If the material allows this group to detect just one more bug in one program, the price of the book will have been recovered many times over.

The second audience is the project manager, who will benefit from the book’s practical information on the management of the testing process. The third audience is the programming and computer science student, and our goal for them is twofold: to expose them to the problems of

www.it-ebooks.info

1

www.it-ebooks.info

Software testing is easier, too, in some ways, because the array of soft-ware and operating systems is much more sophisticated than in the past, providing intrinsic, well-tested routines that can be incorporated into applications without the need for a programmer to develop them from scratch. Graphical User Interfaces (GUIs), for example, can be built from a development language’s libraries, and since they are preprogrammed ob-jects that have been debugged and tested previously, the need for testing them as part of a custom application is much reduced.

And, despite the plethora of software testing tomes available on the market today, many developers seem to have an attitude that is counter to extensive testing. Better development tools, pretested GUIs, and the pressure of tight deadlines in an ever more complex development envi-ronment can lead to avoidance of all but the most obvious testing protocols. Whereas low-level impacts of bugs may only inconvenience the end user, the worst impacts can result in large financial loses, or even cause harm to people. The procedures in this book can help designers, developers, and project managers better understand the value of compre-hensive testing, and provide guidelines to help them achieve required testing goals.

Remember that a scalene triangle is one where no two sides are equal, whereas an isosceles triangle has two equal sides, and an equilateral triangle has three sides of equal length. Moreover, the angles opposite the

www.it-ebooks.info

1. Do you have a test case that represents a valid scalene triangle?

(Note that test cases such as 1, 2, 3 and 2, 5, 10 do not warrant a yes answer because a triangle having these dimensions is not valid.) 2. Do you have a test case that represents a valid equilateral triangle? 3. Do you have a test case that represents a valid isosceles triangle?

9. Do you have a test case with three integers greater than zero such that the sum of two of the numbers is less than the third (such as 1, 2, 4 or 12, 15, 30)?

10. Do you have at least three test cases in category 9 such that you have tried all three permutations (e.g., 1, 2, 4; 1, 4, 2; and 4, 1, 2)?

www.it-ebooks.info

Now, before you become concerned about your score, consider this: In our experience, highly qualified professional programmers score, on the average, only 7.8 out of a possible 14. If you’ve done better, congratula-tions; if not, we’re here to help.

The point of the exercise is to illustrate that the testing of even a trivial program such as this is not an easy task. Given this is true, consider the diffi-culty of testing a 100,000-statement air traffic control system, a compiler, or even a mundane payroll program. Testing also becomes more difficult with the object-oriented languages, such as Java and Cþþ. For example, your test cases for applications built with these languages must expose errors associ-ated with object instantiation and memory management.

In an ideal world, we would want to test every possible permutation of a program. In most cases, however, this simply is not possible. Even a seem-ingly simple program can have hundreds or thousands of possible input and output combinations. Creating test cases for all of these possibilities is impractical. Complete testing of a complex application would take too long and require too many human resources to be economically feasible.

In addition, the software tester needs the proper attitude (perhaps‘‘vision’’ is a better word) to successfully test a software application. In some cases, the tester’s attitude may be more important than the actual pro-cess itself. Therefore, we will start our discussion of software testing with these issues before we delve into the more technical nature of the topic.

www.it-ebooks.info

When you test a program, you want to add some value to it. Adding value through testing means raising the quality or reliability of the program.

Raising the reliability of the program means finding and removing errors.

Human beings tend to be highly goal-oriented, and establishing the proper goal has an important psychological effect on them. If our goal is to demonstrate that a program has no errors, then we will be steered sub-consciously toward this goal; that is, we tend to select test data that have a low probability of causing the program to fail. On the other hand, if our goal is to demonstrate that a program has errors, our test data will have a higher probability of finding errors. The latter approach will add more value to the program than the former.

This definition of testing has myriad implications, many of which are scattered throughout this book. For instance, it implies that testing is a destructive, even sadistic, process, which explains why most people find it difficult. That may go against our grain; with good fortune, most of us have a constructive, rather than a destructive, outlook on life. Most people are inclined toward making objects rather than ripping them apart. The defini-tion also has implications for how test cases (test data) should be designed, and who should and who should not test a given program.