本书的内容基于爱尔兰国立大学梅努斯和浙江大学的一系列课程讲稿。该书覆盖了软件测试的基本原理和主要方法，可以作为软件测试全英课程或双语课程的教材。

无

Preface

This book is based on a series of lectures given at Maynooth University and Zhejiang University. It provides a textbook for a number of courses, describing the fundamentals of software testing. The material has been developed over the past ten years, and reflects both the experience of 20 years in industry from one of the authors, and the authors’ joint experience in lecturing.
There is no one standard textbook on software testing, and this book is the result of many years of extracting and interpreting test techniques from a wide and varied number of sources. These include testing classics such as The Art of Software Testing by Myers, Software Testing by Roper, Testing Object- Oriented Systems by Binder, and Introduction to Software Testing by Ammann and Offutt; standard software engineering textbooks such as Software Engineering by Pressman , and Software Engineering by Sommerville; software process books such as Software Testing in the Real World by Kit and extreme Programming explained by Beck; and ISO and IEEE standards related to software quality and testing.
Software testing is a challenging task–it is as important for businesses and government as it is for research institutions. It is still as much an art as a science: there are no accepted standards or norms for applying the various techniques, and interpretation is required. There is no well established research on the effectiveness of the different approaches. The techniques are easy to understand, but generally difficult to apply to real-world software. By providing extensive worked examples, this book aims to provide a solid basis for both understanding, and applying, various test techniques
In the second edition, many of the examples have been updated, and the sections on Integration Testing and System Testing have been expanded. The examples have been updated to work with the latest version of the tools used, and a companion website is provided (www.softwaretestingbook.org).
The software tools used in this book are merely representative–they have not been selected as the best example of each, but rather as good examples of the range of capabilities that such tools exhibit. The key goal of the book is that the reader should understand the principles of software testing, and be able to apply them in practice. The book does not endorse or recommend any particular tool. Only a subset of the tool features are covered in this book: the reader should refer to the tool-specific documentation for more details.



前　　言
本书的内容基于爱尔兰国立大学梅努斯分校和浙江大学的一系列课程讲稿。该书覆盖了软件测试的基本原理，可以作为许多课程的参考教材。本书的内容历经十年发展，既反映了其一位作者二十年的工业界经验，也融合了多位作者在教学方面的共同经验。
在软件测试领域，目前还没有统一的标准教科书，而本书是通过对各种不同的软件测试技术进行多年的提炼、阐释而形成。这些测试技术的来源包括一些经典的测试书籍如Myers的《软件测试的艺术》、Roper的《软件测试》、Binder的《面向对象系统测试》、Ammann和Offutt的《软件测试基础》；标准的软件工程教材如Pressman的《软件工程：实践者的研究方法》、Sommerville的《软件工程》；软件过程类书籍如Kit的《现实世界中的软件测试》、Beck的《极限编程》以及软件质量及测试相关的ISO和IEEE标准。
软件测试是一项具有挑战性的任务，它对科研机构、企业及政府具有同等的重要性。软件测试既是门科学也是门艺术。针对如何应用不同的软件测试技术，目前还没有广泛认同的标准，因此对各种软件测试技术的阐释是必须的。针对各种不同技术的有效性，目前还没有成熟的研究成果。总的来说，各种测试技术容易理解，但如何将其应用到软件产品中是困难的。本书旨在通过广泛丰富的实例，为读者对理解和如何应用各种不同的软件测试技术提供一个坚实的基础。
第2版对第1版的内容进行了全面更新和修订，新增和扩充了集成测试和系统测试两章。示例已更新，以使用新版本的工具，并提供了一个配套网站（www.soft-wareetestingbook.org）。
本书中使用的软件工具仅仅是这些工具中的代表—我们选择这些工具并不是因为它们是最佳工具，而是这些工具能够较好地展示我们想要说明的功能。本书的主要目标是让读者了解软件测试的原理，并能在实践中应用。本书不支持或推荐任何特定的工具。书中只介绍了工具的一部分特性：读者应该参考特定工具的详细文档。

计算机\软件测试

软件测试是一项具有挑战性的任务，它对科研机构、企业及政府具有同等的重要性。软件测试既是门科学也是门艺术。针对如何应用不同的软件测试技术，目前还没有广泛认同的标准，因此对各种软件测试技术的阐释是必需的。针对各种不同技术的有效性，目前还没有成熟的研究成果。总的来说，各种测试技术容易理解，但如何将其应用到软件产品中是困难的。本书对各种不同的软件测试技术进行提炼和阐释，旨在借助广泛丰富的实例，为读者理解和应用各种不同的软件测试技术提供一个坚实的基础。
全书内容基于爱尔兰国立大学梅努斯分校和浙江大学的一系列课程讲稿，融合了五位作者多年的行业经验和教学实践。本书主要是面向软件测试的本科课程来设计相关内容，覆盖了软件测试的基本原理和主要技术，辅以许多实际的例子来说明，可以作为软件测试本科课程的教材。

本书特色
凝练作者多年的行业经验和教学成果，以丰富的实例，系统阐释软件测试的基本理论和技术。
附录中给出大量与实际应用相关的练习，便于读者理解和巩固所学知识。
涵盖一些高级主题，如测试技术选择、用浮点数测试、用复杂数据结构测试、GUI组件的单元测试、嵌入式软件测试等，需要读者在接下来的学习和工作实践中不断深入研究和思考。
第2版对第1版的内容进行了全面更新和修订，新增和扩充了集成测试和系统测试两章，介绍了近年来出现的软件测试新方法，并增加了大量例题和练习，更加符合工业界需求和教学要求。

Preface ii
1 Introduction 1
1.1 The Software Industry 1
1.1.1 Software Testing and Quality 2
1.1.2 Software Testing and Risk Management 3
1.2 Mistakes, Faults and Failures 3
1.2.1 Software Faults 4
1.2.2 Software Failures 5
1.2.3 Need for Testing 6
1.3 The Role of Specifications 8
1.4 Overview of Testing 9
1.4.1 Testing in the Development Process 9
1.4.2 Test Automation 9
1.5 The Theory of Testing 10
1.5.1 Exhaustive Testing Example 11
1.5.2 Implications 11
1.5.3 When To Finish Testing 12
1.6 Book Structure 13
1.6.1 Order of Testing 14
1.6.2 Programming Language 14
1.6.3 Level of Detail 14
1.6.4 Code Examples 14
1.7 Notes on Terminology 14
2 Principles of Software Testing 15
2.1 Static Verification 15
2.1.1 Walk-throughs 16
2.1.2 Code Reviews/Inspections 16
2.1.3 Formal Methods and Testing 18
2.2 Dynamic Verification 19
2.3 Black-Box and White-Box Testing 19
2.3.1 Errors of “Omission” and “Commission” 21
2.4 Test Approaches 22
2.4.1 Black-Box Testing 22
2.4.2 White-Box Testing 23
2.4.3 Fault Insertion 24
2.5 Test Design Process 24
2.5.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.2 Generating Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.5.3 Generating Test Data . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.5.4 Implementing Tests . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.6 Analy sis of Software Specifications . . . . . . . . . . . . . . . . . . . . . . 27
2.6.1 Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.6.2 Parameter Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.6.3 Equivalence Partitions . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.6.4 Boundary Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6.5 Combinations of Values . . . . . . . . . . . . . . . . . . . . . . . . 30
2.6.6 Sequences of Values . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.7 Analy sis of Software Components . . . . . . . . . . . . . . . . . . . . . . 40
2.7.1 Control Flow Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.7.2 Decisions and Conditions . . . . . . . . . . . . . . . . . . . . . . . 43
2.7.3 Paths . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.7.4 Data Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.8 Analy sis of Targets for Fault Insertion . . . . . . . . . . . . . . . . . . . . 47
2.8.1 Offutt’s 5 Sufficient Mutations . . . . . . . . . . . . . . . . . . . . 47
2.9 Test Artefacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
3 Unit Testing 50
3.1 Techniques 50
3.2 Usage 51
3.3 Black-Box Techniques 52
3.3.1 Equivalence Partitioning (EP) 52
3.3.2 Boundary Value Analysis (BVA) 53
3.3.3 Testing Combinations of Inputs 55
3.3.4 Testing Sequences of Inputs/State-Based Testing 56
3.3.5 Testing with Random Data . . . . . . . . . . . . . . . . . . . . . . 56
3.3.6 Error Guessing/Expert Opinion . . . . . . . . . . . . . . . . . . . . 58
3.4 White-Box Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
3.4.1Statement Coverage (SC) . . . . . . . . . . . . . . . . . . . . . . . 60
3.4.2 Branch Coverage (BC) . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.4.3 Condition Coverage (CC) . . . . . . . . . . . . . . . . . . . . . . 62
3.4.4 Decision Condition Coverage (DCC) . . . . . . . . . . . . . . . . . 64
3.4.5 Multiple Condition Coverage (MCC) . . . . . . . . . . . . . . . . .65
3.4.6 Modified Condition Decision Coverage (MCDC) . . . . . . . . . .66
3.4.7 Path Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
3.4.8 Dataflow Coverage (DU Pairs) . . . . . . . . . . . . . . . . . . . . 68
3.5 Fault Insertion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
3.5.1 Strong Mutation Testing . . . . . . . . . . . . . . . . . . . . . . . . 69
3.6 Test Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4 Unit Testing Examples 71
4.1 Example One: fits() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1.1 Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.1.2 Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
4.1.3 Note on Black-Box Testing . . . . . . . . . . . . . . . . . . . . . . 72
4.1.4 Equivalence Partitioning . . . . . . . . . . . . . . . . . . . . . . . . 72
4.1.5Boundary Value Analysis . . . . . . . . . . . . . . . . . . . . . . . 77
4.1.6 Combinational Testing . . . . . . . . . . . . . . . . . . . . . . . . . 78
4.1.7 Random Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
4.1.8 Testing Sequences of Inputs . . . . . . . . . . . . . . . . . . . . . . 82
4.1.9 Elimination of Duplicate Black-Box Tests . . . . . . . . . . . . . .82
4.1.10 Important Note on White-Box Testing . . . . . . . . . . . . . . . .83
4.1.11 Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.1.12 Statement Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 84
4.1.13 Branch Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.1.14 Condition Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 87
4.1.15 Decision/Condition Coverage . . . . . . . . . . . . . . . . . . . . . 88
4.1.16 Multiple Condition Coverage . . . . . . . . . . . . . . . . . . . . . 90
4.1.17 Modified Condition Decision Coverage . . . . . . . . . . . . . . . .90
4.1.18 Path Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.1.19 Dataflow Coverage (DU Pairs) . . . . . . . . . . . . . . . . . . . . 93
4.1.20 Elimination of Duplicate Tests . . . . . . . . . . . . . . . . . . . . 95
4.1.21 Test Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.2 Example Two: premium() 97
4.2.1 Description 97
4.2.2 Specification 97
4.2.3 Equivalence Partitioning 97
4.2.4 Boundary Value Analysis 101
4.2.5 Combinational Testing . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.2.6 Random Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
4.2.7 Elimination of Duplicate Black-Box Tests . . . . . . . . . . . . . .105
4.2.8 Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.2.9 Statement Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 107
4.2.10 Branch Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
4.2.11 Condition Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.2.12 Decision/Condition Coverage . . . . . . . . . . . . . . . . . . . . . 111
4.2.13 Multiple Condition Coverage . . . . . . . . . . . . . . . . . . . . . 113
4.2.14 Modified Condition Decision Coverage . . . . . . . . . . . . . . . .116
4.2.15 Path Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.2.16 Dataflow Coverage (DU Pairs) . . . . . . . . . . . . . . . . . . . . 119
4.2.17 Elimination of Duplicate Tests . . . . . . . . . . . . . . . . . . . . 121
4.2.18 Test Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 121
5 Unit Testing Object-Oriented Software 123
5.1 Characteristics of Object-Oriented Software 124
5.2 Effects of OO on Testing 124
5.3 Object-Oriented Testing Models 125
5.3.1 Conventional Models 125
5.3.2 Combinational Models 126
5.3.3 State Machine Models 127
5.3.4 Specification & Design Models 129
5.3.5 Built-In-Test 130
5.4 Example 1 131
5.4.1 Class CarTax 131
5.4.2 Black-Box Testing in Class Context 131
5.4.3 Equivalence Partitioning 132
5.4.4 Boundary Value Analysis 134
5.4.5 Combinational Testing 134
5.4.6 White-Box Testing in Class Context 134
5.4.7 Combinational Testing at the Class Level 135
5.4.8 State-Machine Testing 137
5.4.9 Specification/Design Testing 141
5.4.10 Built-In Testing 141
5.5 Example 2 143
5.5.1 Classes Shape and Square 143
5.5.2 Specification/Design Testing 143
6 Integration Testing 145
6.1 Principles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
6.1.1 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
6.1.2 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.1.3 Test Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.1.4 Test Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.2 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.2.1 The Premium System . . . . . . . . . . . . . . . . . . . . . . . . . 148
6.2.2 Sequence Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
6.2.3 Source Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.2.4 Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
6.2.5 Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.2.6 Test Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.2.7 Test Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 153
6.2.8 Interpretation of Test Results . . . . . . . . . . . . . . . . . . . . . 153
6.3External Integration Testing . . . . . . . . . . . . . . . . . . . . . . . . . 153
6.4 Unit Testing during Software Integration 154
6.4.1 Drivers and Stubs 154
6.4.2 Top-Down Integration 154
6.4.3 Bottom-Up Integration 155
6.4.4 Incremental Feature/Continuous Integration 155
7 System Testing 156
7.1 Introduction 156
7.1.1 System Test Categories 156
7.2 Usage 158
7.2.1 System Level Functional Testing 158
7.2.2 Test Cases 160
7.3 GUI Testing Example 160
7.3.1 Example System 161
7.3.2 Testing the Interface Behaviour 167
7.3.3 Testing the Interface Navigation 168
7.3.4 Testing Software Features 170
7.3.5 Testing Use Cases and Scenarios 173
7.3.6 Testing a Web-Based GUI 174
7.4 Field Testing and Acceptance Testing 175
7.4.1 Field Testing 175
7.4.2 Acceptance Testing 175
8 Software Test Automation 176
8.1 Unit Test Automation 177
8.1.1 TestNG 177
8.1.2 Basic TestNG Features 177
8.1.3 Other TestNG features 179
8.1.4 Documenting Automated Tests 181
8.1.5 Automated Unit Testing in an IDE 182
8.2 Automated Unit Test Example: fits() 182
8.2.1 Testing fits() with Inline Data 182
8.2.2 Parameterised Tests for fits() 183
8.3 Coverage Measurement Automation 186
8.3.1 EclEmma Example 186
8.3.2 Lazy Evaluation 187
8.4 Automated Object-Oriented Unit Testing 188
8.4.1 CarTax Example 188
8.4.2 Automating Inheritance Testing 188
8.5 Automated System Testing 190
8.5.1 Swing-Based GUI Example 191
8.5.2 Web-Based GUI Example 197
8.6 Selenium IDE (Integrated Development Environment) . . . . . . . . . . .205
8.7 Other Test Automation Tools and Techniques . . . . . . . . . . . . . . . .205
8.7.1 Customising the Test Runner 205
8.7.2 Stress Testing 206
8.7.3 Mutation Testing 207
9 Testing in the Software Process 209
9.1 Test Planning 210
9.2 Software Development Life Cycle 211
9.3 The Waterfall Model 211
9.4 The V-Model . 212
9.5 Incremental and Agile Development 213
9.5.1 Incremental Development 214
9.5.2 Extreme Programming 215
9.5.3 SCRUM 217
9.5.4 DevOps 218
9.5.5 Process-Related Quality Standards and Models 219
9.6 Repair-Based Testing 219
9.6.1 Specific Repair Test 219
9.6.2 Generic Repair Test 219
9.6.3 Abstracted Repair Test 220
9.6.4 Example 220
9.6.5 Repair-Based Test Suites 220
10 Advanced Testing Issues 221
10.1 Philosophy of Testing 221
10.2 Test Technique Selection 222
10.3 Design For Testability (DFT) 222
10.4 Overlapping and Discontinuous Partitions 223
10.4.1 Overlapping Input Partitions 223
10.4.2 Overlapping Output Partitions 223
10.4.3 Inband Error Reporting 224
10.5 Handling Relative Values 225
10.5.1 Classic Triangle Problem 226
10.6 Pairwise Testing 227
10.7 Modified Combinational Testing 228
10.8 Improved Condition Coverage 229
10.9 Testing with Floating Point Numbers 229
10.10 Testing with Complex Data Structures 231
10.11 State-Based or Sequential Testing 231
10.12 Inserting Data Faults 235
10.13 Automated Statement and Branch Testing 235
10.14 Automated Random Testing 235
10.15 OO Dataflow Testing (DU Pairs) 237
10.16 Verifying State in Object-Oriented Testing 239
10.17 Advanced OO Testing 240
10.17.1 Multiplicity in UML Class Diagrams 241
10.18 Text-Based Value Representation 242
10.19 Unit Testing of GUI Components 243
10.20 Some issues in Testing Web-Based Applications 244
10.21 Responsive UI Design Testing 244
10.22 Testing Concurrent and Parallel Software 244
10.22.1 Unit Testing 245
10.22.2 System Testing 245
10.22.3 Static Analysis 245
10.22.4 Tools 245
10.23 Testing Embedded Software 246
10.24 Testing Network Protocol Processing 248
10.24.1 Text-Based Protocols 248
10.24.2 Binary Protocols 249
10.24.3 Protocol Stacks and DFT 249
10.25 Including Testing In The Build Procedure 249
10.26 Research Directions 250
APPENDICES
A Terminology 252
B Software Testing Tools 254
C Exercises 255
C.1 TestNG and Eclipse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
C.2 Unit Test - Exercise 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
C.3 Unit Test - Exercise 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
C.4 Unit Test - Exercise 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
C.5 Unit Test - Exercise 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
C.6 Unit Test - Exercise 5 260
C.7 Unit Test - Exercise 6 261
C.8 Unit Test - Exercise 7 262
C.9 Unit Test - Exercise 8 262
C.10 Unit Test - Exercise 9 263
C.11 Exercise 10 - Test Projects 264
D Source Code 265
D.1 Chapter 4 - Unit Testing Examples 265
D.1.1 CharterFlight.java 265
D.1.2 CharterFlightTest.java 266
D.1.3 Insurance.java 267
D.1.4 InsuranceTest.java 268
D.2 Chapter 5 - OO Testing 269
D.2.1 CarTax.java 269
D.2.2 CarTaxTest.java 270
D.2.3 CarTaxWithAssertions.java 271
D.2.4 CarTaxWithAssertionsTest.java 273
D.2.5 Shape.java 273
D.2.6 Square.java 274
D.2.7 ShapeTest.java 274
D.2.8 SquareTest.java 274
D.3 Chapter 6 - Integration Testing 275
D.3.1 PremiumCalculator.java 275
D.3.2 Validator.java 276
D.3.3 PremiumCalculatorIntegrationTest.java 276
D.4 Chapter 7 - System Testing 277
D.4.1 FlightBooker.java 277
D.4.2 SimpleSwingTestFramework.java 279
D.4.3 FlightBookerBehaviourTest.java 282
D.4.4 FlightBookerNavTest.java 283
D.4.5 FlightBookerFeatureTest.java 285
D.4.6 FlightBookerScenarioTest.java 286
D.4.7 FlightBookerWebTest.java 288
D.5 Chapter 8 - Software Test Automation 292
D.5.1 Demo.java 292
D.5.2 DemoTest.java 293
D.5.3 DemoTestM.java 293
D.5.4 DemoTestA.java 294
D.5.5 DemoTestB.java 294
D.5.6 Demo2.java 295
D.5.7 Demo2Test.java 295
D.5.8 testng-1.xml 295
D.5.9 CharterFlightTestInline.java 296
D.5.10 CustomRunner.java 296
D.6 Chapter 10 - Advanced Issues 298
D.6.1 Lamp1.java 298
D.6.2 Lamp2.java 298
D.6.3 Lamp1Test.java 298
D.6.4 Lamp2Test.java 299
Select Bibliography 300