PREFACE

Testing our products gives us confidence that they perform as intended. If we do not find any bugs, then everything should be fine… or not? What, if we do not really have confidence in the test itself? Are we sure we test the parts that are really important? Is our test complete and, if we think so, are we able to prove this? What about the cost of testing? Is it possible to spare time or money in the testing phases?

Questions like this brought the testing community to think about using models for testing purposes, and to develop a new testing approach called model-based testing (MBT). The initial idea of MBT dates back to 1970s, but its industrial application on a larger scale started only during the last decade. Today, the community of MBT adepts is growing constantly and the topic attracts more and more attention. Therefore, the International Software Testing Qualifications Board (ISTQB) decided to issue a new certification scheme, the “ISTQB Certified Tester Foundation Level – Model-Based Tester Certification.”

The ISTQB standard glossary of terms used in software testing [1] defines MBT as “testing based on or involving models.” By model, we mean a somehow formal or structured representation of the system under test, its environment, or the test itself, which directly supports test analysis and design activities, but also test planning and control, implementation and execution, and reporting activities.

Why MBT is worth reading a book or attending a training course? In 2014, we performed a user survey among MBT practitioners and asked them about their expectations regarding MBT (see Figure 1). 1

Figure 1 Expectations of MBT practitioners

(from 2014 MBT User Survey).

The top four expectations clearly relate to the questions we asked about testing practices in the beginning of this section:

• Better tests

  We expect that MBT will help us to increase the effectiveness of our test process, for example by discovering more bugs during the various test cycles.

• Cheaper tests

  We are interested in automated test generation to reduce the costs of test creation and maintenance and, thus, to increase the overall efficiency of our test process.

• Managing complexity

  Models are abstract representations of some existing or planned reality. They proved to be very helpful in system architecture and design. Using them in the test process should help us to keep the overview and to focus on what matters for our test objectives.

• Start test design earlier

  One of the major promises of MBT is early requirements validation. We can start the modeling activities long time before the system under test exists in reality. Thus, MBT should help us in implementing the ISTQB good practice of early testing.

This book will show you how MBT can fulfill these expectations based on the current state of the practice in this area as taught in the ISTQB Model-Based Tester – Foundation Level certification syllabus. Furthermore, we focus on essential aspects of model-based testing in practice. Beyond the syllabus, we provide additional information on MBT and share with you our personal experience with the technique.

The 2014 MBT User Survey

In 2012, Robert V. Binder conducted the first survey among MBT practitioners to collect data and experience on the usage of model-based testing [2]. One year later, we started an independent initiative at ETSI's User Conference on Advanced Automated Testing. The 2013 survey was more technical and aimed at validating a common MBT classification scheme. Finally, we decided to join both initiatives and started the 2014 MBT User Survey.

The survey was open from mid-June 2014 to early August 2014. One hundred model-based testing practitioners answered, providing a relevant picture of the current state of the practice in the MBT area. We wanted to learn how MBT users view the efficiency and effectiveness of the approach, what works, and what does not work. The results are publicly available [3]. For a detailed analysis, please refer Ref. [4].

THE ISTQB CERTIFIED TESTER FOUNDATION LEVEL – MODEL-BASED TESTER

There are many different approaches to model-based testing. Some adepts place their expectations in complete automation, both of test case generation and test execution. Others use the models mainly to describe the test idea and to generate test cases for manual execution. All those approaches are valid and have their advantages, but without a structured introduction to the topic, newcomers are rapidly lost. This is where the ISTQB certification scheme helps.

The ISTQB MBT certification extends the ISTQB Certified Tester Foundation Level and provides additional skills to professional testers. The qualification scheme addresses people having reached the first level of achievement in their software testing career, including testers, test analysts, test automation engineers, test managers, as well as system and software developers and architects. It may also be beneficial for anyone who wants a deeper understanding of software testing and of the use of models for test generation, such as project managers, quality managers, product managers, business analysts, and business process experts. In general, an ISTQB Certified Model-Based Tester has acquired the necessary skills to contribute successfully to MBT projects in a given context.

Figure 2 provides a summary of the ISTQB MBT syllabus content. The syllabus is divided into five chapters, covering the introduction of MBT (Chapter 1), the main phases of the MBT process (Chapters 2–4), and the evaluation and deployment of an MBT approach (Chapter 5).

Figure 2 ISTQB MBT syllabus – content overview.

The syllabus covers 36 learning objectives at different cognitive levels of knowledge (so-called K-levels):

• 9 at level K1 (Remember),
• 22 at level K2 (Understand),
• 5 at level K3 (Apply).

In addition, the syllabus includes 10 MBT-specific terms, which are equally part of the examination.

Throughout this book, we provide you with the required knowledge to pass successfully the ISTQB MBT certification. Various exercises invite you to practice modeling and test generation activities. If you are not interested in passing the exam, you may skip them, but if you are, beware of the fact that they are essential to reach cognitive level K3.

HOW THIS BOOK IS ORGANIZED?

This book consists of 12 chapters plus 3 appendices. We tried to stay as close as possible to the structure of the ISTQB MBT syllabus. However, for pedagogic reasons, there are some exceptions from this rule. To make the mapping easier for you, each chapter of this book starts with a reference to the corresponding syllabus section(s).

In Chapter 1, we introduce the topic of model-based testing and tell you what you can realistically expect from this test approach.

In Chapter 2, you find the definition of the MBT-specific terms that are part of the ISTQB MBT examination. Moreover, we define additional terms used in this book to make sure we all understand them in a similar way. Section 2.3 is very important. We present the two simple graphical modeling languages proposed by the ISTQB MBT syllabus. Unless stated otherwise, all example models in this book respect the rules of one of those two sample languages.

Chapter 3, then, goes to the root of the matter. We explain how a test process employing MBT looks like, including its activities, actors, and roles, as well as its input and output, and its integration into the global software development lifecycle.

Modeling being a core activity in MBT, three chapters of this book deal with this topic. Chapter 4 points out some fundamental aspects you should consider before starting to write an MBT model.

Chapter 5 presents a variety of different modeling languages. It shall enable you to select the most appropriate language for your particular application focus. In addition, we provide some general information on MBT modeling tools.

In Chapter 6, we present common good modeling practices and draw your attention to typical mistakes and pitfalls. Unlike the other sections of Chapter 6, Section 6.8 is not based on the syllabus. Instead, it contains a longer list of recommendations regarding modeling practices from other sources including our personal experience.

In Chapter 7, we have a short glimpse on classic test design techniques as taught in the ISTQB Certified Tester Foundation Level course. We see how MBT relates to those techniques, which it does not replace, but supports.

Chapter 8 is entirely dedicated to test case selection. We present a taxonomy of selection criteria, examples for their practical application, pros and cons, and report some recommendations. The last section of this chapter briefly deals with automated test generation in general.

Next to test generation comes test implementation and execution, which is the topic of Chapter 9. We explain the specifics of MBT test implementation, the dependencies on modeling and test generation activities, and the additional steps required to automate the generated test cases.

Chapter 10 is the last chapter having a direct correspondence in the ISTQB MBT syllabus. It deals with the introduction of MBT in a company. It is not the classic “Do a pilot first” stuff. Instead,...