Maintain tests, get requirements for free
Requirements Engineering (RE) tends to be blasphemous in agile development. But numerous studies argue that projects fail unless you develop products in line with what users want. How can so many agile projects succeed without capturing the user expectations in requirements? One agile practice is to put a lot of effort into developing test cases, and then use them as requirements. This case study shows three flavors of this type of agile RE.
How come agile projects succeed without requirements engineering? Well, they don’t – not if you interpret requirements engineering as an inclusive user-oriented development activity. Agile or not, of course successful projects develop products that is appreciated by the market or bespoke customers. Agile requirements engineering typically doesn’t put a lot of upfront effort on writing a formal requirements specification, instead the user perspective is inherent in the overall development process.
One agile practice to close the gap between users and developers is to maintain a suite of realistic test cases, representing the customer perspective. Moreover, this suite is then to a large extent replacing traditional requirements, i.e., the test cases are used as requirements. In this empirical study, we returned to the rich interview data gathered during the “EASE Theme D grand publication” on aligning requirements engineering and testing – because requirements and test cases can not be more aligned than completely merged!
From reject to best paper nominee
The study has an interesting history. When collecting all interview data for the grand publication, we were convinced that we would analyze it from multiple perspectives and thus write several interesting papers. It did not really happen in the first years, instead this paper was the first attempt… And initially it failed – reject at the REFSQ conference as the interviews were not designed in line with the goal of exploring an agile requirements engineering approach. Then it was accepted at the XP conference and nominated for the best paper award! Clearly the agile community liked it more than the requirements engineering researchers. We got invited to extend the paper for a special section in IST, and after extending the discussion and adding Stapel and Schneider’s FLOW notation (see top figure) – here is the result!
The case companies use Test Cases as Requirements (TCR) in at least five different ways. We call these 1) de facto TCR, 2) behavior-driven TCR, 3) story-test driven TCR, 4) stand-alone strict TCR, and 5) stand-alone manual TCR. For each of these we discuss how they were applied in their corresponding development contexts, and we highlight benefits and drawbacks. All details obviously available in the preprint!
Implications for Research
- A rich set of interview data can indeed be reanalyzed for a new purpose.
Implications for practice
- Using Test Cases as Requirements (TCR) can be a useful component of agile requirements engineering.
- TCR enforces improved communication as the requirements turn into an actively maintained test suite -supporting eliciting, validating and managing new and changing customer needs.
- With TCR there is no need to discuss traceability between individual requirements and test cases – they are the same.
- TCR poses challenges to the elicitation and verification of quality requirements – it is often difficult to capture quality aspects in a test suite.
Elizabeth Bjarnason, Michael Unterkalmsteiner, Markus Borg, and Emelie Engström. A Multi-case Study of Agile Requirements Engineering and the Use of Test Cases as Requirements, Information and Software Technology, 77, pp. 61-79, 2016. (link, preprint)
Abstract
Context. It is an enigma that agile projects can succeed ‘without requirements’ when weak requirements engineering is a known cause for project failures. While agile development projects often manage well without extensive requirements test cases are commonly viewed as requirements and detailed requirements are documented as test cases. Objective. We have investigated this agile practice of using test cases as requirements to understand how test cases can support the main requirements activities, and how this practice varies. Method. We performed an iterative case study at three companies and collected data through 14 interviews and two focus groups. Results. The use of test cases as requirements poses both benefits and challenges when eliciting, validating, verifying, and managing requirements, and when used as a documented agreement. We have identified five variants of the test-cases-as-requirements practice, namely de facto, behaviour-driven, story-test driven, stand-alone strict and stand-alone manual for which the application of the practice varies concerning the time frame of requirements documentation, the requirements format, the extent to which the test cases are a machine executable specification and the use of tools which provide specific support for the practice of using test cases as requirements. Conclusions. The findings provide empirical insight into how agile development projects manage and communicate requirements. The identified variants of the practice of using test cases as requirements can be used to perform in-depth investigations into agile requirements engineering. Practitioners can use the provided recommendations as a guide in designing and improving their agile requirements practices based on project characteristics such as number of stakeholders and rate of change.
