Agile testing best practices – Gherkin-based BDD versus Visual ATDD

Discuss and formulate acceptance test scenarios to reinforce requirements elicitation has become a strong agile practice in the recent years.

They are called ATDD – Acceptance Test-Driven Development, BDD – Behavior-Driven Development and even sometimes SBE – Specification By Example.

The test practices behind these acronyms have similar philosophy, but different positioning and implementations. In this blog post, we will compare two approaches that are positioned at different levels in the agile cycles of requirement clarification:

The objective of this article is to compare visual ATDD and BDD Gherkin and to show the similarities and differences in their nature on a concrete example.

These approaches share the same philosophy: the production of test scenarios aims to reinforce the requirements elicitation and the acceptance criteria through use cases examples, which become acceptance tests.

In the ISTQB advanced level syllabus “Agile Technical Tester”, BDD Gherkin is defined as « … a technique in which developers, testers, and business representatives work together to analyze the requirements of a software system, formulate them using a shared language, and verify them automatically ». This definition also applies to visual ATDD !

This definition clearly articulates the key points:

Another emerging concept of Agile is common to both Visual ATDD and BDD-Gherkin: acceptance test scenarios are a living documentation of the product.

Epics and User Stories are volatile information: when a User Story is implemented in an iteration and accepted after the test run, this expression of the requirement has an historical value, but this cannot make a requirements repository.

Acceptance test scenarios however are bound to last over time, particularly because they are implemented into automated regression tests and often automatically executed. Therefore, they are always up to date and in-synch with the application. They are easy to read and understand, making them an adequate support of communication for technical and non-technical members of the Agile team.

The format of the scenarios is of course the first difference:

The main difference clearly appears in these two definitions: in visual ATDD, the test scenario represented by the graphical workflow presents a global vision of the addressed functionality or business process, whereas in BDD-Gherkin, it is about covering the acceptance criteria of a User Story in an atomic fashion.

Let’s take the example of a company that wants to develop a business travel management function within its HR software.

This project is sponsored by the HR department, which presented the expected functionality during a requirement clarification workshop as follows:

Travel management:

The system should enable

– an employee to create a travel plan,

– his/her manager to validate the created travel plan

– the HR department to proceed to the reimbursement of the travel expenses

This statement corresponds to a “Minimal Marketable Feature”, defined by the Agile Alliance as a minimal functionality that makes sense for the Business. This minimal dimension allows the team to focus on the first value to be delivered, which can then be complemented.

The visual ATDD test workflow for this functionality is presented in Figure 1. This workflow was discussed and formulated during a “3 Amigos” workshop attended by two representatives of the HR department (source of the business requirement), the HR application Product Owner, a tester and two developers from the team. The development of the workflow aims at aligning the whole team on the different functions to be implemented. Acceptance test scenarios and their corresponding automated scripts will directly derive from these test workflows.

The team uses the Personas method (archetypes of users of the system). Three Personas have been defined: Peter is an employee of the company; Emma is a Manager and Joe is a person from the HR department.

Several decision tables are associated with this workflow to refine business rules. For example, if the Manager rejects the request, he will have to give predefined reasons (Too expensive, Trip too long, Other reason), as shown in the following table associated with the task “Validation of the request” in the above workflow.

Test scenarios in visual ATDD are designed to cover the different use cases from the test workflow. The visual scenario, highlighted in green, is an example of a test obtained from the workflow.

Each test case produced from the workflow is formatted as textual scenario and as an automated test script. The test documentation below corresponds to scenario highlighted in the workflow of figure 2.

The Gherkin language is not well-suited to write examples at the global workflow level of an entire application module.

If one tries to write the previous test case in Given – When – Then format, one reaches the readability limits of a Gherkin scenario that becomes too long and complex (due in particular to the change of role between Peter, Emma and Joe). And remember! Our “travel management” example is simplified: in real projects, test workflows are more complex and longer than the one described in this paper. Imagine a BDD Gherkin scenario with more than 15 or 20 test steps! The readability and maintenance of such BDD Gherkin scenarios would be close too impossible.

BDD Gherkin scenarios are meant to write test for the acceptance criteria of refined User Stories, and not for application or inter-application scenarios.

An example of a good use of the BDD Gherkin is to consider a test case at the level of a User Story acceptance criteria. Let’s consider the User Story “Trip Validation/Refusal” which has been defined by refining the requirement for the “Travel Management” functionality. An acceptance criteria for this User Story is the rejection for the “too expensive” reason, for which BDD Gherkin scenario would look as follows:

User Story – Request validation
As Manager
I can refuse a travel request from a member of my team
to manage my budget

Scenario: rejection of a travel request for the reason « too expensive »

Given I’m connected as Emma
When Peter issues a travel request form
and Peter is in Emma’s team
and Emma rejects Peter’s travel request for the reason « too expensive »
Then Peter’s travel request is rejected with the reason « too expensive »

The principle of the BDD Gherkin is to develop scenarios focused on an acceptance criteria and to produce the corresponding atomic automated code. This BDD Gherkin scenario could be parameterized by a data table giving the different reasons for rejection. We won’t do this parametrization in this paper for concision reasons.

Visual ATDD and BDD Gherkin are positioned at different levels in the Agile cycle of requirements clarification, as illustrated in the following figure:

o Test workflows stand at the functionality level – Minimal Viable Features.

o Decision tables describe the User Stories at User Story acceptance criteria level.

o the Given – When – Then Scenarios are positioned at the level of the User Stories acceptance criteria.

The following figures show the respective positioning of the Visual ATDD product (Figure 5) and the Gherkin BDD (Figure 6) in relation to the backlog.

AUTOMATION WITH VISUAL ATDD AND WITH BDD GHERKIN

Test automation in Visual ATDD and BDD Gherkin follows a similar approach, based on keywords and, if useful, data tables. Test scripts are automatically generated by the supporting tools and the test developers or automation specialists code and maintain the automation keywords.

The following figure shows a Robot Framework script generated with YEST for the scenario in figure 3 of the Visual ATDD.

The CucumberStudio environment for the Gherkin BDD also allows to link the sentences of the scenario with implementation code in different languages (glues or Fixture).

So, which approach to choose: Visual ATDD or BDD Gherkin?

Well it obviously depends on your context and your objectives in implementing Agile testing practices.

If your context is that of applications in enterprise IT systems, and your objective is to define, implement, and test business workflows (or business processes), then the visual ATDD approach is relevant because it allows you to involve business-oriented roles in valuable discussions on application workflows examples. Graphical workflow formalism is familiar to business analysts and functional analysts, and decision tables will help to discuss and detail the different implementation cases of business rules.

The BDD Gherkin is a good approach to refine in detail, with business stakeholders, the acceptance criteria at the level of each User Story, discuss the examples, formulate the scenario and automate the corresponding script.

Depending on your objectives and application context, you may use Visual ATDD for a global view of the Feature and produce application or cross-application tests, and/or use the BDD Gherkin to script tests at the User Story level and produce short, atomic tests at the Acceptance Criteria level.