BDD: Documentation Over Testing!

Friday October 10 2025

Introduction

The NWO TiCToC (Testing in Times of Continuous Change) project held a workshop on Behavior-Driven Development (BDD) with partners from both academia and industry. The TiCToC project investigates methods and tools to manage and reduce the combinatorial explosion of testing complex hightech systems. Throughout our BDD day, engaging discussions and interactions unfolded around the use and application of BDD, and what BDD truly is—or should be. In this article, we aim to share our experiences and insights from the day. To summarize, we learned that BDD excels in problem description and exploration, and as such is effective for testing the business value. Scaling BDD for bigger systems is seen as a challenge, in particular for thorough testing, including bad weather testing.

BDD Day Highlights

Our objective for BDD day was to align on the definition, practical use, and ideal application of BDD. The event was a component of the TiCToC project, which fosters collaboration between industry and academia. Universities involved are the University of Nijmegen and the University of Twente, while our industrial partners are ASML, Canon Production Printing, and TNO ESI. User committee members include Axini, Cap Gemini Engineering, CGI, ICT Group, Malvern Panalytical, Nederlandse Spoorwegen, TOPIC Embedded Systems, and ZettaScale

From the outset, it was evident that opinions on BDD ranged widely—from seeing it as a hype to viewing it as a panacea, from management idea to practical tools. This diversity set the stage for a productive day.

We began with an introductory session led by BDD expert Kaspar van Dam, ensuring everyone had a common understanding. We then examined real-world BDD examples shared by industry partners, focusing particularly on whether BDD should serve as a specification tool or a testing method:

• Many participants stressed that BDD should primarily define problems and foster a shared understanding of the business problem, leaving implementation details out. Scenarios should be abstract, allowing for multiple solutions.
• To utilize BDD for test automation, it must be connected to a real system. This seems in conflict with the point above, also shown by the fact that none of the examples fully adhered to the solution-free principle of BDD, as they varied in abstraction and detail. For example, multiple When-Then pairs in sequence (needed for test automation), are considered an antipattern for clear BDD scenarios.

Breakout Sessions

Post-lunch, participants divided into groups for two sessions. In one room, discussions focused on BDD documentation created for TKH Airport Solutions. Participants engaged inreverse discovery, deducing the intended behavior from the documentation, establishing criteria for effective BDD scenarios.

In the other room, discussions centered on scalability and expression power of BDD, followed by its role within the software development process, in particular its relationship with testing.

Conclusion

By the end of the day, a shared understanding of BDD and its strengths emerged, though not without differing opinions. Key conclusions included:

• What is BDD? BDD is an approach to problem description and exploration, fostering shared understanding. It’s not primarily a testing technique but maintaining a direct connection to the system under test is vital
• Completeness of BDD Scenarios: While BDD can provide a comprehensive problem space description at an abstract level, it shouldn’t strive for exhaustive examples that cover every potential issue or failure, because it does not scale well.
• Application of BDD: BDD fits at the top of both the specification and testing pyramids.
• Documentation vs. Testing: BDD should not have testing as its sole purpose, as it is not suited for rigorous testing; other methods like model-based testing, TDD, or ATDD are recommended for thorough testing.
• Organizing BDD: We see two ways that work in practice to implement BDD in an organization
  • One way is to define and use one way of working with BDD and enforce that over the entire organization. This requires clear ownership to prevent problems with test automation, maintenance of keywords and step definitions.
  • Another way is to implement BDD on a team level and give autonomy to the team. This makes it easier to create and maintain tests but requires coordination between teams to make sure that documentation stays uniform and understandable.
• BDD in the test pyramid: BDD is one among many techniques and should be complemented with others, depending on the context and domain (for example adminis- trative systems vs. technical systems). For example, if you take the test pyramid, BDD is located at the top and is complemented with testing techniques such as unit- testing, model-based testing, manual testing and scripting.
• Natural Language Challenges: Despite efforts in precise formulation, natural language can lead to varied interpretations. Domain knowledge should be included as context, though formalization of BDD scenarios can aid precision. Formalization of BDD scenarios, a topic within the TiCToC project, is helpful here.

Practical Takeaways

• A feature file should describe one business problem.
• Multiple When-Then pairs signal potential abstraction issues or misuse for testing rather than problem description.
• Example and Rule keywords are underused.
• Not all feature files require automation, which remains a point of contention.
• Good BDD scenarios have persons or other systems as actors, formulate actions on a black-box system level, and use Given to allow freedom for implementation.

Download report