Axini develops tools for model based testing (MBT) and model based software engineering (MBSE). Model based testing is a software testing approach in which test cases are automatically generated and executed from a model, a formal specification of the system under test. This approach allows for a high degree of test automation and more thorough testing.
The Axini Modeling Platform implements coverage metrics such as state coverage and transition coverage and has test generation algorithms (strategies) to optimize the achieved coverage.
However, our strategies have various limitations when it comes to difficult cases. Our models generally consist of multiple processes that are simulated in parallel. To reach a desired part of the model, the strategy would need to reason about the interaction between processes. Similarly, our models incorporate data in the form of variables, assignments and conditions. Again, a strategy would need to reason about this data domain in order to reach a desired part of the model. Students could research what is needed to include these concerns in the test case generation algorithms.
Furthermore, there are various coverage metrics that we currently ignore altogether. Parallel models allow for different interleavings of (input and output) actions. And how to measure coverage of the data domain?
Finally, the Axini Modeling Platform also allows models to express time constraints in which input or output is allowed. How can we measure coverage over this time domain and what algorithms are needed to generate test cases to optimize these measurements?
In the past, various students have explored parts of the problems described and have developed partial solutions. However, more research is required to create a unified theory/strategy about coverage of symbolic transition systems that includes parallelism, data and time aspects.
Possible research questions
There are several puzzles and research questions that students can work on.
Recent work
Recent work at Axini on this topic:
- Mark Bebawy (2022) implemented and evaluated a transition pair coverage metric and strategy.
- Sjoerd van der Heijden (2019) implemented and evaluated a strategy that aims to optimize the trace coverage metric.