Axini develops tools for model based testing (MBT) and model based software engineering (MBSE). Model based testing is a software testing approach in which test cases are automatically generated and executed from a model, a formal specification of the system under test. This approach allows for a high degree of test automation and more thorough testing.
An important part of testing is deciding what data to use in your test cases. In Model-Based Testing (MBT), this data is automatically generated by SAT solvers based on the data constraints defined in the model. This approach ensures that test cases are systematically derived, but it raises questions about the completeness and effectiveness of data usage in testing.
Coverage, another vital aspect of testing, measures the extent to which test cases explore the system under test. At Axini, we track several coverage metrics based on the model, including transition and state coverage. These metrics help assess the thoroughness of testing efforts and identify gaps in test case execution. Coverage metrics can also serve as valuable input for a testing strategy, which in turn will aim to maximize the coverage of that specific type.
However, there is currently no established method for tracking data coverage in the Axini Modeling Platform. This gap partly stems from the challenge of defining what data coverage means and how it should be expressed. Understanding data coverage could significantly enhance the ability to evaluate and optimize test cases, ensuring they not only cover transitions and states but also effectively utilize data constraints.
Possible research questions
There are several puzzles and research questions that students can work on.
What constitutes data coverage in the context of MBT test cases?
What mechanisms can be developed to track data coverage effectively?
How can data coverage be visualized or otherwise conveyed to users?
How can we develop a test case strategy that maximizes data coverage levels?