Testing software is inherently incomplete. No matter how thoroughly you test a system, there remains always some probability that one or more faults are present. Still, observing correct behavior increases our confidence in the correctness of a system. To quantify this increase of confidence, several notions of 'test coverage' and 'risk' have been developed. However, many of these notions are very basic, and therefore do not predict the quality of a test process in a very accurate manner.
Axini’s tooling incorporates well-known concepts such as state coverage and transition coverage. In addition to states and transitions, our model also incorporates data in the form of variables, assignments and conditions. Selecting test cases such that data coverage is optimized remains a puzzle. Moreover, the data domain can be relevant when conditions need to be satisfied to reach certain part of the model.
In the past, various students have explored parts of this problem and have developed partial solutions. However, more research is required to create a unified theory/strategy to cover both the state/transition domain as well as the data domain.
- Possible research questions:
- How does existing knowledge integrate into a unified theory?
- How to develop a test case generation algorithm that scales?
- How to combine on-line testing with off-line strategies?
- Some theory that unifies previous work on test case generation
- An improved data coverage strategy
- A practical application in Axini's domain with a proof of concept, preferable a case study or prototype