A12-TMT: Efficient Test Management for Software Projects of Any Size

A12-TMT is mgm’s new Test Management Tool: We spoke to Development Manager Oliver Storch (OS) and Product Owner Maria Kramer (MK) to gain an insight into its features and benefits. Find out how A12-TMT helps companies to make quality assurance in software projects of any size more efficient and successful.

insights: What is the new mgm A12-TMT Test Management Tool and who is it for?

OS: In summary, A12-TMT is a test management tool that supports all activities in the software testing process, focusing on the concept of interdisciplinary and risk-based test coverage. Any company involved in software development can benefit from A12-TMT: for example, it is suitable for custom software vendors, companies in the automotive industry, and regulated industries such as banking or insurance. But also public institutions with software development activities, such as the IT department of a government agency or state administration, can use A12-TMT. The Test Management Tool is not only aimed at large enterprise software in complex IT landscapes. The software is also suitable for small and medium-sized projects, even without reference to enterprise software.

However, mgm operates in an enterprise environment and manages very complex projects with a well-defined professional structure that are developed over a long period of time. This experience has gone into the development of our new Test Management Tool: A12-TMT enables the successful completion of even the most complex projects. A12-TMT also allows you to react quickly and flexibly to changing requirements.

insights: What does “interdisciplinary” mean in this context?

OS: Interdisciplinary in this context means that a software project consists of different testing disciplines. This includes manual end-to-end testing, test automation, performance testing, model testing, load testing and others. The aim is to be able to orchestrate all these aspects within or through A12-TMT.

Although A12-TMT was originally developed for manual testing, its API allows the integration of tools from other disciplines. This allows the TMT to orchestrate different tools and act as a comprehensive overall testing tool. The results of all participating tools can be aggregated in A12-TMT and presented as a report to provide a comprehensive view of the current test status and the software project.

insights: Does the ability to integrate tools into A12-TMT via an API mean that customers can customise the tools as needed?

OS: It is definitely possible to customise the tools. The standard set that comes with the Test Management Tool is specifically designed for manual end-to-end testing. It includes a clear structure and hierarchy of test suites with associated test cases that serve as a comprehensive test catalogue. These can be organised into test runs to execute the tests and generate meaningful reports. Additional tools from other disciplines beyond manual testing can be optionally integrated into A12-TMT via the API. We offer some proprietary tools such as the GUI test automation tool QF-Test, but third party tools such as Jira as a ticket tracking system can also be integrated.

However, customers always have the option to integrate their own tools. This can either be done by us as part of a project or by the customer themselves.

insights: You mentioned at the beginning that A12-TMT is “risk-based”. What does that mean?

OS: Being risk-based means that we use specific methods to prioritise. In an agile enterprise software project with sprint and release cycles of two to three weeks, it is not practical or possible to test the entire software every time. Similar challenges arise with traditional waterfall project cycles. Therefore, prioritisation is required to determine the core objectives of the next tests. In A12-TMT we use a risk-based approach that assesses the risk of test cases. Based on a risk matrix or a risk pie chart, test cases with high risk – combined from probability of occurrence and severity – are identified and prioritised accordingly for execution.

insights: How does prioritisation work in risk-based testing?

OS: Risk-based testing is about minimising risks. These are the risks that occur when we put the finished software into production. There are different risks that can occur and cause different levels of damage. For example, a faulty payment system in an online shop can cause significant financial damage to a company. However, if only an image is not displayed or the newsletter cannot be subscribed to, the risk is lower and the potential damage is smaller.

Risk can be assessed in a number of ways: First, by the tester’s judgement. Their personal experience and professional expertise help them to assess the likelihood and impact of a risk. This approach already provides an informed risk assessment. Calculating the probability of an event based on prior knowledge and new evidence is the essence of a Bayesian forecasting model, whose statistical methodology has been implemented in A12-TMT in combination with the Pareto principle.

insights: Are there other approaches to prioritisation?

OS: Yes, there are machine learning and artificial intelligence approaches. These use insights from previous test runs or training data to predict where certain risks might occur and how they should be scored. Based on this, suggestions are made for prioritising test cases. Typically, new use cases or those that have changed from the previous version of the software are considered particularly risky and therefore prioritised for testing. The aim is to identify potential problems before the software goes into production.

insights: And how much artificial intelligence is already built into the A12-TMT?

OS: Within mgm, we have already initiated several proof-of-concept projects in this area. One is focused on generating valid test cases and data for the A12 models that form the basis of A12-TMT. We are exploring various options and collaborating internally within MGM as well as with two Bavarian universities. Our goal is to define a robust process with an appropriate model, language patterns and training data to minimise the risks of AI. AI can provide valuable insights, enable quick decisions and process data efficiently. However, the benefits should outweigh the risks, as AI can make mistakes. It is important that the benefits over manual testing are clear. Human decision-making skills remain crucial, similar to self-driving cars where the human ultimately decides and the machine assists.

insights: Exciting! The human factor is still important. How does mgm help companies build new test teams and what consulting services do you offer?

MK: With our Professional Services, we have specialised teams that can support the entire quality assurance value chain – from test case development and preparation to test execution. These teams act as an extension of the department and provide support all the way through to reporting. They are familiar with both the tool and the QA methods and processes and can provide comprehensive support in both areas.

insights: Apart from the risk-based approach, are there any other special features of A12-TMT?

MK: One feature that was particularly important to us during development was versioning, to be able to map complex structures over time. This may not sound exciting, but it is extremely helpful in a business environment or for testers in their daily workflow. Without versioning, we’d literally have thousands of test cases that would have to run in different versions at the same time – whether it’s the version in development and testing, or the version in production. Versioning is an effective time saver, as it allows us to respond to problems in the production system and test hotfixes without the changes already introduced affecting the tests.

Another key aspect of the A12-TMT is the change comparison. This is often overlooked, but is very useful in day-to-day testing. The ability to quickly identify and understand changes between versions is crucial for responding to customer requests and maintaining a clear overview of project progress. These checks are often manual or overlooked, which can lead to information gaps.

Support for searching large numbers of test cases in a complex project context, for example with tens of thousands of test cases, is also critical. The tool-based navigation support, which eliminates the need to manually develop workarounds, is extremely helpful.

insights: How do companies benefit from the pre-built test case catalogues in A12-TMT?

OS: Our optional component of A12-TMT allows customers to use pre-built test case catalogues based on our mgm expertise. These currently cover three main areas: E-Commerce, Accessibility and Web Security. In the e-commerce domain, typical functionalities such as product catalogues, login, registration, payment processes and checkout are covered, supported by years of project experience. Although occasional customisation may be required, these catalogues provide a solid foundation and save the client time and resources as they already cover around 80 percent of the test cases. The catalogues are flexible enough to be customised as required. For accessibility, we follow established standards such as the BITV tests to ensure a consistently structured test result. Thanks to these pre-built catalogues, we can apply tests to different e-commerce solutions and evaluate accessibility efficiently.

insights: And how can companies make the best use of their resources and successfully implement projects with A12-TMT?

MK: It’s really a question of where the company is and how much expertise they have. They can either rely on internal resources or they may lack specific knowledge and a broad-based team. That’s where our quality assurance team comes in, to strengthen the customer’s team and provide them with the necessary tooling expertise. If required, we can also provide an entire team or individual experts for ongoing support. Ultimately, tool delivery and quality assurance go hand in hand – our comprehensive quality landscape, which includes not only A12-TMT and QF-Test but also other QA tools, is the best proof of this.

insights: Maria, Oliver – thank you for the insightful interview!