For quality assurance (QA) in complex, long-lived enterprise software, the software company mgm has developed its own holistic approach. In an interview with mgm’s editorial team, QA experts Maria Kramer (test management and manual quality assurance), Lilia Gargouri (technical quality assurance: UI test automation) and Martin Varendorff (technical quality assurance: load and performance tests) describe what this looks like and what is important.
Let’s start with a simple but fundamental question: How is quality assurance understood at mgm?
Martin Varendorff: At mgm, we always make sure that we represent the end user’s point of view. We want the software we develop to run reasonably and cleanly. That is the most important lens through which we look at the application. In the Quality Team, everyone has internalized this and testing is done from this point of view – whether manual or automated. It is always important to consider: How does it feel to the end user? Is it practical? Is it good? And of course: is it correct? Only when the application meets all the requirements of the end user, then everything really fits.
Lilia Gargouri: mgm attaches great importance to the quality of its software. That is why mgm’s quality assurance has evolved remarkably well in recent years in not just one, but several areas of quality assurance. This ranges from test conception, test management, manual testing, UI test automation, automated security tests to load and performance tests. With the mgm Quality Team, the QA experts of this colorful mgm quality landscape have been brought together to form a competence team. They support mgm projects in efficiently setting up and implementing their QA needs. This important role of mgm Quality Assurance is therefore highly appreciated by our customers.
What challenges does the mgm Quality Team face, especially in long-lived, complex enterprise software?
Lilia Gargouri: In large complex business applications, the tests, their maintenance, extension, test code, test data as well as test execution time scale up over time. Consequently, quality assurance efforts scale with time, sometimes significantly. Therefore, it is very important, ideally right at the beginning, to specify and manage the test cases thematically and in a structured way in a test management tool. The QA thus gets an overall view of the test cases and of the test coverage of the respective application at any time over the years. For this purpose, we at mgm use the mgm Test Management Tool “TMT”, which provides various useful functionalities from mgm’s best-practice point of view.
Modularization, structuring, compactification, and centralization of test code as well as refactoring and perspective thinking help to keep test execution time, maintenance as well as extension of test code moderate to low for long-life extensive projects. The mgm Quality Team attaches great importance to its efficiency – even for long-lived extensive applications.
And the customers – what do they think about quality assurance?
Maria Kramer: Over the years, in discussions with our customers, we have developed their awareness of the relevance of quality assurance. When our customers see what quality assurance can do, they usually have respect – especially when we find bugs in deep, “hidden” layers of code. Once the application is in production, fixing bugs becomes very expensive. That’s why it’s important to integrate quality assurance into the process from the very beginning. This is independent of the size of the project. It is important for customers to understand: With every failure or with every non-function, a piece of their own reputation towards the users of the application is always lost. And our goal is to save the customers from that.
Martin Varendorff: Of course, we clarify from the very beginning in dialog with the customer what quality assurance they need and what consequences too little quality assurance will have. Unfortunately, there is still too little awareness overall of why good quality assurance is worth its price. It is often necessary to explain exactly how much time is required for each task and to justify why several testers are needed, for example. But there are also very positive examples: Those responsible for the ELSTER project, for example, realized very quickly what a difference holistic quality assurance makes.
What about the QA know-how in the project teams on the customer side?
Maria Kramer: It varies greatly. In some cases, due to the change of people in charge, important knowledge holders are missing who know the historical development of the application in its technical depth and can build on it in terms of testing methods. At mgm, on the other hand, we have many experienced employees who deal exclusively with QA in the projects throughout the year and benefit from the mutual transfer of know-how in quality assurance. Because we have also distributed the knowledge over many shoulders, the coverage of the technical aspects of the customer’s project is always very high. And it is a great feeling when customers reflect their appreciation for our work and entrust us with QA for further development of the application.
Lilia Gargouri: For the customer, efficient quality assurance of his application’s business cases is foremost. It is important for the customer that the automated tests not only run stably, but also return the test results quickly and reliably. Understandably, the customer usually has no idea which technologies to use in UI test automation for stable recognition and replay of the project-specific components, and how. It needs profound knowledge and many years of experience in the multi-faceted and sometimes very complex UI test automation. The mgm Quality Team has this expertise and accompanies mgm projects in setting up and implementing their test automation needs. It offers not only a test automation library, but also a set of standardized tools and software solutions that make UI test automation an enjoyable and rewarding endeavor for the projects.
What exactly constitutes good quality assurance?
Lilia Gargouri: From the customer’s point of view, good test coverage of business processes makes good quality assurance. Here, the level of test coverage is not measured by the number of test cases, but by their quality, depth, and coverage. The business aspects, the focus of the test scenario, the quality of the selected test data and a clean and easy-to-maintain implementation of the test case all play an essential role in each test case. In short, high-test coverage starts in the test case.
“Good quality assurance requires intelligence, creativity, efficiency, combining knowledge and techniques, a feel for situations and circumstances, strategies, and perseverance. If you POSSESS MANY OF THESE, you can do magic.”
What does years of testing mean for QA project teams?
Martin Varendorff: Yes, I know many external testing groups that test practically the same thing for years. You become absolutely blind to the business. It gets boring, and as a result you lose test coverage. That’s why we always try to bring new momentum into the team, so that creativity remains in the process of finding defects. In concrete terms, this means always giving new impetus to keep the thought process fresh and awake, even away from the requirements. For example, in one of our QA project teams in the public sector, we kept adding new aspects to the tasks so they wouldn’t be repetitive. Repetition is the death of thinking.
Lilia Gargouri: Regression tests are most qualified for test automation. Repeating tests is easily done by test automation and at almost no additional cost, either within a release cycle or from release to release. Moreover, the automation of regression tests relieves the manual testers and provides them with additional time for exploratory and creative testing of the application, especially its new features. It is our responsibility to motivate our team to always shift perspective and view in order to continuously improve ourselves and the quality of our tests.
Are there any specific organizational approaches to quality assurance?
Maria Kramer: In one of our commerce customer projects, it has proven successful to always set up two mgm-side project teams for QA: one team that takes care of QA during development and one that tests after the application has been delivered. Why testing one more time? With the second team, we support the acceptance test on the customer side in close exchange with the business departments and IT professionals, because the applications are complex and have many releases per year. This often exceeds the capacities at the customer. Nevertheless, they must check the general correctness of the functionalities themselves for the final release to confirm that the result meets their internal expectations.
Lilia Gargouri: For some time now, we have also been pursuing the structural approach of assigning quality assurance experts across projects to a central competence team, the mgm Quality Team. They practically act as service providers in the projects – equipped with a “suitcase” of standardized QA tools. This is highly efficient, because it allows us to professionally expand our knowledge and drive innovations such as ATLAS or ATA. Furthermore, it helps us to standardize the QA approach and simplify the QA tools used in the projects.
What opportunities do model-based platforms such as mgm’s low-code platform A12 bring for quality assurance?
Martin Varendorff: Basically, in A12 projects, a certain amount of quality is already specified from the outset by the A12 concepts for modeling, UI/UX, platform and components, and architecture and technology. By having QA complete in A12 and its components properly tested, a lot of time can be saved in the development of these model-based projects. We no longer test the functional requirements, but the modeled requirements. That is, we no longer test whether the button is clickable at the location, but whether it is the right one and has the right name. That’s a completely different way of testing: focusing on the functional details of the application.
Lilia Gargouri: A12 offers us excellent opportunities with its model-based approach. The low-code platform gives us an incredible advantage because most information needed for test data generation, rule validation testing or UI test automation is already available in the models. Currently, the mgm Quality Team has three mgm-owned model-based quality assurance tools in its tool chain: A12 UI Automated Test Automation (ATA), the Test Data Generator (TDG) and the “Qualle” (Jellyfish) for testing the validation rules of models.
That all sounds very exciting and dynamic. What’s next for QA at mgm? What next steps are planned?
Martin Varendorff: We will push ahead with the expansion of our Quality Team. In the future, there will be a mgm-wide pool of QA experts who can be deployed in all projects. A very important step in this regard is the current expansion of our QA team in our Vietnamese branch in Da Nang. And of course, we are constantly working on further developing our QA tools. So, it remains exciting!
Maria Kramer: A good point! Because with our new tool TMT developed by mgm for the management of test processes and scenarios, we are adding another important tool to our tool chain: From structured test case creation and execution to documentation and reporting, TMT will provide professional support for quality assurance. This offers major potential for the future.
Lilia Gargouri: In TMT, the test coverage overview of each project will compromise both manual and automated tests. Test runs in TMT will include a selection of both manual and automated tests. Their test results will then be collected in TMT centrally. This contributes to generating holistic test reports and simplifies the coordination in the test phases significantly.
Maria, Lilia, Martin – thank you very much for the interesting conversation.