Automated Generation of Test Code and Test Data for Model-Based Low-Code Applications

Note: We have anonymized the actual name of the customer project to respect the sensitivity of our customers.

The Civil Platform has been specifically designed for tax officials. It enables centralized and easy processing of various forms in a tax-related context. The Civil Platform supports intuitive communication between officials and citizens without unnecessary detours. In addition, the platform offers the ability to securely and digitally send legal notices, eliminating the cumbersome and time-consuming paper trail.

Quality Assurance Challenges for the Civil Platform

From a quality assurance perspective, the Civil Platform faced numerous challenges. The sheer volume of forms to be tested, spanning 16 states, and the wide variability of these forms were significant obstacles. In addition, the large number of relevant test scenarios due to the variety of form variants and fields per variant added to the complexity. The complex subject matter heavily influenced the test scenarios, further complicating quality assurance efforts.

A particularly challenging aspect was the verification of authorizations across federal states and their hierarchical levels. It was critical to ensure that the documents automatically sent to citizens were compliant with laws and regulations, and that sensitive data was correct. All of these factors made quality assurance an extremely critical, time-consuming, complex, and costly process.

Short testing periods and an ever-increasing testing workload exacerbated the situation and demanded the highest levels of creativity, efficiency and precision from the mgm quality team.

Civil Platform: Tasks of the Quality Team

The Quality Team of the Civil Platform project had several key tasks: It was responsible for verifying the permissions of the different user groups, especially with regard to the different federal states and their hierarchical levels. This included ensuring that caseworkers had the necessary access rights to edit and submit the appropriate forms, and vice versa.

The testers evaluated the platform by preparing, editing and completing the forms to ensure that the process ran smoothly. After completing the forms and submitting the resulting documents, they verified the accuracy of the generated results to ensure that the notices and documents generated complied with legal requirements and contained all relevant information correctly.

Strategic Challenges and Solutions in the Civil Platform Project: Automation, Maintenance and Efficiency Improvement

However, the Civil Platform project identified some potential issues that needed to be addressed first. “Without automation, the enormous manual testing effort would have been unmanageable,” said Lilia Gargouri, project manager for the mgm quality team. However, implementing automated tests for each form was challenging due to the large number of fields and test scenarios. In addition, the project was in danger of falling into a “maintenance trap” due to the regular changes made to the forms.

Of particular concern was the large number of critical functionalities with potentially significant federal implications, particularly in terms of legally creating and delivering documents to citizens with the correct sensitive data. These features consumed most of the quality team’s available capacity, resulting in low test coverage during the short test phases. The risk of critical errors remained.

In addition, changes to the forms varied from year to year and state to state, making test automation difficult. The need to dedicate many resources to address these issues was costly and unsustainable. It was clear that focusing solely on manual testing or haphazard test automation would not provide sustainable solutions. “Therefore, only a strategic approach was feasible,” emphasized Lilia Gargouri. This approach focused on critical functionality and aimed to ensure efficient use of available resources.

Creative and Innovative Solutions for Efficient Automated Testing in the Civil Platform Project

To address the quality assurance challenges in the Civil Platform project, the idea arose to leverage the data and UI models of the mgm A12 Enterprise Low Code platform. As a result, two creative model-based approaches were implemented and combined.

1. Model-based Approach: mgm A12-TDG for Automatic Test Data Generation

Valid test data sets for manual and automated testing of A12 forms were generated in the Civil Platform project using the model-based mgm A12 Test Data Generator (A12-TDG).
A12-TDG exclusively uses the A12 data models with their defined fields, validation and calculation rules from the A12 Low Code Platform. These are first transformed into mathematical models and then processed by SMT solvers, which compute solutions for the mathematical models. This is a formal system that allows statements to be made and verified about variables in various theories such as arithmetic, set theory, and other mathematical structures. This process facilitated the automated generation of test data for very complex models in the Civil Platform project.

In addition to generating test data, A12-TDG detects a specific type of error, thereby revealing inconsistencies in the rules. This is especially important for complex domain models where human error is difficult to detect, such as in the tax context of the Civil Platform project, but also in large insurance products.

Simon Bergler, a quality assurance expert in the Civil Platform project, explains:

“By automating the preparation of the form variants to be tested, it was possible to identify defects with a stable reproducibility rate of 100%. This was critical to identifying the root cause of the defects and resolving them. This measure not only greatly facilitated the testing process, but also accelerated the reproducibility of error situations and thus error resolution”.

2. Model-Based Approach: mgm A12-ATA for Automatic Code Generation for UI Test Automation

Let’s take a complex tax form with many fields. In the past, we implemented the necessary controls to automate the testing of a complex tax form only after the user interface (UI) was complete. If the UI changed, we had to change our code accordingly.

Now, using the A12 data model, which contains the field definitions, and the UI model, from which the tax form UI is created, we generate the code for QF-Test automation of the A12 forms using A12-ATA. In addition, A12-ATA includes an interpreter with a library that can control all A12 widgets on the UI using QF-Test.

With A12-ATA, automatic filling or checking of A12 forms is possible with very little implementation: If the A12 models and thus the UI change, A12-ATA automatically generates new test automation code that can be executed directly with QF-Test. The model-driven test automation tool thus enables very early UI test automation without falling into the maintenance trap of automated tests.

Lilia Gargouri, who developed A12-ATA, explains:

“As one can imagine, A12 Low Code Test Automation with ATA is highly appealing, especially for projects with numerous forms to test. The tool saves us significant implementation and maintenance effort and accelerates the automation of our testing scenarios.”

Of course, A12-ATA requires valid test data sets to populate the A12 forms. These are automatically generated using the model-based tool A12-TDG and then retrieved by A12-ATA. For example, where a manual test of a complex tax form in the Civil Platform would have taken two hours, using A12-ATA took only ten minutes. This allowed not only the quality team, but also developers and business analysts to be involved, making it more efficient and easier to address technical and business issues.

Testing More Efficiently with A12-ATA and A12-TDG

The model-based tools we implemented for quality assurance in the Civil Platform project have been a resounding success: The implementation of the solution contributed significantly to improving the efficiency and quality of the customer’s project. The rapid deployment of low-maintenance regression tests enabled flexible bug detection and resolution, resulting in improved software quality. The quality team was freed up to focus on more demanding tasks.

Lilia Gargouri concludes, “Quality assurance in the digital age requires creative, generic solutions. By using the model-based tools A12-TDG and A12-ATA in combination, we were able to increase efficiency and meet the growing quality demands of the Civil Platform project.

Looking ahead, she ventures, “After this enriching experience, we will certainly continue to drive the development of more creative, generic tools to facilitate and accelerate enterprise software quality assurance.”

Further articles:

• Blog article “Quality Assurance in Model-Based Enterprise Software“
• Blog article “mgm A12-Test Data Generator: Automatic Test Data Generation for Complex Forms in Model-Based Enterprise Software“
• Blog article “mgm A12-ATA: A12 End-to-End Test Automation“