ERP Testing Strategy: How to Test a Software Package Before Go-Live

A testing strategy is a decisive step in every ERP solution design. The approach strengthens risk management: testing a software package before implementation lets you improve and optimize it before defects ever reach production. The stakes are considerable.

"In 2022, poor software quality cost the United States economy at least US$2.41 trillion." Source: Consortium for Information and Software Quality (CISQ), 2022 report

Why do you need an ERP testing strategy?

An ERP testing strategy reduces implementation risk by concentrating the testing effort on the most complex portions of the solution, the ones where defects would have the greatest impact. A large project generally includes 15 test categories. These categories fall into two broad classes: solution tests, which validate business processes and configuration, and technical certification tests, which confirm the reliability of the infrastructure, performance, and security.

Every project is different, so every testing strategy will be too. It is developed after analyzing the full range of potential problem areas in the solution, then the depth of testing is adjusted according to the complexity and impact of each portion.

Throughout execution, business process experts and software package experts work side by side to prepare and run the test scenarios. Pooling this knowledge is essential: it is what produces an optimal configuration of the package.

How should you structure test execution?

Test execution rests on three pillars: specialized test management tools, controlled IT environments, and dedicated technical support. Test management tools control the activities of each category, measure progress, manage scenarios and results, track anomaly resolution, and support status reporting.

Tests are run by the project team in controlled IT environments. PlanAxion recommends installing a total of eight environments in addition to the production environment.

Adequate technical support must also be planned for the full duration of testing. A team of experts in infrastructure, operating systems, databases, and software packages is dedicated to the project: it handles installations, administers the environments and their performance, applies the necessary patches, manages backups, and refreshes the environments as needed.

How should you manage test scenarios?

Test scenarios are a lasting asset: they are reused in later upgrade or enhancement projects, feed business process documentation, and provide the foundation for user training material. They should therefore be preserved and kept current.

Scenario preparation can be accelerated by drawing on a bank of predefined scenarios. Some software vendors and consulting firms hold sets of test scenarios in their intellectual capital and make them available to their clients.

As for test automation, the investment is not always worthwhile. The decision depends on three factors:

• the number of potential regression test cycles;
• the number of releases per year;
• the nature of the targeted tests.

Who should control configurations and test variables?

A central group, independent from the test teams, must manage changes to the package's configuration and settings. This control ensures that modifications are made in a controlled way and replicated uniformly and consistently across every environment. It also documents the differences between test environments and minimizes situations where no one can explain why the package behaves differently from one environment to another.

The purpose of the testing strategy is precisely to control every variable introduced during the various test categories, so anomalies can be isolated, diagnosed, and resolved more easily. The five variables to control are:

• Infrastructure: guarantee stable, high-performing, properly administered environments.
• Software packages: track the versions and patches applied in each environment.
• Configuration: replicate settings uniformly and consistently through a central group.
• Data: work with representative data sets and refreshed environments.
• Interfaces: verify exchanges with other systems without stray variables.

What should you remember to succeed?

A successful testing strategy combines test categories tailored to the project, controlled environments, reusable scenarios, and central configuration control. By concentrating effort where complexity and impact are greatest, and by having business experts and package experts collaborate, the organization improves and optimizes its solution before go-live instead of suffering its defects in production.

Frequently asked questions

What is an ERP testing strategy?

An ERP testing strategy is a structured approach that plans every test to run before a software package goes live. It strengthens risk management, targets the most complex portions of the solution, and lets the organization improve and optimize the package before deployment instead of discovering defects in production.

How many test categories does a large software package project include?

A large project generally includes 15 test categories, grouped into two broad classes. Solution tests validate the business processes and the configuration of the package. Technical certification tests confirm the reliability of the infrastructure, the performance of the environments, and the security of the overall solution.

How many environments do you need to test a software package?

PlanAxion recommends installing eight test environments in addition to the production environment. Controlled, regularly refreshed environments let the team run each test category under stable conditions, isolate anomalies quickly, and explain why the package behaves differently from one environment to another when discrepancies appear.

Should you automate software package testing?

Not always. The investment in test automation depends on the number of potential regression test cycles, the number of releases per year, and the nature of the targeted tests. A project that anticipates few regression cycles rarely justifies the initial automation effort and its ongoing maintenance.