The Importance of Maintenance testing

AKA, What happens Once software is live?

My last two blog pieces looked at software testing during the development phase. Now it is time to focus upon what happens once software is live. If you think that testing is no longer necessary, perhaps this article will change your mind!

Software testing does not end when a piece of software is developed and installed. On the contrary, it must be maintained to keep the quality standards high and to apply potential improvements.

We all have regular software updates on our computers, phones, and apps. Many of us have experienced downtime on certain websites. This is the result of maintenance work which is used to fix bugs and to introduce new features.

The objective of maintenance testing is slightly different from what it was during the development phase. Maintenance testing is performed to ensure the quality target achieved earlier is maintained when further changes are made.

Remember, there is no such thing as a bug-free system!

Bugs are sometimes identified after the go-live period or after software has been used for a while. A lot of the time, decisions are made to change the existing functionality and add new features to improve the overall quality. There is also the chance that software might need to be migrated from one platform to another. All these occurrences will trigger the need for testing, to ensure that the systems continue to work as expected.

Maintenance testing is sometimes even more critical than testing during development. This is because the software is already live and so any changes must be properly controlled before the update is rolled out.

Project resource is usually reduced after go-live, with any changes made to the software from that point on being ‘owned’ by the client. That often means the new changes are not managed to the same standard as those during the development phase. This in turn can lead to a noticeable drop-off in quality over time; how then can we ensure that the quality standard remains high and that the software continues to improve?

Here are a few tips:

• Priority and Scoping

Setting the priority and scope of what will be fixed and implemented are essential tasks to identify the effort and resources required over the short and long term. For testing, this enables you to identify the level and type of testing required so that you can estimate the test effort accordingly.

Understanding the scope of the changes also allows for deeper analysis of the impact upon the software. This means the test team can identify areas that may be affected with those planned changes. During this time, the test team can review the test cases and either apply modifications or make new additions where necessary.

This will ensure that the testing is planned and executed efficiently and correctly.

• Planning and scheduling releases (while being mindful of unscheduled releases)

System vendors may have set internal schedules for the release of a new version of their software. This does not mean you need to upgrade the software in your environment each time, however.

Planning and scheduling releases with vendors ensures that the list of items related to your business are released in a controlled manner. Testing can then be scheduled in accordance with those release schedules.

It is important to note that unscheduled releases will often need to be factored in. By their very nature these are difficult to plan for, but awareness is key.

• Scaling the test effort

“How much testing is required?”

This is a very common question, and the answer depends on the planned releases and the impact they may have upon the existing software.

We cannot test the entire system every single time, and nor do we need to. Testing must therefore focus on the areas that are likely to be affected by the changes. You should also plan to run the whole regression suite on a regular basis. Having the ability to scale the testing effort is a way of making the testing more efficient, thereby reducing unnecessary costs.

Even once software has been developed and set live, testing continues to play an important role. It can be used to measure the quality of the software as it continues to evolve. Testing is a way to provide feedback on performance, so that plans can be made, and priorities reassessed to achieve your end target.

Testing is an area that is often undervalued. There are, however, great reasons why it is important. I hope that this article and the previous two have demonstrated that. I also hope that you can see how testing can provide value to the overall quality of software. I believe that as technology continues to develop and change, testing will remain vital. Testing techniques and approaches will also evolve so that testers can continue to provide crucial information used to measure software quality against business expectations.