By, Roland Campbell, Director of Solutions Engineering, Usablenet
Effective UAT (User Acceptance Testing) without good test planning is like building a fancy new deck without developing the blueprint first. The first requirement for performing great QA is to develop high quality test cases. Typical unit testing is not enough to ensure that granular, context-specific UAT is done to assure the quality of every aspect of the digital experience you are getting ready to launch.
Let’s take a closer look at why test plans matter.
A test case is a set of step-by-step instructions required to complete a task, and is to QA what unit tests are to development. Just writing down the steps provide valuable insight into your application. By performing the task and writing down the steps, you might find gaps in your design (meaning a user can’t get to that action from here!), and also discover UX issues. The latter will prevent users from having to take unnecessary steps to complete an action (you have to WHAT to check out?).
Once the steps are written down, and the intended outcome of the test is determined, the QA team can start to test your application. Every major flow should be documented and both happy and sad paths included. Your QA team can then simply execute on the test cases and ensure the basic quality of the release. The QA team will be more efficient, and as such, will have more time to work on edge cases and real user simulation to ensure true end-to-end quality, both in functionality and UX.
In addition, test cases provide the basis for QA automation, as you need to be able to program your automation tool in a stepwise fashion. Great test cases provide those steps.
Test cases should not be static. As new features are released, bugs are found and the UI is changed, your test cases need to keep up. You also need to ensure that test cases are neither too granular, or not granular enough. Basically, you need to hit the Goldilocks version, just right. Too granular will produce excessively sized reports and take extra time. Not granular enough will skip steps and miss problems.
Comcast, a US-based media and technology company, is the provider of a range of on-demand viewing applications, including Xfinity. Usablenet creates test cases to perform full regression tests of the Xfinity website across all channels. (A regression test is a type of testing to ensure that software previously developed and tested still performs correctly after it was changed or interfaced with other software.) Xfinity is a dynamic offering with frequent changes that may include software enhancements, patches, configuration changes, etc. Xfinity currently produces a set of test cases to create a “smoke test”. (A smoke test is preliminary testing that will reveal simple errors that are critical enough to stop a release.) In today’s fast paced, minimally viable product world, the smoke tests enable Comcast to be agile and keep pace. Over the past two years, Usablenet has created over 100,000 test cases across many types of applications, devices and browsers. We build great test plans and Comcast draws on this expertise. We have found that the best kind of test plans are those with the right granularity. Further, these plans speed up the testing process and are robust because they have proven processes that ensure test cases stay up to date.
Remember, your UAT is only as good as your test plan.