What to do before you sign off on your MVP
How to do user acceptance testing (UAT).
By Lenka Davis
How will you know if your software is working as expected? How will you avoid the embarrassment of your customers finding issues with the software application or the service you just launched? And, how will you know if the description you wrote in your requirements and the user stories were understood by the developers as you envisioned it all to work?
Of course, you trust your team to build your product to your specifications. You probably have had many conversations with them, maybe daily check-in meetings and lots of reviews about how small details are being added or removed from the product. With all of this happening you still want to “test drive” the finished product before you release it to the world.
This test drive is referred to as User Acceptance Testing or UAT.
What is User Acceptance Testing?
User Acceptance Testing (UAT), sometimes called beta testing or end-user testing, is a phase of software development in which the software is tested in the "real world" by the intended audience or business representative. - Stanford University IT Department
It’s an endurance test, per se, and you want to test the limits of the software or website to know what the boundaries are of its capabilities.
What is User Acceptance Testing for?
It is testing “To ensure a high level of functionality, usability, and alignment with real-world scenarios.” - UserSnap.com. It is also to make sure that the key feature or features are working correctly so when the product is used for a demo to a potential client or VC it all works as intended.
What would happen if you did not test the software!?
In a recent example, a team completed an MVP and set up a user testing schedule. The product owner did not realize the significance of testing key business features, and that resulted in the partially build features being found during a dry-run demo session right before a real customer demo.
Who is UAT performed by?
“UAT type of testing is not intended to be menu-driven, but rather to be performed by business users to verify that the application will meet the needs of the end-user, with scenarios and data representative of actual usage in the field.” - Stanford University IT Department
Before UAT starts the developers will have completed QA (Quality Assurance) testing. UAT is performed by anyone on the team that will be using any part of the product or someone that relies on the product to work in order to do their job.
Tester can be:
Product Owner
Business Unit Manager
Administrator
Business Analyst
Project Manager
And other users needing to work with the software
What should business users expect to do during a UAT session?
What is expected during the UAT is for the testers to use the application as they would for actual use cases. If they find any problems then a written description, screenshot or video capture of any issues found during the testing is shared with the developers. This is to be added into the Jira test case or spreadsheet that the team set up to track the test cases and the resulting issues.
Especially if you have creative or non-technical business users you need to let the testers know what is expected of them from doing the testing. We have found it is highly recommended that the product manager, the project manager along with one or more developers be at the first sessions with the business users. This eliminates testers from getting stuck immediately when beginning the tests.
Also, testers may be asked to provide their computer system, OS version, browser name and browser version so that the developers can recreate the issue in the correct environment.
When do business users perform UAT?
Most often it is done once the software is developed, released into the testing environment and the developer teams have run their own array of QA tests on the software and environment. Once that is done then the business users can begin to test the application.
How to run user acceptance tests
First let the individuals that will be doing the testing know what UAT is and when they will be expected to do it. You can give them a short presentation or explain it to them in a meeting before testing starts. Steps to do to set up User Acceptance Testing for the testers:
Set up User Test Cases
Get test user accounts
Meet with testers either one-on-one or in small groups to walk them through what to expect
Ideally you want to have the test environment set up by the developers, and you have the test cases written. Then, book your business testers for a few sessions on their calendar. Being available during the testing sessions helps reduce a delay and makes the testing process go smoothly.
For very large teams or major releases of software a schedule could include:
1 day to do training and set expectations with the testers
1 to 2 weeks of full day or half day sessions where testers are in a separate area from their office so they can focus on testing.
The first day or two it is highly recommended that someone from the development team be present or on call for the testers
Daily review meetings with the developers to explain what issues were found.
What should be in user test cases?
The test cases mainly have four sections that help the tester get through the test quickly. The four sections are:
A description of what the user needs to do
The test case steps
The expected results
The actual results
The description gives the tester what task they need to complete in a short sentence or two.
The test case steps include the URL where the MVP is staged along with the testing username and password. Other steps you can include are:
Check each test case described in the expected results section
Write down any results in the last section of this ticket. Results can be anything you notice such as:
you can’t complete the task
you find something confusing in the user interface
you get an error message
Attach screenshots or a short video
Include what type of computer you are on: Mac or Windows, etc. What browser you are using: Chrome, Safari, etc., plus the browser version.
The expected results are a more detailed description of what the tester can accomplish. For example, if you are testing a brand new user login page, you will want to test creating an account, and logging in with an existing account. You will also want to be able to change your password.
After all the hard work in defining the product, and then building it, it is rewarding to test the final result. One byproduct of testing is that the users think of new and innovative features that can be added to future releases.
FunFact
In 2023, 1-800-PACK-RAT, a moving company, created a new quote test page. Initially they wanted to have a long multi-page form but after testing this, and listening to the feedback from their testers they moved to a single-page no-nonsense form. Good thing they did that because their conversion rates jumped by 10% after launching the simplified form.
