Welcome to your QA cycle! At Sourcetoad, we take QA very seriously. It’s the last step between development and the release of a new product. Our QA teams are very thorough, however, the better the instructions that we provide to them, the better the QA cycle can be. Please use the following instructions to help us work with you to develop a robust QA plan.
Step 1: Identify Test Environments
When referring to QA, an “environment” is a place where users will utilize the product at hand; such as an Operating System, a Browser, or other piece of software. Our QA team can test any environment needed. Generally, it’s a good idea to identify the oldest environment we’d like to support, and then move forward from there. For most web-based products, the following examples would be considered a good testing environment:
Operating Systems
• Mac 10.5 and higher
• Windows XP SP3 and higher
Browsers
• IE 8 and higher
• Chrome (latest)
• Safari (latest)
• Firefox (latest)
For phone and tablet apps, the following examples would be a good test environment, depending upon if the app supports both Android and iOS:
Operating Systems
• Android 4 and higher
• iOS 6 and higher
Devices
• iPad 2 and higher
• iPad Mini
• iPhone 4s and higher
• Various phones and tablets running Android 4+ from 2011+
The main factor to consider is if the product has any special needs that are uncommon in which the software or product must support. Those must be included in the test environments.
Step 2: Create Testing Focus
The testing focus is the most important part of the QA cycle. This is where you will inform the QA teams as to what to look for when testing the product. Our teams can perform two types of testing: exploratory testing and case-based testing. Case-based testing means you’d first define specific use cases for the product as well as the expected results. The teams will run through these use cases and deviations from the expected results. This requires you to define specific use cases to consider for the product. Exploratory testing is much simpler to define: This is where you’d give a general definition of the product and how it works, then allow the QA teams to interact with the product in a relatively unguided fashion. They will try out as many aspects and features as they can, and use them in different ways.
All products require exploratory testing, and we recommend this to be a part of any initial QA cycle. For an exploratory test, some of the things you will create include, a brief explanation of the product, as well as the user roles and how they’re expected to work together. For example, the following excerpt would be a good exploratory test explanation for a restaurant website with online ordering:
PhilsRestaurant.com is a website for Phil’s restaurant in Boston. The website features information about the restaurant such as photo galleries, directions and phone numbers. It also has contact forms for users to get in touch with the restaurant.
A major section of the website is its’ online ordering system. This system allows users to create an account, browse through the restaurant’s menu, choose items, and place orders. They can also reorder, choose pickup times, etc. Once orders are placed, they are routed to the restaurant through a backend manager interface.
Testers should both create user accounts and use the back-end manager interface to ensure orders are properly handled and routed. Please pay special attention to portion sizing options when ordering, as that has been an issue in the past.
Beyond exploratory testing, complicated products should include case-based testing. If the product is very complex, exploratory testing alone might not suffice in regards to examining important individual use cases closely. Since the testers do not know what use cases are most important, case-based testing helps define them. You do not want to include so many cases that QA teams do not have enough time to adequately test each one. Really, the few important ones should be the main focus. If you are unsure, don’t hesitate to ask us.
The case-based testing will include a step-by-step case definition as well as expected outcomes for each step. Be specific, but do not be so specific that you restrict the testers to only examine the system in one particular way, leaving potential issues unspotted. For example, this would be a good case definition for a game where a user must complete a specific quest:
Step 1: User moves his game character into the town at the top edge of the map.
Expected Outcome: A wizard approaches the user and asks if he would like to go on a quest to Narnia.
Step 2: User either chooses “yes” or “no”
Expected Outcome: If the user chooses “yes”, he is transported to Narnia. If the user chooses “no”, he is offered extra gold to complete the quest. If he still chooses “no”, the wizard disappears.
Step 3: Once user arrives in Narnia, he must continue exploring until he finds the first clue.
Expected Outcome: If the user cannot find the first clue within 5 minutes, he is given a hint from a troll. Once he finds the clue, the wizard re-appears. The Wizard reads the user the clue, and informs the user that he has 10 minutes to solve a puzzle.
Step 4: User attempts to solve puzzle.
Expected Outcome: If the user does not solve the puzzle within 5 minutes, the first part of the puzzle solves itself as a hint. If the user does not solve the puzzle within 8 minutes, the wizard reappears with an additional final hint. If the user does not solve the puzzle within 9 minutes, a countdown appears on the screen.
Step 5: User either solves the puzzle or does not.
Expected Outcome: If the user does not solve the puzzle, the user is transported back to the town at the edge of the map. If the user does solve the puzzle, the wizard appears, gives the user his reward, and then transports the user back to the town at the edge of the map.
Step 3: Identify Bug Values
Bug values allow the QA team to know how important different types of bugs are. There are two different types of values. The first is “Very or Exceptionally Valuable.” These are the bugs most vital to the product, and the ones that need to be fixed as soon as possible. In general, these are bugs that prevent the product from functioning in a major way. The second is “Somewhat Valuable.” These are the bugs that are somewhat important to the product, but aren’t necessarily blocking the product’s main functionality. For example, this would be a good bug value definition for a restaurant website with an online ordering system:
The following issue types will be considered Very or Exceptionally Valuable:
Unexpected behavior related to pricing, adding items to cart, or credit card. Also issues related to orders properly appearing in the management interface.
The following issue types will be considered Somewhat Valuable:
Issues which users would be able to work around easily
