By
When it comes to QA testing, not all approaches are created equal. QA for desktop applications is relatively straightforward when compared to the vast intricacies of mobile devices.
A basic survey of smartphones and other handheld devices will bring up dozens of sizes, system specs, capabilities and features. While for the consumer this means a few dollars more or less, a few more pixels or a couple more options, for the developers that are working endlessly to create a new mobile application or software, this could mean a logistical nightmare.
Unlike the conventional desktop operating systems, which are limited and have relatively few versions, mobile devices are a different issue entirely for developers.
The QA Challenge for Mobile Apps
Years ago, in order to properly test a product, companies sent someone to ‘buy one of everything’ to test their product across all devices. Fortunately for them, at the time there weren’t so many options. With exponentially more phones available today, the ability to truly test a product on each and every operating system out there is time consuming and unrealistic.
In addition to having multiple operating systems (Blackberry, Android, Apple, Windows Phone just to name a few), each operating system has multiple (often dozens) of versions, making QA across all devices a logistical nightmare.
A simple calculation of the number of operating systems, devices and software versions will yield a massive number of QA tests that would need to be run on each and every simple update. The increased fragmentation of available devices has make the QA process challenging for developers in the mobile industry.
Different Approaches to QA
In order to properly, and thoroughly, test out a new application or software, companies must select one QA approach and stick to it. The first decision a company has to make is deciding which operating systems and versions they will release their product for.
Companies, especially that are in the R&D phase of product development, can choose to limit the operating systems and/or versions that their product supports and thus limit the number of devices and system they would need to check. Selecting the most common and popular operating devices and the latest versions is often the best for companies that simply want to test viability and prove their concept. While this is a cost and time effective solution for the short run, it can potentially limit their client base and so may not be the best situation in the long run.
After determining which operating systems and versions to support, companies must decide if they want to use virtual simulators and emulators or properly test individual devices. As with limiting operating systems, virtual simulations and browser extensions are a viable option for companies in the early testing phase of their product, however in the long run, the accuracy of such emulators is not always up to date and thus may not give an accurate representation of the customer experience.
Some hardware vendors such as Samsung, Motorola and even Apple often loaner or rental programs for testing purposes. Different developer tools, some of which may even be free, can provide companies with a hybrid way to properly test their mobile device in a limited capacity.
The alternative to virtual software simulators is hard-testing an app on various devices and operating systems. Even here, companies have a choice – to perform the manual QA in house or to outsource. In-house testing is the best way to preserve the company data and intellectual property, however in order to properly do requires a large amount of manpower and resources.
External QA, particularly when outsourced to overseas companies, raises many IP and trust issues for app developers, however the alternative of in-house testing is often more costly and requires more resources and manpower than a company has – especially if they are still at the development stages.
As mobile apps and software continue to rise in popularity, so do the alternative solutions for QA testing. Today, companies that want to test their product for mobile devices can also choose to outsource their QA to a company that automates the process and repeats predetermined tests over and over on various devices and testing systems.
The Different Levels of QA
Companies in the mobile industry, particularly startups, often believe that their product must be perfect in order to release it as a PoC or beta. Fortunately, we’re here to shed light and announce that is not the case.
Companies that are in the PoC level do not need to undergo rigorous QA testing – they simply need to prove their product on the most popular devices. Once a company has successfully completed an enterprise level PoC, then it is time to launch an extensive QA process in order to enable multiple device access for end users.
As a startup grows and phases out from the PoC stage to the beta stage, soft launch and so on, they will have dozens of product versions and hundreds of opportunities to test the product for any and all possible bugs. A successful PoC does not focus on the functionality over hundreds of devices, but rather sets out to determine the UX/UI, feasibility and scalability of a particular product in the market.
