Today, mobile app quality matters the most as the market is very competitive. We need to focus on user experience as well as the performance of the app. If the app contains bugs, the app ratings will go down, and so will the number of app downloads. If we can make our app bug free, the chances of the app gaining popularity will increase. This can be achieved by performing different types of mobile app testing.
Before we start talking about different types of mobile testing, let's recall what testing is.
Mobile app testing is the process of verifying that an app meets its stated requirements and actual functionality by observing how it works in artificially created situations and using a limited set of tests selected in a certain way.
In simple terms, this is a process of examining and testing mobile apps with two main goals:
- Identify situations when the app behavior is incorrect;
- Check that the app sufficiently meets the requirements and customer needs.
Testing is an integral and essential part of app development. In a broad sense, testing is one of the quality control techniques that include:
- Planning;
- Creating tests;
- Testing and analyzing the results.
It is important to understand that testing includes testing itself and other actions related to the quality assurance process:
- Analysis and planning;
- Development of test scenarios;
- Evaluation of test completion criteria;
- Report writing;
- Reviewing documentation (including source code);
- Performing static analysis.
Types of mobile app testing
All mobile app testing types, depending on the pursued goals, are divided into the following groups:
- Functional;
- Non-functional;
- Changes-related.
Next, we will describe each type of testing in detail, its purpose, and its use when testing mobile applications.
Functional testing
Functional tests are based on functions and features, as well as interaction with other systems, and can be conducted at all levels of testing:
- Component / Unit testing;
- Integration testing;
- System testing;
- Acceptance testing.
Functional types of testing consider the external behavior of the system. The following are some of the most common types of functional tests:
- Functional testing;
- Security and access control testing;
- Interoperability testing.
Functional testing
Mobile functional testing is a type of testing aimed at checking whether the app's functional requirements meet its actual characteristics.
The main task of functional testing is to confirm that the product under development has all the functionality required by the customer.
Like any other testing type, functional testing has its advantages and disadvantages:
Advantages of functional testing:
- It simulates actual system usage.
Disadvantages of functional testing:
- The possibility of missing logical errors in the application;
- Probability of excessive testing.
Security and access control testing
A security testing strategy is aimed at verifying the security of the system and ensuring an all-encompassing approach when it comes to protecting the application, preventing hacker attacks, viruses, and unauthorized access to confidential data.
The overall security strategy is based on three main principles:
- Privacy policy;
- Integrity;
- Availability.
Interoperability testing
Interoperability testing is functional testing that checks an application's ability to interact with one or more components or systems and includes compatibility testing and integration testing.
With the development of network technologies and the Internet, the interaction of different operating systems, services, and applications has acquired significant relevance since any related problems can lead to a decline in the company's popularity, which will entail financial losses. Therefore, the test of interaction should be approached with the utmost seriousness.
Also, it is important to test a mobile application on multiple devices (if it's intended for multi-platform use) to ensure that it delivers a consistently good performance on all of them.
Related: Your Step-by-Step Mobile Application Testing Process [+Mobile App Testing Process Checklist]
Non-functional testing
Non-functional testing describes the tests required to determine an application's characteristics that can be evaluated in different ways. In general, this is testing how the mobile application works.
The main types of non-functional tests are listed below:
- Performance testing:
- Performance and load testing;
- Stress testing;
- Stability / Reliability testing;
- Volume testing.
- Installation testing;
- Usability testing;
- Failover and recovery testing;
- Configuration testing.
Let's talk in more detail about types of non-functional testing:
Performance and load testing
Load testing or performance testing is automated testing that simulates a certain number of business users on a shared resource.
The task of performance testing is to determine the scalability of the application for varying levels of load. Here is what occurs:
- Measurement of the execution times of selected operations at specific levels of intensity;
- Determining the number of users working with the app at the same time;
- Defining the limits of acceptable performance when the load increases (when the intensity of the operations increases);
- Performance analysis at high, extreme, and stressful loads.
Related: An Introduction To Server Testing
Stress testing
Stress testing allows you to check whether the application and the whole system are working under stress and assess the system's ability to regenerate, i.e., to return to normal after the stress has ended.
Stress in this context can be an increase in operations' intensity to very high values or an emergency change in the server configuration. One of the tasks of stress testing may also be to evaluate performance degradation, so the goals of stress testing may overlap with the goals of performance testing.
Stability / Reliability testing
The task of stability (reliability) testing is to check the application's performance during extended usage with an average load level. Operation execution times may play a secondary role in this type of testing. Preference is given to the absence of memory leaks, server restarts under load, and other elements that affect the stability of the operation.
Volume testing
The goal of volume testing is to get a performance score when increasing the amount of data in the application database. Here is what occurs:
- Measurement of the execution times of selected operations at individual intensities of these operations;
- The maximum number of users working with the application at the same time can be determined.
Installation testing
Installation testing is aimed at verifying the successful installation and configuration of the application as well as updating or deleting the application.
Usability testing
Usability testing is a method of defining the degree of usability, trainability, comprehensibility, and attractiveness for users of the developing product under given conditions.
Sometimes we encounter incomprehensible, illogical applications, many of whose functions and uses are not obvious. In such cases, the user would rarely turn to the app again and would sooner look for more useful analogs.
For the app to be popular, it is not enough for it to be functional – it must also be convenient. If you think about it, intuitive applications save the user effort and the employer–training costs while also making your app emerge from the pack. Therefore, usability testing, which will be discussed later, is an integral part of testing any mass-market products.
Failure and recovery testing
This type of testing analyzes the product’s ability to withstand and successfully recover from possible failures caused by application errors, hardware failures, or communication issues (for example, network failure).
The purpose of this type of testing is to check the recovery systems (or the ability to duplicate the main functionality of the systems), which, in the event of failures, will ensure the safety and integrity of the data of the tested product.
Failure and recovery testing is significant for systems that work according to the '24x7' principle. Suppose you are creating a product that will work on the Internet. In this case, you simply can't do without this type of testing because every minute of downtime or loss of data in the event of hardware failure can cost you money, loss of customers and reputation in the market.
For clarity, let's look at some variations of such testing and general methods of their implementation. The object of testing in most cases is very likely operational problems, such as:
- Power failure on the server computer;
- Power failure on the client’s computer;
- Incomplete data processing cycles (data filters are interrupted, synchronization is interrupted);
- The announcement or introduction of arrays of data is not possible or erroneous;
- The failure of the storage media;
- Configuration testing.
Configuration testing is a particular type of testing aimed at checking app performance under various system configurations (declared platforms, supported drivers, different computer configurations, etc.).
Before starting configuration testing, we recommend that you:
- Create a coverage matrix (the coverage matrix is a table where all possible configurations are entered);
- Prioritize configurations (in practice, most likely, you will not be able to check all the desired settings);
- Step by step, review each arrangement in the order of priority.
Changes-related testing
After making the necessary changes, such as fixing a bug/defect, the app should be re-tested to confirm that the problem has actually been resolved.
The following types of testing must be performed after installing the application to confirm that the application is working properly or that a defect has been corrected accurately:
- Smoke testing;
- Regression testing;
- Build verification testing;
- Sanity testing.
Smoke testing
The concept of smoke testing came from the engineering environment:
"When commissioning new equipment ("hardware"), it was considered that the test was successful if the installation did not cause the equipment to release smoke."
Smoke testing is considered a short cycle of tests performed to confirm that after building the code (new or fixed), the installed application starts successfully and completes its primary functions.
Regression testing
Regression testing aims to verify changes made to an application or environment (fixing a defect, merging code, migrating to another operating system, database, web server, or application server) to confirm that the previously existing functionality is working as before.
Regression tests can be either functional or non-functional.
The term "regression testing" itself may have different meanings depending on the context of use. Cem Kaner, for example, described three main types of regression testing:
- Bug regression is an attempt to prove that a fixed bug is not actually fixed.
- Old bugs regression is an effort to prove that a recent change in the code or data broke the correction of old errors, i.e., old bugs began to reproduce again.
- Side effect regression is an attempt to prove that a recent change in code or data broke other parts of the application being developed.
Build verification testing
This type of testing is aimed at determining whether the released version of the app meets the quality criteria in order to start testing.
In its purposes, it is an analog of smoke testing aimed at accepting the new version for further testing or operation. It can go deeper, depending on the quality requirements of the released version.
Sanity testing
Sanity testing is narrowly focused testing sufficient to prove that a particular function is working according to the requirements specification. It is a subset of regression testing.
It is used to determine the state of a specific part of the application after changes made to it or the environment change, and it is usually performed in the form of manual testing.
Some sources mistakenly believe that sanity and smoke testing are the same things. We are sure that these types of testing have very different goals. In contrast to smoke testing, sanity testing is directed only at the function being tested. Smoke testing is directed at as much functionality as possible in the shortest time possible.
Summing up
As you’ve learned in this article, there are many things to keep in mind when using various mobile testing types.
Consequently, it’s easy to forget something or not to pay enough attention to specific areas. For this reason, we strongly recommend that you plan your mobile app testing process thoroughly.
In addition to the right mindset, the right tools and strategу, you also need the right team for testing! If you're ready to use our testing services to make your app a success, contact us today to discuss your project. We check your application on a wide range of real mobile devices and always take that extra step toward ultimate quality.
