Sanity Testing Tutorial: A Comprehensive Guide

Sanity testing run on the stable build of an application. In this tutorial, readers will learn about sanity testing, its features, and best practices.

Sanity testing is a type of software testing executed on the stable build of an application with minor code changes. It is also known as surface-level testing. A sanity test is performed to ensure the application works as expected after the bug fixes or changes in the code.

In the Software Development Life Cycle, software testing is an integral part of ensuring the performance of the software application. It is performed to find any bug or error that could lead to the failure of the software application. Such failure could be costly for the organization as it could not provide the expected user experience. Hence, it is suggested to check everything before the release of the software application in the market.

There are many different types or approaches to software testing, each of which values its purpose. Among those, sanity testing is a part that is executed to ensure the quality of software applications in less time. A quick check of the basic functionality of a software build is done to ensure its functionality.

In most test scenarios, sanity tests are run post-software build to validate the application’s functionality with minor code changes.

What Is Sanity Testing?

Sanity testing is a technique to determine whether new bugs corresponding to adding new features or functionality in software applications are fixed. It is the basic or surface-level test method to determine if a software application or its specific component functions correctly.

It is regarded as the subset of regression testing. Usually, the sanity test is done after the completion of regression testing to fix the bug and error. Hence, it aims to test a software application component to check whether it works after adding new changes. Unlike regression testing, it only finds any dependent missing software application functionality impacted by code changes.

Some of the key points on the sanity test are as follows:

• In the sanity test, testers and developers verify the commands and functions of the software.
• Executed when we needed to complete testing in a short time.
• It is a type of non-scripted testing.
• It is a quick and short testing method that ensures changes made are according to the user’s expectations and specifications.
• Checks minor bug fixes.
• Checks the basic functionality of the software.
• Can be executed when a software application undergoes a major functionality overhaul.
• Executed when a software application undergoes some crucial modification.

Why Sanity Testing?

You may think if we already perform end-to-end testing or execute other types of tests, which check each functionality of the software application, then why do we need sanity tests to be done? The answer to this is very simple. You should understand that every software testing method is needed as per the requirement. Likewise, sanity testing is important to check the new and crucial functionality of the software application.

Here, we will understand why we should not neglect or avoid doing sanity testing:

• Save lots of time: Sanity testing determines thorough testing of software applications. In a situation when sanity testing fails, there is no need to perform any rigorous testing, like end-to-end testing or regression testing. Thus, this saves huge time on testing as errors reported in a software application are quickly sent to the developers to resolve in case of low software quality.
• Quick evaluation: It takes less time to execute the test process. It is narrow-focused on specific functionality or features that require testing. Further, we do not require test scripts and cases to perform sanity testing. Instead, we use an unplanned intuitive approach to conduct a sanity test, which quickly evaluates a software application.
• Alleviate non-required effort: We can save a lot of effort in the testing software application by performing sanity testing. It helps determine whether we are required to test the software any further or not. Additionally, testers are not required to report the issues as they are directly sent to developers to solve.
• Improve regression testing: It helps testers verify the critical error in the application post changes. Regression testing involves re-executing a set of test cases to ensure the application’s accurate working after the incorporated changes. However, with sanity testing, we can perform quick and superficial checks on the important features of software applications. Thus, testers quickly identify the major issues without checking the entire suite of test cases. Hence, the sanity test is a supplement to the retesting process.
• Find compilation and deployment issues: Let’s understand this with an example. Imagine developers failed to compile the build, including all resource files, which could cause an unfriendly user interface. The development team may forget to add any deployment notes. This can lead to non-working and non-loading of the released software application in a test environment. With sanity tests, we can identify and resolve such issues quickly.
• Provide a quick state of software release: Sanity tests give the state of software release, which allows future planning of tests efficiently. For example, if the sanity test fails, the next task on the application development is postponed, and flaws in software release are fixed as a priority.

Example of Sanity Test

The sanity test quickly evaluates the quality of the software before its release and checks whether there is a need to conduct any further tests. But, a sanity test requires crucial understanding practically, which will help you gain greater insight into its application or use. Let’s see this with different examples:

• Technical example: In the software development process, the application comprises multiple code files, and its integration is hectic. Hence, developers develop executable files (software build) by compiling all such code files. This build is shared with the testing team to test its functionality. However, imagine an organization needs the software in a short period; we follow a quick testing method rather than giving a non-tested application. Here, we do a sanity test of the application.
• Non-technical example: Suppose you are developing a mobile application and encounter a payment checkout error. The testing team reported this error to the developing team to resolve it quickly. The development team fixed the error and sent it to the testing team for a fix. They now check for the application’s working based on the changes made.

Further, they also check whether changes impacted the other related functionalities. Now, there is a functionality to make payments on the payment login page. Part of the sanity test involves validating the payment login page to ensure its functioning is in place.

From the above-stated example, it can be said that the sanity test only checks for the modified or defective components of the software application. It does not check end-to-end functions.

Goals and Priorities of Sanity Testing

Sanity tests allow developers and testers to validate their use in the Software Development Life Cycle. In line with this, here are two critical priorities of the sanity test:

1.     Sanity test cases use standard functionality, delivering high software project value to enhance user experience.

2.     Sanity test cases are selected based on regression tests to fulfill the Software Requirement Specification (SRS).

Apart from its set priority, the sanity test also has specific goals that developers and testers must ensure. Some of those goals are as follows:

• To define the expected working of upcoming features.
• To save time and cost when the sanity test fails for a build.
• To ensure new updates do not lead to any changes in the existing function of the software.
• To validate the reliability of newly added features.

Now deep dive into more details on the sanity test and understand this with examples.

When To Perform Sanity Testing

The sanity test focuses on one or more important functions in the testing software applications. It needs to be performed in different scenarios. Some of those are as follows:

• In a situation when small changes in the application’s code are made.
• When new features are added and ready to integrate into the software application.
• After a number of regressions tests are done, and the new build is generated.
• Post bug fixes.
• Prior to production deployment.
• Before integrating features in the software application.

However, how often should we perform a sanity test in a Software Development Life Cycle? Well, it depends on your requirement and the complexity of the software application. As we have addressed the goal and priority of the sanity test, we will now understand the frequency of performing the sanity test.

Sanity Test Frequency

The frequency to perform sanity tests is based on the Software Requirement Specification (SRS) and software development stage. Imagine you are in the software development team aiming to develop an application that performs as per user expectations.

However, a report by Statista indicates that mobile application revenue will increase and reach $613 billion by 2025.

Source: Statista

You are aware of the high revenue growth and highly competitive market. This necessitates testing the new software application to ensure its proper functioning and no error on its release. Thus, you should perform sanity tests during the early stages of software development to ensure it works post-new build. This needs to be continued several times as you improve or edit the code or add new features to the software application.

A sanity test is mainly executed to verify whether the added functionality does not give errors in existing features. Further, it also validates the system is in a “sane” state for moving ahead with future testing.

As the software application becomes stable, the number of times the sanity test should be performed decreases. However, it should be done whenever you need to perform sanity tests. This is only to fulfill the goals mentioned earlier.

In a nutshell, the frequency with which sanity tests are performed is determined by the specific need of the software application, the stage of software development, and the number of time changes made to the application.

You may ask why we need to perform a sanity test for a software application. It is a crucial part of testing software applications and ensuring their quality. However, its significance is not just limited to this.

Challenges and Solutions of Sanity Testing

It is crucial to note that not all software testing types have benefits and importance. It also holds some limitations and drawbacks, which are important to consider. Similarly, the sanity test is not an exception. It also has some challenges, which every tester and developer should know while performing. This is because such challenges do not hamper the Software Development Life Cycle.

A sanity test can be a helpful approach to get new functionality checked in less time. However, specific challenges with sanity tests cannot be ignored. Some of those challenges are:

• Limited scope: The sanity test cannot find all the potential issues in the software application. This often leads to false positives, which mean the application seems to be working accurately, but actually, it is broken or failed in other test environments.
• Incomplete testing: The sanity test only performs a preliminary analysis of software applications.
• Time constraint: In most situations, the execution of the sanity test is done under tight time constraints. This makes it difficult to test the application correctly.
• Limited test environment: With sanity tests, covering all test cases in all test scenarios is impossible. When we say there is a limited test environment in a sanity test, it means we can execute only a few selected test cases on a particular subset of the software application.

·         Test scope: The test’s scope is limited to a specific area or feature of the software, and the sanity test is performed in a controlled environment, which may not represent the actual software development environment.

Solutions to overcome the challenges of the sanity test:

To address the above few challenges, you can follow the below-given approach:

• Always create a test plan and check the Software Requirement Specification (SRS) of applications before performing sanity testing.
• You should check that the sanity test is performed within the context of a test strategy. The approach to this is very easy. You can combine rigorous testing with the sanity test or include regression testing to ensure the accurate functioning of the application.
• You can document the testing process and share the identified bug or error with the development team. This will help address the issue before the software application is released.

However, the sanity test is often used interchangeably with smoke testing. This can create confusion in understanding the solution to overcome the challenges of the sanity tests. The reason is that without knowing the core difference between the smoke test and the sanity test, you won’t be able to perform it efficiently and address its challenges. So let us learn the core difference between sanity and smoke test from the below section.

Sanity Testing and Smoke Testing Differences

Most of the time, we are confused between smoke and sanity tests. Smoke and sanity tests quickly check a software application’s core function. They seem similar but differ in scope, objectives, and timing to perform tests within the Software Testing Life Cycle. To address this, let’s dive into the clear difference between sanity and smoke tests:

Sanity and smoke tests are crucial to develop a software application project. Which one should be executed first? Its answer is simple. The smoke test is executed first and then proceeds to the sanity test. However, their position in the Software Development Life Cycle is different.

Role of Sanity Testing in the SDLC

Sanity testing takes place in the testing phase of the Software Development Life Cycle, which follows a deployment phase. Under the testing phase, it is a part of functional software testing and is executed after minor changes or bug fixes in the software build.

A sanity test is mainly done after unit, integration, and system testing. As the software application passes an initial phase of testing, a sanity test is executed to validate whether changes or fixes made in the application caused any new issues or bugs.

On completion of the sanity test, acceptance testing occurs, where the end-user tests whether it meets the Software Requirement Specification (SRS).

Aspects To Analyze in Sanity Testing

The sanity test is known to test the specific functions of the application. However, other aspects of software applications are tested in sanity testing. Some of those include the following:

• Basic function: The tester performs a sanity test to check the basic working features of the application as per user expectations. For example, if you want to verify the launch of the application, you should check the functionality of the user interface, like button navigation, etc. In such a test scenario, sanity testing evaluates those with accuracy.
• Data integrity: Testers verify the provided data’s accuracy and retrieval in the software application.
• Security: Sanity testing checks the basic features like data encryption, user authentication, and access control of the software application.
• Compatibility: Sanity tests also check for the compatibility of the software application across different browsers, devices, and OS configurations.
• Performance: Sanity tests validate the application’s performance under complex environments.
• Error handling: It refers to how the software application handles errors and exceptions during its execution. Sanity test checks for the error-handling capabilities of the software application by intentionally introducing errors or exceptions in the system and verifying the application can handle them properly.
• Install and uninstall: The sanity test checks whether an application can be installed. This involves ensuring a smooth installation process and how software runs after installation.

Sanity Testing Process

To determine the critical functionalities of the software application, a sanity test follows three steps, which are as follows:

1.     Identification: In the first step, new functionality and features are identified. Additionally, while fixing bugs, new modifications presented in the code are found. This shows we are required to determine the scope of the sanity test in terms of changes to be made in the software application.

2.     Evaluation: In the next step, you have to evaluate and modify the new components of the software application to ensure their functioning is in line with the Software Requirements Specification (SRS).

3.     Testing: The final step is testing, where a small part of the newly associated components of the application is tested in-depth.

Ways To Perform Sanity Testing

We have two approaches to practically implement the above steps: manual testing and automation testing. Choosing the type of approach depends on the requirement or specification of the application or the bugs identified.

Sanity tests can be executed with manual and automated approaches. However, both have their specific advantages and disadvantages. Let’s see each of them:

Manual Sanity Testing

In manual sanity testing, critical test cases are executed to validate the software’s stability and functionality. Such an approach is helpful in case of limited resources and time for testing, and changes done are minor.

How To Perform

The steps for a manual sanity test are mentioned below:

1.     Analyze the test cases to ensure the expected working of the software application.

2.     Create test cases that cover crucial functionality of the software to be tested.

3.     Evaluate the outcome of the test cases to find whether the sanity test is passed. In case the sanity test fails, it is marked as unstable, and then we are required to conduct further tests.

4.     The result is then shared with the development team to address the identified bug or error during testing.

5.     On fixing the error or bug, a re-test of the application is done to ensure its stability.

It should be noted that a manual approach requires highly-skilled testers with good knowledge of software and its functionality.

Automated Sanity Testing

Automated sanity tests involve using automated testing tools and frameworks to verify the basic function of the application. It includes creating automated test cases that run automatically in the application every time a new build or release is developed.

You can prefer the automation method to perform sanity tests for complex applications, which demands significant testing to ensure its reliability and stability.

It is important to note that sanity tests are more often automated as they are a subset of regression tests. It helps save time and resources in the testing process, along with maintaining the quality of the software application.

How To Perform

You can follow below given general steps to perform an automated sanity test:

1.     Define the software application’s functionalities, which require testing.

2.     Now, create test cases that include the defined functionality of the software application.

3.     Select a test execution tool that automates the test cases created by you based on the functionality. Some available tools include Selenium, Playwright, Appium, etc.

4.     You need to set up a test environment with proper configuration. In other words, ensure the availability of software, hardware, and network resources when you execute a sanity test.

5.     Run the sanity test and observe the test results. If the test fails, analyze its actual cause and fix the issue.

6.     On completion of the test run, evaluate the test result to find any bugs.

7.     Now, you have to report those identified bugs to the developers. However, the best practice here includes detailed information in terms of screenshots and test log files.

8.     You can repeat the process to re-check the stability of the software application.

Sanity Testing Tools

Performing sanity tests using automated test tools lower the working load of the testers and developers. It shortens the release time of software applications through the quick execution of sanity tests. Here are some popular automation testing tools to perform sanity tests:

• Selenium: It is an open-source automation testing tool utilized for web application testing. It supports multiple programming languages like Java, JavaScript, Python, etc.
• Cypress: It is an open-source JavaScript automation testing tool built on Mocha. Cypress utilizes unique DOM manipulation techniques and supports multiple programming languages like C#, Perl, PHP, etc.
• Playwright: It is a Node.js library for web automation. It is used to automate Firefox, Webkit, and Chromium with a single API.

The list of sanity testing tools above is just a few examples of many other options in automation tools. You should evaluate different options based on the specific needs of the applications.

However, automation tools have some limitations, which can create hurdles in performing sanity tests. For example, automation tools have limited scope as you can only test to which they are programmed. Further, issues of maintenance overhead and huge costs with automation tools demanded alternative options.

It is preferred to run sanity tests in a cloud platform. Cloud-based testing platforms can offer several advantages for sanity testing in software development, including reliability and scalability. If you want to scale your digital experiences, it is recommended to perform cloud testing. Let us learn this in detail in the below-given section.

Performing Sanity Testing on the Cloud

Now that you know the different ways to perform sanity tests and their advantages, let us dig into performing sanity testing on the cloud. Sanity tests in the cloud-based environment can lower infrastructure costs, give scalability to run automated tests, enable team collaboration, and provide flexibility in the testing environment.

Cloud-based continuous quality testing platforms like help you perform manual and automated web and mobile app testing.

Sanity Testing Best Practices

Following the best practices, you can execute effective sanity tests and ensure the function of the software application as expected:

• Before initiating a sanity test, you should define its objective and requirement to make the test focused on the criticality of a software application.
• Whenever it is possible to automate sanity tests, you should always prefer it over a manual test to speed up the testing process.
• Always use a standardized test environment for sanity tests. For example, you can choose the standardized configuration of software, hardware, and network environment.
• Create test cases based on the criticality and risk of the application. It will help you execute the priority test case first.
• To track the progress and address any issues identified, you should document the outcome of the sanity test. This involves recording test case outcomes, defects found, and actions to solve them.
• Sanity tests should be performed regularly to detect bugs or errors early. You should always do a sanity test after significant changes are done in the software applications.

Conclusion

A sanity test is an approach for testing software applications to check whether new functionality, change, or bug fix is implemented successfully or not. Its scope is narrow and focused, which allows ensuring whether your software release meets the requirement for rigorous testing or is too flawed to be tested.

Many testers and developers are confused between smoke and sanity tests. In this article, we have discussed their major differences, which gives an idea of its distinct purpose. However, they both are essential in the development of software applications.

The sanity test saves cost, time, and effort by addressing certain application functionality early in the Software Development Life Cycle. Further, it ensures the stability and reliability of software applications through its quick check on critical features. Overall, sanity testing is important to maintain high-quality software applications and ensure the satisfaction of end-users.

We Provide consulting, implementation, and management services on DevOps, DevSecOps, Cloud, Automated Ops, Microservices, Infrastructure, and Security

Services offered by us: https://www.zippyops.com/services

Our Products: https://www.zippyops.com/products

Our Solutions: https://www.zippyops.com/solutions

For Demo, videos check out YouTube Playlist: https://www.youtube.com/watch?v=4FYvPooN_Tg&list=PLCJ3JpanNyCfXlHahZhYgJH9-rV6ouPro

If this seems interesting, please email us at [email protected] for a call.

Relevant Blogs: