Mobile App Testing Essentials: A Guide for Developers and QA Professionals

Key Things for mobile apps testing:

Install App on SD Card

If associate app is put in from the Google Play Store, it’s put in on the device memory. However, navigation to Settings, Application and use the move to SD card choice to move the app.

Things to know:

• Take away SD Card?
• Any strange behavior once the device is rebooted and therefore the SD card isn’t mounted?
• What if the app is in use and SD card is un-mounted?
• Will the app exit graciously once sure directories area unit deleted from SD card and app is launched?

Try on Associate in Nursing encrypted Mount Rushmore State card
— Use constant card on another device?
Tools to use: Use the tool Moborobo for Windows to put in apps directly on Mount Rushmore State cards

App Upgrade

Install associate existing app, build needed configurations and upgrade it. whereas upgrading associate app, Android replaces existing .jar files if there area unit any changes and modifies the information files if needed. Any unwanted files from the previous version area unit deleted.

Things to know:

• Existing app settings can be drained
• User may receive notifications that were already discharged
• Upgrade might corrupt existing information
• Needed dependency files can be deleted
• Encrypted user files is also decrypted throughout associate upgrade and not encrypted once more

Tools you’ll use:

1. you’ll upgrade associate app via instruction use the subsequent command adb install -r isn’t needed if the app isn’t put in already)
2. Add the file to the device’s American state card and open it through the file manager. Replace existing file
3. Upgrade it through Google Play
4. Use a tool like Moborobo to put in it over existing application.

Claims Testing

Before you install the app from the Store (Google Play or iTunes App Store),browse the outline of the app.Check if the claims are fulfilled or not.

• Check supported versions and pre-requisite for the app to figure properly
• Check the app name, app icon, supplier and therefore the app size. ensure the data once installation
• Is that the claim on Google Play on device same because the claims on the app website?

App Permissions

During installation, app ought to raise the user for permissions to use the device resources or settings. What if you settle for and later take away the permissions?

• Build a note of the desired permissions prompted throughout installation. once installation, faucet on Applications, app information and check if there’s any change?
• Strive ever-changing the permissions one by one. Example: amendment cryptography on American state card, amendment the settings of the device from the Settings menu
• On Android and iOS, there area unit totally different settings to manage these permissions. a number of the settings area unit highlighted in screenshots here

Device Administrators

On Android a security application needs to be a device administrator to perform user specified actions. These actions are useful to control the device remotely, should the device be stolen or fall into wrong hands

Things to know:

• Device administrator ought to be associate nonmandatory feature unless it’s associate email or MDM application
• App ought to prompt the user to authorize as a tool administrator
• Remote management activity on the device should be communicated firmly via SSL, policies applied on the device ought to be encrypted
• If the app supports inter-app communication, make sure that the third party app or service is whitelisted and trusty by the app with device admin controls

Review Feedback on Google Play

Simply learn from Google Play feedback

Things to know:

• How does your app perform in the wild?
• What are they talking about it?
• Do they post issues instead of reviews? Can you help the developer replicate the issue?
• Does that help you review your test ideas?
• Do they talk about app crash, do you have access to these logs or can you approach someone who can share the logs?
• Finally is the review a spam? Would it affect app sale? Can you remove it?

Launch App From Notifications

When you tap on a notification, a dormant app will be launched and the UI — the notification is meant for is brought up. Sounds simple? With built-in actions within notification, there are many areas where notifications could go wrong.

Things to know:

• Does tapping on a notification launch the right UI?
• What happens when you tap on an action such as ‘Reply’ on an email app? Does it let you compose and send or does it pull up the email UI?
• Are notifications meant for the latest OS, backward compatible?
• Unless the app is currently active such as audio/video player, does the app really require a persistent notification on the notification bar?
• As seen in the screenshot, one can control the number of notifications too in iOS
• Also, test for the types of Alert Styles

Splash Screen

A splash screen is the first screen you see once you tap on the app icon.

Things to know:

• Does the splash screen convey any information about the app? Is the app logo, app name consistent with the App Store?
• Does it appear and disappear too quickly?
• Does the splash screen really initiate tasks in background or have just a fake timer?
• Pause/Re-launch, Kill/Re-launch — do you see splash screen?

Landing Screen

The first screen after the Splash screen is the Landing screen for the app. Does it create a positive impression or makes you wait like the one in the screenshot?

Things to know:

• If you can configure the landing screen for your app, try that test
• Landing screen must have a simple layout and be highly usable. As a user, I don’t prefer several options thrown at me just after app launch.

Uninstall Using Google Play

Do you install your apps from Google Play? Have you tried uninstalling apps using Google Play?

Things to know:

• Once the app is installed, launch Google Play and search for the particular app. Tap on ‘Uninstall’ button. Is the app removed?
• Is the shortcut removed from the device pages?
• Does this process leave any residue in terms of data or images or files?
• Are there any settings reset to default settings?
• How different is this from Application Manager > Uninstall?

WiFi and VPN

The interesting thing about Smartphones is their ability to connect to Wi-Fi, but that doesn’t end there, you can even use it to share your mobile’s data network.

Things to know:

• In an MDM scenario when the device is restricted from connecting to a random Wi-Fi network, connecting to a Wi-Fi hotspot created on a secondary device comes in handy
• How would you ensure that the app doesn’t make too many Wi-Fi network calls that could drain the battery?
• Can the app also fall back to Wi-Fi for location in case GPS is turned OFF
• Will newer devices selectively restrict apps from using Wi-Fi and data networks?
• Connect to your corporate network via VPN and test app that requires connectivity.

Lock Screen

A very simple yet effective test. Launch your app and lock the phone. Unlock the phone and observe the app behavior

Things to know:

• Does the app reset the current state and start from scratch?
• On unlocking the device, do you see app screen or home screen?
• Is there any popup for user intervention?You can try both manual lock and auto-lock on the device

Blocking Mode

Set the Blocking mode to On. This feature is useful to test how your app behaves when notifications are blocked.

Things to know:

• App should continue to function properly
• Once notifications are enabled, pending (appropriate) notifications must be displayed
• Existing notifications in the notification drawer should not be removed
• Try combining the notifications feature with alarm/timer popup

Workflow: Settings > Device > Blocking mode

Dual SIM

If your mobile phone supports dual-sim, have both the SIM cards activated and insert them into the device.

Things to know:

• If the app reads data from the SIM card, can you control which SIM card to be used?
• What happens if one or both the SIM cards are removed in between app operations?
• How about different network states of the SIM cards? What if one SIM card is set for data usage only?

Workflow: Settings > Device > SIM Card Manager

Brightness

A mobile app is used in more locations than one can imagine — a coffee shop, while you are waiting for the bus, in the bus, dark meeting room or in the playground

Things to know:

• Set your device brightness to minimum, maximum and check your app display. Does any app screen color hurt the eyes
• Is it difficult to view the app screen in low or high brightness?
• Is there an option to set brightness specific to app?

Power Saving

Have you used the CPU and Screen power saving mode?

Things to know:

• Using low power for CPU means sluggish performance on the device. How does the application perform in this mode?
• Is your app UI optimized to handle low powered screen brightness?
• Does your app query these power saving modes and optimize itself accordingly?

One-handed Operation

I found this feature by accident. Once enabled, the keypad shifts to either right or left of the screen. The user can complete most of the tasks using only one hand.

Things to know:

• Using this feature, a lot of spelling mistakes were observed. The fingers used to a particular position on the keypad.
• This can be used to check an operation, its cancellation and usability of the app
• This might not be available for many devices
• Watch out for inconsistent app behavior

Font

Every user is unique and has varied tastes. One such variety is the type of font one likes. With most of the smartphones giving the user an option to choose a font, users are no longer with the default font. How does your app behave when there is a font change?

Things to know:

• Does the app continue to use the default font?
• Some of the symbols would not be displayed in the non-default fonts
• Change font when app is in use
• Which font is given higher priority — app font or device font?

Font Size

Once you select the font of your choice, why not change the default font size? Some like it tiny so that they can see lot of text at one shot while others prefer Huge text so that they don’t strain their eyes.

Things to know:

• Which category of user groups do you belong to — The Teenager or The Grandma?
• Does text wrap itself when font size is increased?
• Does text overlap when font size is changed?
• Will the settings take effect immediately or need app restart?
• Is the font size applied consistently across app?

Language

How many languages can you read and write? The more you know, the better especially if you are a software tester. This test idea is all about Device Language

Things to know:

• Change device language from the default language to any other language.
• Change the language before launching the app. Repeat this once the app is launched. Is the app behavior affeected?
• If your app has a setting to change language, which setting is given the higher priority: Device language or app setting?
• Look out for partially translated text. Also, some of the strings might be garbled.
  As seen in the screenshot, there is a different setting for Language and Voice Control in iOS. Try having two different languages selected and see how the app behaves.

Backup and Restore

Let us look at backup and restore at app context and MDM context separately

Things to know:

• Backup and restore an app using a third party app. Are you able to use the app? Or at least does the app rollback to fresh install state?
• What happens when the application is backed up and restored from the cloud? Is the app data intact as expected to be?
• Does backing up application data alone and restoring have an effect on the app?

Learning from MDM context:

• Should apps be restricted from backup?
• Are restricted apps that are backed up to the cloud and restored, securely wiped?
• What happens to the data that was securely wiped, if restored from the cloud?
• Are they monitored and wiped again?

Date Format

Does your app display or takes date as input? Have you tried with different date formats?

Things to know:

• As seen in the screenshot, there are at least three ways to display date. Which one is the default for your app?
• As a user, can you change the format? Does the change in system defaults affect the app setting?
• A test idea closely related is the Time Zone test idea. Test both of these together
• On devices that have been reset to factory settings, the date and time is not set to the correct values. How does the app behave?
• Simulate the above scenario by doing a Reset Factory Settings, Install the app and then check the date format displayed

TimeZone

How does the app behave when you travel frequently across countries? What if there are multiple time zones within the same country? Aware of the day light savings?

Things to know:

• If your app is dependent on time zone a lot, check how the changes affect the behavior of the app
• Change time zone before and after launching the app
• Perform few operations after changing the time zone and reset the time zone
• As a user, can you guess what will be thebehavior? Is the app behavior consistent and correct?
• A good test idea would be to test on multiple time zones and monitor the app for more than 24 hours.

Accessibility

There have been companies sued for creating products not catering to Accessibility needs. Don’t let your company be one of them.

Things to know:

• Few Accessibility guides:
  iOS:https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/iPhoneAccessibility/Accessibility_on_iPhone/Accessibility_on_iPhone.html
• Android: http://developer.android.com/tools/testing/testing_accessibility.html
• Tizen: https://developer.tizen.org/documentation/ux-guide/basic-interactions/accessibility
• Use device Accessibility settings and test for each of the option with respect to app screen
• Take feedback from challenged people about the app
• Some of the additional settings vary across platforms. Use them
• Read the Accessibility guides and check if you are violating many basic principles of Accessibility

Encrypt Device or SD Card

Encryption is good, alright, but have you wondered how does your app perform while accessing an encrypted SD card or for that matter on an encrypted device?

Things to know:

• Does the app make best use of encryption? Did you know that apps using encrypted content can use secure video capabilities of attached displays (link: http://developer.android.com/about/versions/jelly-bean.html) ?
• Does your app prompt for permissions to access encrypted SD card content (refer screenshot)
• Compare performance ratings for your app installing on devices with and without encryption

Fake GPS with Mock Locations

Mock locations was introduced to help developers simulate location related scenarios on the Android Emulator. Mock locations provide an easy way to test out geo-fencing related features without having to wander around. But hey, mobile applications are all about mobility, so do spend some time field testing your app.

Things to know:

• Allow Mock Locations is available under Settings > Developer options
• Fake GPS is the app that helps you simulate mock locations if this option is turned ON
• Does the app pick up the simulated location?
• Disabling Fake GPS is the best way to find if the app falls back to network or WiFi based locations
• This test idea is a good reminder that one can use third party apps to test a particular app. Scout for such apps and build your tool-kit

Do not keep activities

The Android team was unable to replicate a CPS ticket. Customer reported that when the user opened a document on a third party viewer and pressed the back button to return, the app crashed.

Things to know:

• An Activity is the ‘Window’ equivalent in Android
• When multiple UI (or Window) stack up and stay unused for a longer period, Android removes them from the memory to avoid performance issues
• When user presses the back button, the app has to retrieve the UI that was closed to show it to the user
• Do Not Keep Activities under developer options simulates this scenario to help the developer handle such instances where UI is removed by the OS but retrieved by the app
• Keep this option checked while testing an app during the QA phase

About Device

Many testers seem to struggle in identifying details about the device. Most of the devices have a separate option presenting details about the device.
Use these screens to understand the device hardware and OS

Logs Can Speak

iPCU or ADB, when a device is connected and watched for logs, it could speak volumes about the app.

Things to know:

• Violation of privacy through personally identifiable information such as name, phone number, address, phone’s IMEI,MEID, UUID shall not be logged
• Sensitive XMLs/JSONs, URLs accessed by app may not be logged. Also, app exceptions, DB locks, DB data can be traced through logs
• Passwords should not be in clear text

Tools you can use:

• http://www.satisfice.com/tools/log-watch.zip
• iPCU in iOS
• ADB in Android
• Access built-in log generation tools on certain apps

Look for log viewers in Google Play, iTunes App store

Crash Reports

What to do when app crashes? Click on Report and send email to the developer. If you can include your email, add it. Or else, take screenshot of each of the screens and send it to developer.

Things to know:

• If you use Android Debug Bridge (ADB) on Android or iPhone Configuration Utility (iPCU) on iOS devices, it is easy to capture these logs
• It is a good idea to always have the device hooked up to ADB/iPCU while testing and monitor the interactions
• Over a period of time, learn to analyze logs

Error Messages

Every app developer must ensure that error messages are given equal importance before app release. As software testers, it is a good idea to force the app to spit out all the error messages.

Things to know:

• Remember the FAILURE Mnemonic by Ben Simo.
  F: Functional
  A: Appropriate
  I: Impact
  L: Log
  U: Usability / UI
  R: Recoverability
  E: Emotions
• Does the app let the user know why the error message is displayed? Is the error message meaningful?
• The attached screenshot is a good example of an error message. All the relevant details are highlighted

Network Interruptions

There are situations when there is no network or weak signals. How does the app react in such situations?

Things to know:

• Does the app let the user know immediately or the user is left to figure out?
• How does the app recover in case the connection is retrieved?
• Test with different signals — 2G, weak/strong signal, 3G, Wi-Fi

Home Button vs Back Button

On Android application when an app is closed by pressing the device’s home button, the app is left in a suspended state, whereas on a back button, the app is removed from the memory.

Things to know:

• When app is closed using the home button, app should resume from where the user left
• What is the performance impact on the device when an app is launched from a suspended state?
• When closed using a back button the app should be removed from the memory, when re-launched database and UI states should be reinitialized
• The home key was pressed when I was playing the below game. On re-launch, the Paused screen was displayed
• The user should be able to resume from the suspended state without any loss of data

QWERTY Keyboards

Devices with QWERTY keyboards can be obsolete but on older devices with QWERTY keyboard, the app should perform as expected.

Things to know:

• App should change orientation once the keyboard is pulled out
• When user taps on any input field, On-screen keyboard should not pop up if the QWERTY keyboard is already pulled out
• Test out apps such as financial calculator, scientific calculator, games, productivity apps which involve calculations
• Do test QWERTY keyboard on a localized app.

Longer Operations

Have you seen this error while using an app ‘’Application not responding’. This is the dreaded ‘not responding’ equivalent of Windows on Android or iOS caused by longer operations by the app.

Things to know (plus best practice):

• The application must handle any long operation by showing a processing icon
• If larger file downloads are in progress, appropriate progress bar should be shown
• Database encryptions or compacting files or folders can take longer time, this has to be visually indicated to the user (in the screenshot provided, the app actually encrypts docs, converts db to latest version among several other things)

Downloading resource files for images especially on games app, installing updates, downloading large documents should be avoided on a low battery

Resumable Downloads

Any download within the app should be resumable not only for usability but it also conserves network and battery.

Things to know:

• Large downloads over 2G networks drains battery. It could trouble the user if the app does not provide an option to resume downloads from where it left
• You may also have to test by closing the app using the back button during the download, reopen it to ensure that the download continues

Clear App Data

This is one of the best hacks you can perform on an Android device. Install an app and open Applications under Settings, and clear the app data.

Things to know:

• Most Android applications store data on a SQLlite database or XMLs. Clearing the app data could make the app unstable.
• On security apps, app data cannot be cleared most of the times
• Most of the times, the app must roll-back to a fresh install when the app data is cleared

Kill Background Process

There are different types of apps, especially in Android, background, foreground and service apps. Let us take for example the email app on Android. Although the email app appears to be one, it spins off several ‘worker apps’ or service apps that takes care of specific functionalities such as email sync, downloading attachments, exchange connector etc.

Things to know:

• How does the app handle when its background processes are killed?
• How does a primary app handle when its core service app could be killed and removed?
• Does killing the primary app have any effect on the service app too?
• In iOS, you can double press the Home key and get the list of running apps. Long press on the app icon. Once the red colored — icon appears on the app icon, tap it to kill the app

Other App Interactions

Two ways an app can interact with other apps. * Passing on files for a functionality like document viewer, or attachments via emails * The developers could agree and whitelist each other’s app to perform a functionality like an MDM app can configure VPN on Cisco AnyConnect app

Learning Case 1:

• Are the third party apps prompted for a user to open, relevant to the file or operation that is about to be performed?
• Does your application gracefully exit or stay suspended when the third party app is open inthe foreground?
• Does the third party app require the user to login? Post login does the process continue?

Learning Case 2:

• What are the security measures used or required for both apps to communicate securely?
• Do both the apps support SDK interaction? If so, what APIs can be called? Are they secure? Can any fake app call the same APIs?
• What happens when a newer version of the third party app is released?

Rooted Device

A rooted device is a potential playground for a tester. You can get access to the app’s data directory, push or pull files from the app directory, remove files and also pull out the database.

Things to know:

• On Android device, once you unlock the bootloader and install an ‘su’ binary (application), it gives super-user access to apps that desires to control certain aspects of the device
• Usually App data such as database, files, XMLs are stored under Data > Data > <App Bundle ID> directory
• You may have to provide permissions through adb shell in order to browse through these directories
• What happens when a file that is critical to an application is deleted?
• Is the SQLlite db unprotected? Does the db contain sensitive information?
• What happens when a file currently accessed by the app is deleted? Are the files encrypted?
• What happens when the files are replaced with dummies?What happens when the app is given Super User permissions?
• Does it utilize abnormal amount of resources?

Voice Search

An unexpected component that you might encounter on certain app’s search bar is voice search.

Things to know:

• Look for accuracy of the search
• Voice search on certain apps could crash the app
• If an MDM policy such as disable microphone is enabled the voice search may not be allowed and handled inappropriately
• Try with different accents and voice pitch

Voice Dictation

Did you know that certain keypads provided voice dictation options? Have you verified if your app supports voice dictation?

Things to know:

• Validate all input fields that requires keyboard using voice dictation
• Check whether the dictation is localized or not
• If a localized keyboard is open, unsupported dictation must go away

Show Layout Boundaries

Few devices have the option ‘Show layout boundaries’ under Settings > Developer options > Drawing. As seen in the screenshot, the layout boundaries help identify any alignment issues.

Things to know:

• Enable the option and navigate through all the screens of the app
• Watch out for position, size and alignment of text, icons and borders
• This is a quick test to check on the device rather on the monitor or design software

Performance Indicators

Battery, CPU & Memory are paramount for any smartphone to operate to its optimum capacity. Apps installed can drain battery, consume more CPU or clog the memory, thereby slowing down the device.

Things to know:

• Excessive logging could drain battery, consume memory
• Excessive use of network could drain battery
• Continuous UI activity, excessive wake locks on Android devices to keep the device ON, could consume more CPU

Tools you can use:

• On iOS, Instruments, packaged with Xcode can be used to profile the app
• For Android, third party profilers such as ‘Little Eye’ can be used
  Apps like Battery mix, SystemPanelLite (CPU/Memory), Cool Tool — system stats can be used (CPU/Memory/WiFi), Caffine — to benchmark battery usage when display is ON forever.

Low Disk Space

After taking a lot of pictures, video recording, the device runs out of space. Have you faced such a scenario?

Things to know:

• Remove the SD card and install many apps. Run multiple apps and let them write to the device. Observe the app behavior
• Is the user notified? Does the app handle this scenario gracefully?
  What happens once the disk space is increased?

Low Battery

We have covered low battery related scenarios in other test ideas. So, let us share a few more learning here.

Things to know:

• The app should avoid making network calls on a low battery scenario unless it is very critical
• Can the app destroy itself from consuming more battery and launch on battery recharge?
• Excessive logging on an app can also drain the battery, it is better if logging is switched off automatically when the app identifies low battery
• For further low battery related ideas, do refer this patent for interesting hardware related scenarios (assigned to Qualcomm)
  http://www.google.com/patents/US20120108224

The Hidden Slider

Sliding drawers or sliders is a usability feature that partitions UI for better usability. Sliding drawers are prone to crash since two different layouts are fused together.

Things to know:

• Usability, indication or a handle that shows that a sliding drawer exists, look for accessibility options
• Drawer UI when tested using monkey tester can crash the app
• Drawer behavior between Portrait and Landscape orientation
• Behavior on Phones and Tablets

Tools you can use: Monkey testing via adb adb -d shell monkey -p <package name> -v <event count>

Resources
1.http://developer.android.com/index.html for Android
2.http://developer.apple.com/library/ios/#documentation/UserExperience/Conceptual/MobileHIG/Introduction/Introduction.html for iOS
3.http://www.appqualityalliance.org/files/AQuA_top_10_app_errors_good_qa_helps_you_to_avoid.pdf
4.https://developer.tizen.org/documentation/ux-guide/design-principles
5.https://sailfishos.org/develop-overview-article.html
6.Tap Into Mobile Application Testing — Jonathan Kohl
7.Keep collecting test ideas from every source

Originally published at https://geekautomation.blogspot.com.