July 28, 2020 Jonny Steiner

How Experitest and AccessibilityOZ Incorporate Accessibility Testing into Mobile App Testing

Share this knowledge!

Web and mobile application development has changed gears in recent times. Rapidly innovating technologies and tools, organizations are now able to quickly build a variety of applications. While developers focus on adding new and exciting features to the app, one aspect that cannot be ignored is accessibility. So, adding accessibility testing automation into your development pipeline more important than ever.

An overview of Accessibility

Accessibility, in the computing world, is about developing tools, technologies, websites, and mobile apps that are accessible to all people, regardless of their disabilities or the severity of the impairment. It means everyone should be able to understand, navigate, and interact with the software application.

Disabilities are broadly classified into five categories

  1. Vision
  2. Hearing
  3. Movement
  4. The mind interpreting information
  5. Mental health

According to an infographic from the Centers for Disability Control and Prevention (CDC), 26% of the US adult population has some type of disability. As such, it is important for developers to accommodate accessibility features into the application so that people with disabilities can access all features of the software without being categorized as ‘disabled’.

Accessibility and Legal implications

Accessibility features are not just an option. Software development entities are required to comply with regulations imposed by the US Department of Justice and the Office for Civil Rights (OCR) which is part of the US Department of Health and Human Services (HHS). In addition, they have to comply with Section 508 which mandates that the electronic and information technology developed or procured by US State, Federal and Local Governments should be accessible to all people, including disabled people. Section 504 compliance is about preventing disability discrimination. If your company deals with audio and video communication systems, you are supposed to comply with the Communications and Video Accessibility Act (CVAA) that mandates providing access to innovative 21st-century technologies to all people.

Similarly, every region has its own regulations. In Australia, the Disability Discrimination Act requires all companies and organizations to adhere to disability regulations. In Europe, EN 301 549 specifies accessibility requirements for the procurement of ICT products and services in Europe. Failing to comply with these regulations will result in severe implications.

For instance, the Sydney Olympics website was slapped with a fine of $20,000 and legal costs of $500,000 in 1999 for not following the accessibility regulations. Similarly, Target website in the US had to settle accessibility litigation for $6 million while paying a legal fee of $3.5 million in 2009. Streaming media giant Netflix incurred legal costs of $795,000 for failing to provide captions in 2009. Looking at these numbers, it is evident that software companies should take accessibility features more seriously.

Web Content Accessibility Guidelines (WCAG) is a set of guidelines developed through the World Wide Web Consortium (W3C) process which provides a single standard for web content accessibility. So, organizations should follow the WCAG standard while implementing accessibility testing automation for web and mobile applications. The current technical standard for web accessibility is WCAG 2.1

Native Apps Accessibility Testing Methodology

As accessibility features become a key requirement for software development, accessibility testing becomes a key requirement too. While WCAG 2.0 success criteria are applicable to mobile, it doesn’t cover certain aspects of accessibility mobile features such as touch screen related issues. While WCAG 2.1 tries to cover touch screens, sensors, and small screen devices, there are still certain aspects of mobile accessibility user needs that are still uncovered. To evaluate mobile website accessibility functionality, the ICT Accessibility Testing Symposium has developed a mobile accessibility testing methodology, incorporating accepted mobile accessibility testing standards from around the world.

The important step in this methodology is to define what to test. For a native app, define the main purpose of the app and what function is critical to its purpose that can be tested for workflow, operability, and efficacy from a user experience perspective.

Here are some of the common elements to test while performing accessibility testing automation:

  • Navigation ( Menus, Header, Footer)
  • Login Pages
  • Settings
  • Widgets (Calendars, Date Pickers)
  • Contact Us
  • Realtime Updates (Uber, eBay)
  • Landing Pages
  • Account and Profile
  • Emergency alert pages
  • Privacy Policy
  • Terms & Conditions
  • Interactional / Transactional (Add to Cart / payment / product selection / Help / Q&A)
  • 3rd Party integrations (chat, geo-location maps etc.)
  • And/or high-traffic areas

Testing Native Apps for Accessibility

Here are steps to perform accessibility testing automation on native apps.

Step 1: Test Critical Issues

The first step is to test critical issues of the app. As new features are added, new traps are formed in the app. In Accessibility, a trap is something wherein a user gets trapped with a component and cannot escape from the page except by closing the app. For instance, screen readers users should always be able to activate an item on the current page or navigate to the previous page and next page easily. So, you should check if the screen reader user is able to select an item or gets trapped and closes the app. It is observed that native apps contain more traps than desktop ones.

Step 2: Testing Mobile-specific Issues in Accessibility Test Automation

The second step is to test mobile-specific issues. For instance, the screen orientation doesn’t change from portrait to landscape when you rotate the mobile.

These mobile-specific issues are broadly categorized into the following segments:

  • Display
  • Alternatives
  • Audio and Video
  • Actionable Items
  • Forms
  • Navigational Aids

Step 3: Testing Mobile Assistive Technology and Feature Support

The third step is to test if the mobile assistive technology is able to access actionable items and information.

  • The assistive technology should be able to access all actionable items
  • The assistive technology should be able to access all important content

Experitest is one of the few mobile test automation solutions that can perform this test.

Here are elements to check out on mobile devices

For iOS:

  • Keyboard
  • VoiceOver
  • Zoom
  • Invert Colors
  • Grayscale
  • Keyboard & Switch

For Android Devices:

  • Invert Colors
  • Increase text size
  • TalkBack
  • Grayscale
  • Keyboard
  • Keyboard & Switch
  • Magnification

When you test a kindle, you should test the Voiceview Screen reader. Similarly, when you test a Samsung device, you should check the Voice Assistant for Screen Reader on Samsung.

Accessibility Testing Automation Demo

SeeTest is the Experitest cloud platform that enables you to run test scripts on a variety of mobile devices located across the globe. It offers iOS simulators and Android Emulators for manual as well as automated testing. The tool supports all major browsers to run cross-browser testing.

cloud

Here is a demo about running a few test scenarios related to accessibility testing automation.

Here is a real iPhone device for the test.

iOS device reflection

As you can see from the above image, the user interface screen contains different tabs such as Performance, Logs, Monitors, Automation, Report, and Accessibility. When you click on the Automation tab, you will see a small accessibility switch that will show only elements in the user interface, enabling you not to miss out on any element.

automation tab

For instance, open another application or a page and check the user interface screen.

open snapchat

Now, when you turn on the accessibility switch, the user interface screen shows the elements of that page. Here you see that there are no elements on that page.

snapchat elements

Now, click on the Accessibility tab. The screen will display accessibility test options such as color blind, blur, and shrink.

device accesibility

For instance, select the ‘color blind’ option and the display shows a color-blind view.

accessibility testing - colorblind

You can now check the contrasting view of an application. Similarly, when you click on the blur option, it will blur the display.

SeeTest offers a UI Inspector feature that will let you check the accessibility features with regard to the UI of an application. This is a useful feature that makes your accessibility test automation task simple and easy.

Open an application and click on the ‘ Start Inspector’ button.

accessibility testing inspector

Now, the application is open. It has a screen reader with a voiceover feature.

accessibility testing - previous

You can see a finger swipe option to navigate to the next and previous page. It also shows the text that will be read to the user (Element properties). So, you can verify the order and ensure that there are no traps.

accessibility testing -

When you click on the next or previous, you can check the order in which the text is displayed. Similarly, when you move to a new screen, you can see that the focus is on the element.

accessibility testing - build

Now, when you click the next button (right swipe), the focus is now moved on to the next element, and the same is displayed on the user screen interface under the Element Properties (The text that will be read to the user). You can also see a ‘Dismiss’ option which will close the login action and take you back to the login screen again. So, you can check if there is a trap.

You can also check other features such as orientation.

accessibility testing - build

When you click the orientation button, it will rotate the screen from portrait view to the landscape view.

accessibility testing - eribank

Now, you can run the same scenarios to check how elements work in the landscape view. SeeTest also comes with Audio support. If you use a headset, you can listen to the voiceover from the device.

So, you can run these accessibility test scenarios on multiple devices and browsers using the SeeTest platform.

Summary

Accessibility testing automation is a key requirement for software development companies. Not only does it enable you to cater to your software to a wide range of audiences, but it also enables you to comply with government regulations related to disability discrimination acts. Software companies can follow the ICT Accessibility Testing Symposium methodology to righty evaluate the accessibility of mobile websites instead of trying to reinvent the wheel.

Try our new Accessibility Cloud and test your mobile applications on +1000 real browsers and devices in the cloud to provide an accessible experience for all users.

SeeTest makes your accessibility mobile testing job simple and easy. By using the cloud-based SeeTest platform, organizations can easily test mobile websites and evaluate the accessibility features on different devices, browsers, and platforms. The centralized dashboard allows you to easily monitor and manage the process while generating customized reports.

This article has been updated for accuracy and comprehensiveness since it was initially published November 17th, 2019.

Guy ArieliCTO

Share this knowledge!