Xigua Video iOS Voice Over Accessibility Adaptation Practice

Author: Chen Junlin

In order to solve the difficulties encountered by the elderly, disabled people and other groups in using the Internet and other smart technologies, Xigua Video has launched a special action for barrier-free and aging-friendly transformation since the spring of 2021. It has successively completed multiple transformation needs such as barrier-free theaters, color-blind mode, eye protection mode, large font mode, and external subtitles, fully meeting the needs of special groups such as the visually impaired, hearing-impaired and the elderly.

From a research and development perspective, this article describes how to use Voice Over, how to adapt Voice Over, and how to solve problems encountered during the adaptation process.

1. Introduction to Voice Over

Voice Over is a tool that comes with Apple phones. It allows people to understand page content and information without looking at the screen. Blind users rely on Voice Over to provide auditory feedback when using iOS devices. Apple has implemented the Voice Over feature in various self-developed systems, such as OS X, iOS, watchOS, etc. Through Voice Over, visually impaired users can get the same experience on different devices.

There are many users who use narration to visit Xigua Video, and "Blind Xiao Ma Ge" is one of them. Students who are interested can learn more about narration by watching his videos.

2. Voice Over Usage Guide

This section introduces the development environment that Voice Over developers usually need to configure, and uses Xigua Video as an example to demonstrate the basic operations of Voice Over.

Voice Over Development Environment Configuration

For the convenience of testing, it is recommended that developers set the accessibility shortcut key to switch to VoiceOver. Just press the right switch key three times to turn VoiceOver on/off.

The "General" in the path below is applicable to lower version systems. For higher version systems, you can find the narration by going to Settings -> Accessibility.

Turn Voice Over on/off

Settings -> General -> Accessibility -> VoiceOver

• Set to turn on/off Voice Over by pressing the right power button three times in a row (you can easily turn on/off Voice Over after turning it on)

Settings -> "General" -> "Accessibility" -> "Accessibility Shortcuts", check the Voice Over function.

• Open the Voice Over+ subtitle panel (you can easily view the full reading of the narration after opening it)

Settings -> "General" -> "Accessibility" -> "VoiceOver" -> "Subtitle Panel"

Detailed explanation of common Voice Over gestures

Here we mainly list the most commonly used gestures in iOS system and Xigua Video. For complete operations, please refer to Apple's official documentation (https://support.apple.com/zh-cn/guide/iphone/iph3e2e2281/15.0/ios/15.0).

Getting Started Gestures

• Filter elements using gestures
• Tap or touch an item - select and read the item
• Swipe right - select next item
• Swipe left - select the previous item
• Slide three fingers in a certain direction - page turning operation, which can make UIScrollView (including TableView, CollectionView) turn a page in a certain direction

Responding to element events using gestures

• Double-tap - respond to the event of the current element
• Triple tap - Double-click the currently selected element
• Double-tap with two fingers - respond to developer-defined shortcut events (such as play video, pause video, etc.)
• Swipe left or right with two fingers (move two fingers back and forth three times quickly to form a "Z" shape) - close the pop-up window or return to the previous page
• Two-finger double-tap and hold - user-defined element name

Advanced Gestures

Use gestures to quickly filter elements

• Tap with four fingers near the top of the screen—Select the first item on the screen
• Tap with four fingers near the bottom of the screen—Select the last item on the screen

Use the VoiceOver rotor

You can use the rotor to change VoiceOver settings (such as speaking rate, traversal method, and more), jump from one item to the next on the screen, select a special input method (such as Braille Screen Input or Handwriting), and more.

Use the following gestures to use the rotor.

• Two-finger rotation - select rotor settings
• Swipe up - move to previous item or up (depending on rotor setting)
• Swipe down - move to the next item or down (depending on rotor setting)

3. Quickly adapt to Voice Over / Accessibility

It is not difficult to make an App accessible and usable. Most of the accessibility adaptation work can be completed by setting the accessibility focus, setting the accessibility copy, and adjusting the focus order. This section provides a quick introduction to the above three steps of adaptation and uses the Xigua homepage as a practical example to explain it to everyone.

Set Accessibility Focus

Focus is the only way for visually impaired users to access the app. When Voice Over is turned on, if an element is not set as focus, then the element will not be accessible on the screen. Users can "browse" content by moving the focus on the screen. Correct focus labeling and focus order can give screen reader users a better experience.

When VoiceOver is turned on, the current focus point will be highlighted with a black frame. You can switch focus points by tapping a location or swiping left or right with one finger.

Which scenes need focus:

• Functional components other than decorative components require focus, such as title, author avatar, likes, favorites, comments, and more.

Which scenes need to merge focus:

• Focus points with the same function and close distance need to be merged, such as the author's name and author's avatar (their functions are to jump to the personal homepage and they are close to each other).
• Scenarios that require efficient information filtering need to be merged, retaining the main function of the merge focus, and adding other functions to the rotor, such as Feed. Refer to the design of each business party for details.

How to set accessibility focus:

For most cases, we can use the following properties to adjust the focus.

isAccessibilityElement: Use this property to adjust the accessibility visibility of a single element. For most basic UI elements, when the parent control is an accessible element, the child control will not be able to get focus.

accessibilityElementsHidden: This property can be used to adjust the accessibility visibility of a single element and its subviews, generally used to shield the entire component.

accessibilityViewIsModal: Use this property to make the element the only visible one among the same level views. Common scenarios see that the focus is covered but still accessible.

Setting up accessible copy

Good accessible copy can let users clearly understand the purpose of the current focus. Generally speaking, we can divide elements into buttons, input boxes, search boxes, etc., and then mark them.

After we configure the subtitles according to "Development Environment Configuration", the text read by the narration will be displayed at the bottom of the screen.

What accessibility copy should include:

Accessibility copy needs to accurately describe the function and current status of a focus (such as selected, liked, or tab bar). If a focus contains multiple different contents, all visible information in the focus should be read aloud according to the priority of the information.

How to configure accessible copy:

From a code perspective, we can set accessible text using the following properties.

accessibilityLabel: Generally speaking, any element needs to have a Label set. The Label should be a short phrase such as "Like" or "Collect";

accessibilityValue: (optional) used for frequently changing label data, such as the specific value of the like button of a video (xxx people in total);

accessibilityTraits: When an element serves as an accessibility focus, we need to describe the category of the focus, such as button, selected, etc. In this case, we need to use Traits to mark the element;

accessibilityHint: (optional) When the element label cannot clearly indicate the result of the action, a hint should be given;

accessibilityFrame: When the focus of an element is too small and you need to manually adjust the focus to a larger value, you can use Frame

VoiceOver will connect these properties together. Generally, the reading order is accessibilityLabel→accessibilityValue→accessibilityTraits→accessibilityHint.

Special case: If accessibilitytraits contains UIAccessibilityTraitSelected, "Selected" will be read at the forefront.

Adjust the focus order

The focus order should follow from left to right & from top to bottom. Normally, the focus order is consistent with the order of SubViews, and we don't need to manually set the focus order. If there is a problem with the focus order, we can use UIAccessibilityContainer to adjust the focus order on the screen.

accessibilityElements: Through this property, we can specify all accessible child elements contained in an element. Resetting in UITableViewCell requires extra caution, as focus loop issues may occur. Take the following figure as an example. If C, D, and E are accessible visual elements, we can arrange the accessible elements in the order of D, E, and C by setting them as follows.

 A . accessibilityElements = @ [ B , C ]
 B . accessibilityElements = @ [ D , E ]

A.accessibilityElements = @[B, C]B.accessibilityElements = @[D, E]

Watermelon homepage adaptation practice

Below, we take the Xigua homepage as an example to demonstrate the basic barrier-free adaptation.

Taking Xigua homepage as an example, we have five main areas from top to bottom that need to be set as focus. They are search bar, channel bar & channel editor, author dynamic bar (Story), video card (card is a focus, users can perform other operations through the rotor), and bottom tab selector. The following is an explanation of barrier-free adaptation for each part.

Search Bar

 searchBox . isAccessibilityElement = YES ;
 searchBox . accessibilityLabel = @ "Search box" ;
 searchBox . accessibilityValue = @ "Hot search term 1: the first city with a GDP of 4 trillion. Hot search term 2: the launch of a special train from Nanchang to Taiwan" ;
 searchBox . accessibilityTraits = UIAccessibilityTraitSearchField ;

Channel Bar & Channel Editor

 /// Accessibility adaptation of the channel bar
 categoryCell . isAccessibilityElement = YES ;
 categoryCell . accessibilityLabel = @ "Recommended Channel" ;
 categoryCell . accessibilityValue = @ "Hot search term 1: the first city with a GDP of 4 trillion. Hot search term 2: the launch of a special train from Nanchang to Taiwan" ;
 categoryCell . accessibilityHint = @ "Double-tap to switch homepage channels" ;
 /// When not selected
 categoryCell .accessibilityTraits = UIAccessibilityTraitTabBar ;
 /// When selected
 categoryCell . accessibilityTraits = UIAccessibilityTraitTabBar | UIAccessibilityTraitSelected ;

 /// Accessibility adaptation of channel editor
 // If the panel is added to the interface in the form of addSubview, it is necessary to block access to the underlying elements
 @implementation categoryEditViewController ​

 - ( void ) viewDidAppear {
 categoryEditView . accessibilityViewIsModal = YES ;
 /// Move the user focus to the channel editing panel.
 UIAccessibilityPostNotification ( UIAccessibilityScreenChangedNotification , categoryEditView );
 }

 - ( void ) viewDidDisAppear {
 categoryEditView . accessibilityViewIsModal = NO ;
 }

 @ end

Author dynamic column

 storyCell . isAccessibilityElement = YES ;
 storyCell . accessibilityLabel = isLive ? @ "Irie Shanshan, live streaming" : @ "Irie Shanshan" ;
 storyCell . accessibilityHint = isLive ? @ "Double-tap to enter the author's live broadcast and watch the video" : @ "Double-tap to enter the author's video list and play the video" ;

 @implementation storyCell ​
 // When TableView and CollectionView are nested, cells may not scroll naturally.
 - ( void ) accessibilityElementDidBecomeFocused {
 [ self . collectionView scrollToIndexPath : self . indexPath ];
 }

 @ end

Video List

 videoCell.isAccessibilityElement = YES ​
 videoCell . accessibilityLabel = [ video title, author name, number of likes, number of favorites, number of comments, etc. ]
 videoCell . accessibilityHint = @ "Double-tap to enter the details page to play the video, swipe up or down to use the rotor to access more functions"

 UIAccessibilityCustomAction * action1 = [[ UIAccessibilityCustomAction alloc ] initWithName : @ "Like" target : self selector : @ selector ( action1 )];
 UIAccessibilityCustomAction * action2 = [[ UIAccessibilityCustomAction alloc ] initWithName : @ "Collection" target : self selector : @ selector ( action2 )];
 UIAccessibilityCustomAction * action3 = [[ UIAccessibilityCustomAction alloc ] initWithName : @ "more" target : self selector : @ selector ( action3 )];

 videoCell . accessibilityCustomActions = @ [ action1 , action2 , action3 ];

4. Introduction to Voice Over Related Protocols

UIAccessibility Accessibility Labels

 @interface NSObject ( UIAccessibility )
 /// Mark whether an element is an accessible element
 @ property ( nonatomic ) BOOL isAccessibilityElement ;
 /// A brief description of the accessibility element that marks an element, such as (like, favorite)
 @ property ( nullable , nonatomic , copy ) NSString * accessibilityLabel ;
 /// (Optional) A detailed description of the accessibility element that marks an element, such as (tap twice to like with xxx people, tap twice to add the video to the favorites list)
 @ property ( nullable , nonatomic , copy ) NSString * accessibilityHint ;
 /// (Optional) Mark the specific value of an element's accessibility element, such as 50%, etc.
 @ property ( nullable , nonatomic , copy ) NSString * accessibilityValue ;
 /// (Optional) Mark an element's accessibility features such as (buttons, tabs, etc.)
 @ property ( nonatomic ) UIAccessibilityTraits accessibilityTraits ;
 /// (Optional) Customize the focus size of an element in accessibility mode
 @ property ( nonatomic ) CGRect accessibilityFrame ;

 @ end

UIAccessibilityAction accessibility gesture response

 @interface NSObject ( UIAccessibilityAction )

 /// Customize accessibility events to isolate click events from ordinary users.
 - ( BOOL ) accessibilityActivate API_AVAILABLE ( ios ( 7.0 ));

 /// When an element is defined as a variable element, the following methods can be implemented, such as volume and brightness adjusters.
 - ( void ) accessibilityIncrement API_AVAILABLE ( ios ( 4.0 ));
 - ( void ) accessibilityDecrement API_AVAILABLE ( ios ( 4.0 ));


 - ( BOOL ) accessibilityScroll : ( UIAccessibilityScrollDirection ) direction API_AVAILABLE ( ios ( 4.2 ));
 /// Usually the floating window or page needs to implement this method. After implementing this method, the user can use the "Z" gesture to exit the page
 - ( BOOL ) accessibilityPerformEscape API_AVAILABLE ( ios ( 5.0 ));
 /// This method can receive the user's double-finger double-click event. It can be used to pop up more panels or define some important operations such as pause.
 - ( BOOL ) accessibilityPerformMagicTap API_AVAILABLE ( ios ( 6.0 ));

 /// The core method of the rotor. After assigning accessibilityCustomActions, you can swipe up and down to access the rotor
 @ property ( nullable , nonatomic , strong ) NSArray < UIAccessibilityCustomAction * > * accessibilityCustomActions API_AVAILABLE ( ios ( 8.0 ));
 @ end

UIAccessibilityFocus Accessibility focus response

 @interface NSObject ( UIAccessibilityFocus )

 /// Focus response event, which can capture whether the element has become the focus. Usually rewrite these two methods.
 - ( void ) accessibilityElementDidBecomeFocused API_AVAILABLE ( ios ( 4.0 ));
 - ( void ) accessibilityElementDidLoseFocus API_AVAILABLE ( ios ( 4.0 ));

 /// You can determine whether the element is focused. Usually, do not process this method.
 - ( BOOL ) accessibilityElementIsFocused API_AVAILABLE ( ios ( 4.0 ));

 @ end

UIAccessibilityContainer Accessibility Custom Focus

 @interface NSObject ( UIAccessibilityContainer )

 /// Returns the total number of accessible elements in the current container
 - ( NSInteger ) accessibilityElementCount NS_SWIFT_UI_ACTOR ;

 /// Returns the element corresponding to index
 /// You need to ensure that the returned instance is a real instance (not allocable)
 - ( nullable id ) accessibilityElementAtIndex :( NSInteger ) index NS_SWIFT_UI_ACTOR ;

 /// Returns the index of the corresponding element
 - ( NSInteger ) indexOfAccessibilityElement : (id ) element NS_SWIFT_UI_ACTOR ;
 ⚠️If you use the above three methods, you generally need to implement them all at the same time.
 ⚠️If you implement the above three methods, then do not manually set accessibilityElements

 /// The accessible element List of the container. After setting, the scope of accessible elements on the page will be limited to the List, and the order is the same as the order of elements in the List.
 @ property ( nullable , nonatomic , strong ) NSArray * accessibilityElements API_AVAILABLE ( ios ( 8.0 )) NS_SWIFT_UI_ACTOR ;

 /// default == UIAccessibilityContainerTypeNone, in most cases just keep None. If you set the Type manually, you need to implement the specified protocol for Type.
 @ property ( nonatomic ) UIAccessibilityContainerType accessibilityContainerType API_AVAILABLE ( ios ( 11.0 )) NS_SWIFT_UI_ACTOR ;

 @ end

5. Common Problems and Solutions

This section lists several solutions to common accessibility issues based on Xigua's real business practices, such as unclear focus purpose, focus jumping, and nested container ScrollView not being able to follow scrolling.

Focus loss/focus redundancy

Loss of focus is the most basic but most serious issue in accessibility bugs. In extreme cases, it can cause fatal blocking bugs (such as the back button losing focus). Generally speaking, some buttons, links, check boxes, images, edit boxes, etc. on products require focus to facilitate user interaction.

In addition, focus redundancy will seriously affect the user experience. After some decorative elements have redundant focus, the user's information screening efficiency will be seriously reduced. Generally speaking, components that do not have a clear response event after the user clicks should not become the focus.

"Solution": In general, the system will automatically add focus to interactive controls such as standard controls such as UIButton, and other custom controls need to be added/changed manually. Generally, manually specifying the property isAccessibilityElement to YES or NO can solve this problem.

 self . isAccessibilityElement = YES ;
 self . accessibilityLabel = @ "Like" ; // Usually the focus description is also missing and needs to be supplemented.

 /// For some custom UI controls, you can also rewrite the Get methods of the above two objects.
 - ( NSString * ) accessibilityLabel {
 return @ "Like" ;
 }

 - ( BOOL ) isAccessibilityElement {
 return YES ;
 }

Focus too fine/Focus merging

Focus merging refers to combining views with the same purpose, the same operation, or a combination of meanings. When the focus is not merged, the problem is described as "merging focus". For example, the friend's avatar and nickname in the friend list should be merged into one focus. Both have the same function, which is to jump to the personal profile interface.

"Solution": Set isAccessibilityElement to YES at the superView where focus needs to be merged.

 self . isAccessibilityElement = YES ;

Wrong focus information (unclear purpose, redundant information, etc.)

Unclear focus purpose means that the focus on the page only reads out the basic copy, but does not read out the type of the control (button, tab bar, link, etc.), or does not read out the events that will occur after the focus response (double-tap to enter the details page to play a video, etc.).

"Solution": Refer to the product's accessibility labeling specifications. Adjust the accessibilityLabel, accessibilityTraits, accessibilityHint and other fields of the problem focus so that the focus information is read accurately.

 /// Button
 // When the Like button is not selected
 self . accessibilityTraits = UIAccessibilityTraitButton ;
 self . accessibilityHint = @ "Double-tap to like with xxx people" ;

 // When the Like button is selected
 self . accessibilityTraits = UIAccessibilityTraitButton | UIAccessibilityTraitSelected ;
 self . accessibilityHint = @ "Double-tap to cancel like" ;

 ///TabBar (including top Tab and bottom Tab)
 // When the object is TabBar
 self . accessibilityTraits = UIAccessibilityTraitTabBar
 // The selected TabBar
 self . accessibilityTraits = UIAccessibilityTraitTabBar | UIAccessibilityTraitSelected;

Improper focus

The focus is lost after the page is refreshed, the focus is lost after the view or control is operated, and the focus stays in an inappropriate position when a new interface is opened. The problem is described as "improper focus stays". For example, after entering some pages, the focus stays in the original position by default, or stays on the return button of the new page.

"Solution": Set the VoiceOver focus to a specific control when entering the interface. You can use UIAccessibilityPostNotification to send a notification to a specific control to change the VoiceOver focus.

 @implementationMyViewController ​ ​
 - ( void ) viewDidAppear :( BOOL ) animated
 {
 [ super viewDidAppear : animated ];

 UIAccessibilityPostNotification ( UIAccessibilityScreenChangedNotification ,
 self . myFirstElement );
 }
 @ end

Focus order error problem

When the focus order does not conform to the logical order, it is usually described as: "The focus order does not conform to the logical order"; when the focus order does not conform to the visual order, it is described as "The focus order does not conform to the visual order".

「Solution」: Manually set accessibilityElements. In extreme scenarios, you can actively implement the UIAccessibilityContainer protocol.

 /// Normal scenario. Do not use in TableView Cell

 self . accessibilityElements = @ [ View1 , View2 , View3 ];

 /// In complex scenarios, be sure to implement all three methods, otherwise unexpected problems may occur

 - ( NSInteger ) accessibilityElementCount {
 return 3 ;
 }

 - ( nullable id ) accessibilityElementAtIndex :( NSInteger ) index {
 return Views [ index ];
 }

 - ( NSInteger ) indexOfAccessibilityElement :( id ) element {
 return [ Views indexOfObject : element ];
 }

The problem of jumping around when the focus is selected

This problem mainly occurs in CollectionView. The general phenomenon is that when you adjust the selection state of a Cell and reloadData, the focus will first shift to a random position (usually the next Cell after the selection), and then jump back to the correct position.

"Solution": Use reloadData of UICollectionView and UITableView with caution. When you need to refresh the selected state of a Cell, try to implement an updateVisibleCell yourself. ReloadData will cause random reuse of Cells, which will eventually cause focus jump exceptions.

 ❌
 [ self reloadData ];

 ✅
 [ self updateVisibleCells ];

 - ( void ) updateVisibleCells {
 NSArray < UICollectionViewCell * > * visibleCells = self . collectionView . visibleCells ;
 [ visibleCells enumerateObjectsUsingBlock : ^ ( UICollectionViewCell * _Nonnull obj , NSUInteger idx , BOOL * _Nonnull stop ) {
 // Todo something
 }];
 }

Floating window issue / Still accessible when focus is covered

In Xigua Video, floating windows exist in many different forms, such as XIGALert (privacy pop-up window, etc.), XIGToast (network anomaly, etc.), half-screen floating window (keyboard, clarity panel, speed panel, etc.). When a pop-up window appears, users can switch the focus to an invisible view by swiping left or right. The related events generated by these views may cause the program to crash and cause unexpected things to happen.

"Solution": The general approach is to make the underlying control elements lose focus to achieve the effect of shielding the underlying elements, set the gray darkened area as a single large focus, set the label to "Close" or "Collapse" and other prompts, and double-click to close the floating layer. When a pop-up window or layer pops up, the first control of the upper container (such as the title or notification content) automatically gets the focus, and visually impaired users do not need to switch focus twice, which is more in line with the user experience of visually impaired users and more humane.

 self . accessibilityViewIsModal = YES

Setting properties for the mask will make the views of the same level as the mask not respond to VoiceOver, but the subviews of the mask can respond. As shown in the figure below, if you want E to be accessible and CD to be inaccessible, you need to set B's accessibilityViewIsModal to YES.

Nested container ScrollView cannot follow the scrolling problem

In UICollectionView, a UIScrollView or UITableView is nested. When the focus slides in TableView or ScrollView, automatic scrolling will not occur, resulting in a white screen on the page.

"Solution": Use accessibilityActivate, and when the focus is on a Cell, manually call scrollToRowAtIndexPath: atScrollPosition: animated:. If a Cell contains multiple functions, the Cell also needs to be adapted for the rotor.

 - ( BOOL ) accessibilityElementDidBecomeFocused {
 [ self . tableview scrollToRowAtIndexPath : indexPath atScrollPosition : UITableViewScrollPositionNone animated : NO ];

 return YES ;
 }

Horizontal sliding and dragging problem (gesture replacement problem)

There are many scenarios in the application that require horizontal swipe gestures, such as horizontal swipe to delete, horizontal swipe to edit, etc. Such events cannot be responded to when Voice Over is turned on. Therefore, some other means are needed to enable users to access the horizontal swipe function.

「Solution」: Use rotor to modify the function.

 UIAccessibilityCustomAction * action1 = [[ UIAccessibilityCustomAction alloc ] initWithName : @ "Enter details page" target : self selector : @ selector ( action1 )];
 UIAccessibilityCustomAction * action2 = [[ UIAccessibilityCustomAction alloc ] initWithName : @ "Delete this video" target : self selector : @ selector ( action2 )];
 UIAccessibilityCustomAction * action3 = [[ UIAccessibilityCustomAction alloc ] initWithName : @ "Hide this video" target : self selector : @ selector ( action3 )];

 self . accessibilityCustomActions = @ [ action1 , action2 , action3 ];