Exposing hidden actions on Android

Exposing “hidden” actions to accessibility services used to require providing a custom AccessibilityDelegate or using a third-party library like Novoda’s accessibilitools.

No longer! There’s a new API dropping in the latest version of AppCompat (1.1.0) which includes a handy way to serve this information to the system with minimum effort.

What is it?

The function adds additional information to a View which can be read by Android accessibility services.

It associates a View with an “action”, where an action consists of a label and executable block of code.

Accessibility services, like TalkBack, can present these actions to users in a way that’s familiar to them. The GIF shows two actions, “watch” and “download”, which have been associated with a View, and how a user can access them with TalkBack:

How do I use it?

Use ViewCompat to set an action on a View, passing the label for the action, and the action to perform:

val watchLabel = getString(R.string.action_watch)
ViewCompat.addAccessibilityAction(itemView, watchLabel) { _, _ ->
    onWatchClicked()
}

We’ll repeat this for each action we need to associate:

val downloadLabel = getString(R.string.action_download)
ViewCompat.addAccessibilityAction(itemView, downloadLabel) { _, _ ->
    onDownloadClicked()
}

How does it work?

It uses an AccessibilityDelegate under the hood to add actions to AccessibilityNodeInfo.

Here’s how we used to do it in accessibilitools, with a Menu:

@Override
void onInitializeAccessibilityNodeInfo(View host, AccessibilityNodeInfoCompat info) {
    super.onInitializeAccessibilityNodeInfo(host, info);
    for (int i = 0; i < menu.size(); i++) {
        MenuItem item = menu.getItem(i);
        val action = new AccessibilityActionCompat(
            item.getItemId(),
            item.getTitle()
        )
        info.addAction(action);
    }
    usageHints.addClickEventUsageHints(host, info);
}

The Novoda version uses the Menu resource paradigm which is familiar to Android developers.

On the client side, this meant creating a MenuItemClickListener, inflating a Menu, and setting an AccessibilityDelegate ourselves:

MenuItemClickListener menuItemClickListener = ...
Menu actionsMenu = ActionsMenuInflater.from(view.getContext())
        .inflate(R.menu.tweet_actions, menuItemClickListener);
UsageHints usageHints = new UsageHints(view.getResources());
usageHints.setClickLabel(R.string.tweet_actions_usage_hint);

AccessibilityDelegateCompat delegate = new ActionsMenuAccessibilityDelegate(
        menu,
        menuItemClickListener,
        usageHints
);

ViewCompat.setAccessibilityDelegate(view, delegate);

Compared to this, the AppCompat API is a nifty alternative. It uses View tags which allows it to add actions over multiple calls without passing them all at once, but we’re limited to 32 actions per View.

Should be enough.

When should I use this?

This is useful when we have actions on a View that aren’t accessible with a simple click—e.g. a swipe-to-delete gesture.

See the “Single-action elements” section in “What’s Next? A Practical Introduction to Accessibility on Android” for more ideas on when this might be useful.