Implementing Audio Graphs in SwiftUI

Preface

Graphs are one of those complicated things when it comes to accessibility. iOS 15 introduces a new feature called Audio Graphs.

Next we’ll learn how to build an audio representation for any SwiftUI view by using the accessibilityChartDescriptor view modifier, rendering a chart like a custom bar chart view or an image.

DataPoint Structure

Let’s start by building a simple bar chart view in SwiftUI that displays a set of data points using vertical bars.

 struct DataPoint: Identifiable { let id = UUID() let label: String let value: Double let color: Color }

Here we have a DataPoint structure that describes a bar in a bar chart view. It has an id, a label, a value, and a fill color.

BarChartView Structure

Next, we can define a bar chart view that accepts a set of DataPoint structure instances and displays them.

 struct BarChartView: View { let dataPoints: [DataPoint] var body: some View { HStack(alignment: .bottom) { ForEach(dataPoints) { point in VStack { RoundedRectangle(cornerRadius: 8, style: .continuous) .fill(point.color) .frame(height: point.value * 50) Text(point.label) } } } } }

As shown in the example above, we have a BarChartView that takes a set of DataPoint instances and displays them as rounded rectangles of varying heights in a horizontal stack.

ContentView Structure

We can easily build a bar chart view in SwiftUI. Next, let’s try using our new BarChartView with some sample data.

 struct ContentView: View { @State private var dataPoints = [ DataPoint(label: "1", value: 3, color: .red), DataPoint(label: "2", value: 5, color: .blue), DataPoint(label: "3", value: 2, color: .red), DataPoint(label: "4", value: 4, color: .blue), ] var body: some View { BarChartView(dataPoints: dataPoints) .accessibilityElement() .accessibilityLabel("Chart representing some data") } }

Here, we create a sample array of DataPoint instances and pass it to the BarChartView. We also create an accessible element for the chart and disable accessibility information for its child elements. To improve the accessibility experience of the chart view, we also add accessibility labels.

Finally, we can start implementing the audio graph functionality for our bar chart view. The audio graph is available through the knob menu. To use the knob, rotate two fingers on the screen of your iOS device as if you were turning a dial. VoiceOver will speak the first knob option. Continue rotating your fingers to hear more options. Release your fingers to select the audio graph. Then slide your fingers up and down on the screen to navigate.

Audio charts allow users to understand and interpret chart data using audio components. VoiceOver plays sounds with different pitches as you move over bars in the chart view. VoiceOver uses high pitches for larger values ​​and low pitches for smaller values. These tones represent the data in the array.

Implementing the protocol

Now, we can discuss how to implement this functionality in BarChartView. First, we must create a type that conforms to the AXChartDescriptorRepresentable protocol. The AXChartDescriptorRepresentable protocol has only one requirement, which is to create an instance of the AXChartDescriptor type. An instance of the AXChartDescriptor type represents the data in our chart, presented in a format that VoiceOver can understand and interact with.

 extension ContentView: AXChartDescriptorRepresentable { func makeChartDescriptor() -> AXChartDescriptor { let xAxis = AXCategoricalDataAxisDescriptor( title: "Labels", categoryOrder: dataPoints.map(\.label) ) let min = dataPoints.map(\.value).min() ?? 0.0 let max = dataPoints.map(\.value).max() ?? 0.0 let yAxis = AXNumericDataAxisDescriptor( title: "Values", range: min...max, gridlinePositions: [] ) { value in "\(value) points" } let series = AXDataSeriesDescriptor( name: "", isContinuous: false, dataPoints: dataPoints.map { .init(x: $0.label, y: $0.value) } ) return AXChartDescriptor( title: "Chart representing some data", summary: nil, xAxis: xAxis, yAxis: yAxis, additionalAxes: [], series: [series] ) } }

All we need to do is conform to the AXChartDescriptorRepresentable protocol and add a makeChartDescriptor function that returns an instance of AXChartDescriptor.

First, we define the X and Y axes by using the AXCategoricalDataAxisDescriptor and AXNumericDataAxisDescriptor types. We want to use string labels on the X axis, that's why we use the AXCategoricalDataAxisDescriptor type. In the case of a line chart, we will use the AXNumericDataAxisDescriptor type on both axes.

Implementing Line Chart

Next, we define the points in the chart using the AXDataSeriesDescriptor type. There is an isContinuous parameter that allows us to define different chart styles. For example, for a bar chart it should be false, while for a line chart it should be true.

 struct ContentView: View { @State private var dataPoints = [ DataPoint(label: "1", value: 3, color: .red), DataPoint(label: "2", value: 5, color: .blue), DataPoint(label: "3", value: 2, color: .red), DataPoint(label: "4", value: 4, color: .blue), ] var body: some View { BarChartView(dataPoints: dataPoints) .accessibilityElement() .accessibilityLabel("Chart representing some data") .accessibilityChartDescriptor(self) } }

As a last step, we use the accessibilityChartDescriptor view modifier to set an instance conforming to the AXChartDescriptorRepresentable protocol as the one describing our chart.

Example screenshots:

Summarize

The audio chart feature is a major improvement for visually impaired users. The great thing about the audio chart feature is that you can use it with any view you want, even image views. Just create an instance of the AXChartDescriptor type.