Voice-activated and audio-enabled technologies are now an everyday part of life for many people. They’re also striking a chord in the culture sector as museums and digital artists seek to offer inclusive experiences to vision-impaired users, as Rebecca Mileham found out at the latest workshop in the Inclusive Digital Museum Innovation project.
In the last year or so, I have changed the way I catch up on the news. I use a newspaper app, but rather than peering at my phone screen, I tap a button or two and a voice reads the articles to me. It’s the same when I ask my hands-free smart speaker to play music or check train times, or when I get directions from my navigation app. It’s convenient, seamless – and becoming the new normal.
What does this kind of technological accessibility mean for the culture sector? Inclusive technology for vision-impaired museum visitors was the theme of the third international workshop in the Inclusive Digital Museum Innovation project, which took place on 8th November 2022. Partners and delegates from organisations in Korea and the UK gathered online to explore the physical and digital challenges that stand in the way of digital equality.
Audio tools in digital culture
Yuwon Lee from Korea’s Banjiha Games company shared a case study from the cutting edge of digital culture. South Korea has one of the most advanced computer games industries in the world, with two thirds of citizens enjoying digital activities from E-sports to mobile gaming, consoles to virtual reality.
As Yuwon explained, his company had recently published a new text-based mobile adventure game. Seoul 2033 depicts a city devastated by nuclear war, in which you and your family have to survive and defeat your enemies. ‘We came up with a story where players explore a world and select the story they experience,’ explained Yuwon. Every game is different and leads you down a different path depending on your choices.
The game had a positive response, and the Banjiha team planned to develop a version for vision-impaired players. But then came the surprise. A visually impaired user told them that they were already playing the game using a function called VoiceOver on their iPhone. ‘VoiceOver detects objects on the screen, and then reads out the features to the user,’ said Yuwon. A similar system called VoiceAssist exists on Android phones too.
Delighted with this discovery, the team made tweaks to optimise the experience: ‘Some of the features were interfering with each other, so we streamlined them, adding voice labels to the user interface icons. We also gave descriptions to some of the elements, for example so that instead of saying ‘image’ it read out ‘close button’.’
The company then took on a vision-impaired story designer. ‘We wanted to diversify our team, but also simplify our work process’, Yuwon explains. ‘The story designer draws on her experience to help the users.’ A small but significant number of the daily active users for the game now interact using the VoiceOver feature.
What about feedback from vision-impaired players? ‘We were stunned by how excited users were,’ says Yuwon. ‘Strategies can be just as fun as the visual features in a game – it can all be done using the voice. The response from players was that finally they have a game they can enjoy with family and friends.’
Museum access via voice
Whether visiting a museum with friends or solo, for vision-impaired visitors the quality of the available pre-visit information is crucial, according to Barry Ginley. Barry spoke at the workshop from his perspective as a highly experienced disability and access consultant, and as someone with visual impairment.
‘A key factor is for museums to develop their websites to the international WCAG 2.1 guidelines that ensure vision-impaired people can use screen reading technology,’ he said. ‘It avoids lost business and visits – most people will quickly click away from an inaccessible museum website.’
Once at a museum or gallery, there are two apps that are transforming the experience of vision-impaired visitors.
Waymap is a phone-based navigation app that maps both outdoor and indoor environments. A UK project, it emerged from experiments in audio navigation by the Royal Society for Blind Children, alongside breakthrough advances in indoor location technology.
‘As more people use the app and visit museums, it learns from the data provided. It’s very good at mapping complex old buildings, and it navigates you to within half a metre of your selected objects or locations using spoken guidance,’ explains Barry. The app also updates automatically from user data if, say, a lift is out of order. Although take-up is in its early stages, Waymap has signed an agreement to cover all Washington DC’s transport network, streets and public buildings.
Smartify is an app that ‘opens up the museum’s cases’ as Barry describes it, by making an institution’s content management system accessible via a visitor’s phone. Smartify has been devised for museums and collections, and – prior to the pandemic, at least – was gaining traction.
‘Once the Smartify system is enabled in a museum, and a visitor has the app, it’s simple,’ says Barry. ‘You point your camera towards an artwork and the app picks out the object and gives label information, or alt text for an image. If audio or sign language interpretation is available, the app can provide that.’
Both these apps have wide-ranging potential – and if they were to join forces, it would be even better. ‘Museums can be drivers for this kind of change,’ Barry points out.
Strategy, awareness and innovation
Voice-enabled apps and technologies have clear benefits for all kinds of visitors. So, as Theano Moussouri, one of the Digital Inclusion project leaders, asked, what prevents museums from joining the dots and providing such access for visitors?
Barry Ginley reflected that younger staff are sometimes more open to using digital technology than more established colleagues. But he added, ‘As we all become more digitally aware, accessibility will improve.’
Seo Hye-ran, former director of the National Library in Korea, had direct experience of creating audio content. ‘I volunteered to make audiobooks for the blind in the library’, she said. ‘There was no problem with the text but it was a pity that I couldn’t deliver picture information like maps and portraits well. There are still areas that are technically difficult to solve to accurately convey content.’
One delegate raised the interesting question of whether we are seeing a splintering of audience experience, as different kinds of provision are created for people with different needs. Others felt that the new experiences reflected the reality that seeing something is only one way of interacting with it. More may be to come as we move towards haptic technology, where a glove can provide a virtual touch sensation of an object in a case.
Jungwha Kim, Founding Director of Seoul Museum of Craft Art, commented that a museum’s aim was often to provide a comparable experience for vision-impaired visitors and others, with tactile exhibits and audio guides. However, the biggest challenge was in securing the dedicated budget and personnel to deliver. Her view was that structural change is the only way to achieve this, with responsibility placed at senior level.
Rafie Cecilia, Research Fellow at the UCL Institute of Archaeology, and session chair, agreed that while some larger UK museums have dedicated inclusion staff, this was rare across the country. This left colleagues in smaller museums trying to do a huge amount of access work as well as act as advocates.
Fiona Slater of the Science Museum Group noted that museums and galleries often want to innovate and use the latest technology – particularly when a new project allows the resources to try something new. The audience, meanwhile, may prefer consistency, using something simpler and more familiar.
The agility of the games industry, and its receptiveness to user feedback, seems to have helped Banjiha Games to innovate. ‘Like museums, we are limited in resources,’ says Yuwon Lee.
But he saw great opportunities ahead for museums and digital games alike. ‘There are audiences waiting for cultural experiences, and great empty spaces that we all need to walk into and fill. What we offer carries enormous value for people.’
And voice-activated and audio-enabled technologies are a resounding success story that is here to stay. ‘Before working on Seoul 2033, I didn’t know VoiceOver existed,’ admits Yuwon. ‘Having it in the game has made a lot more people aware of it, which I’m quite proud of. For me, it’s the new normal.’
To play Seoul 2033
On Android: https://play.google.com/store/apps/details?id=com.banjigamaes.seoul2033_global&hl=ko&gl=US
On iPhone: https://apps.apple.com/kr/app/seoul-2033/id1585938647