Voice in media: the AI voices of Jan de Hoop and Philip Freriks speak to us
open voice #07 - Media & entertainment
Moderator and co-organizer of Open Voice, Maarten Lens-FitzGerald, presented his State of Voice at the Cognizant Digital Studio in Amsterdam. He's Global Head of Voice at Nodes Agency. Lens-FitzGerald showed that in the Netherlands we are still in the phase of innovators and early adopters in the field of smart speakers. In America, which has been at the forefront of this for some time now, the consumer market is moving towards the early majority.
There has been a lot of work on privacy and voice lately. Google, Apple and Amazon were in the news because of 'leaks' in their voice assistants. "But by now everyone has said sorry and all providers have worked on improving privacy," said Maarten Lens-FitzGerald.
.
Lens-FitzGerald also said that the Voice Committee of the trade association DDMA is finalising a legal primer. This document can serve as a fact checker to remove the legal hurdles in the roll-out of a voicetrack. "Useful when the compliance and legal departments ask you what you're doing." They are also working on UX guidelines.
Seamless cooperation
Amazon came up with the Voice Interoperability Initiative. This agreement with a group of three companies aims to ensure that voice-assistant Alexa can work seamlessly with other voice-driven services. "A good initiative", said Lens-FitzGerald, "although I don't think it should be the industry itself that sets up such an agreement."
Lens-FitzGerald also pointed out that it is not necessary to use the platforms and speakers of the well-known, major players Amazon, Google and Apple. "You can also develop something quite easily by buying your own chipsets and working with white-label providers."
Learning from voice. This is what KRO-NCRV is doing in practice. Lieneke Grollé is an online content specialist at KRO-NCRV. She works in the innovation department of De Slimste Mens and is product owner Voice of De Slimste Mens.
Her job is to explore the possibilities of voice. So far, KRO-NCRV has deployed voice in three ways around four different TV programmes:
• Service: BinnensteBuiten
• Storytelling: Verhalen uit het Nest
• Entertainment: Willem Wever Challenge and De Slimste Mens
The happy path of Willem Wever
The Challenge around Willem Wever, the television programme for children, was relatively easy to set up, said Grollé. "It's a game with questions that you have to answer with true or false. Over the years, we've built up a huge database of questions. We put the Challenge on the website of Willem Wever to learn: what is the happy path in the user experience, what is the unhappy path? How can we optimize this?"
Grollé shared some figures about the use of the Willem Wever Challenge. By the end of 2018, more than 12,000 users have already played the Challenge more than 140,000 times. In August 2019, 1,400 people launched the action 16,000 times. "That means that a user comes back about 11 times in one month. We are also constantly adding new questions, so that it is worthwhile to return."
For BinnensteBuiten, a TV programme with inspiration for in and around the house, KRO-NCRV has developed a voice assistant with the voice of chef Sharon de Miranda. It is the first Dutch voice-controlled cooking assistant. She explains step by step how to prepare the recipes from the program BinnensteBuiten.
For the little ones: interactive listening stories
Almost every parent does it once in a while: put your child in front of the TV or the tablet, so that he or she can watch a programme in peace and quiet. But before you know it, they're stuck on the screen for a very long time. Grollé said that KRO-NCRV wants to gain experience with interactive listening stories for the youngest (between 4 and 7 years old). The broadcaster developed an interactive listening book entitled Verhalen uit het Nest. During the story, the little ones are presented with choices: 'What do you think brother and sister should do: go to the cat? Or run away?’
With this interactive voice-story, children learn more about the power of collaboration in a playful way. With their answers, they determine the line of the story. Grollé: "A lot of time has also gone into the sound design. The series Verhalen uit het Nest (Stories from the Nest) takes place in nature. The sounds of the forest and so on are very important for the atmosphere and the experience. The power of sound design should not be underestimated; that makes or breaks a story. ”
What creates tension?
Atmosphere is also a very important factor in the voice project on De Slimste Mens, said Grollé. The TV knowledge quiz with presenter Philip Freriks and jury member Maarten van Rossem is a concept. How do you capture that uniqueness in a voice application? "We developed a working prototype that we could test on people in the target group. What are the elements that create tension in the program? Important is the time pressure when answering the questions. The sound of a ticking clock. But also the humor. Maarten van Rossem regularly breaks into his stories. We also tested what the players of the voice application thought of these interruptions. We gave them a choice as to whether or not they wanted to hear his stories."
The sound effects, the soundtrack of the program, and of course the voice of Philip Freriks all contribute to the right experience of De Slimste Mens in the voice-application. "We just can't keep going to Philip to have him record the questions. That's why we're automating his voice. We are now testing whether we can use this to give the feeling that it is really Philip who is asking the questions." Soon the voice-app of De Slimste Mens will go live.
Learning from the experiences
KRO-NCRV does not spend any noteworthy publicity on the voice projects. As a public broadcaster, KRO-NCRV does not have any marketing objectives with this kind of projects, Grollé explained. How do you measure success, someone from the audience wanted to know. "It's especially important to learn from it," said Grollé. "We can use that knowledge in other projects as well. Our experience with the automation of Philip Freriks' voice can also be applied to projects in the public interest."
A little jealous of Netflix and Google
At the other end of the spectrum we traditionally find the commercial broadcasters. But Tom Jessen of RTL Nederland explained that these days, audiences and commercial people, together with some jealousy, more often look at parties such as Netflix and Google, which increasingly have the attention of the viewer and the listener.
Tom Jessen is project leader Voice at RTL. Together with his colleagues, he tries to counterbalance the influence of Netflix and upcoming platforms such as Disney+ and the Chinese tech giant Tencent with smart applications. "The storm has yet to erupt. It's exciting in that area.”
Catch the eye
Jessen focused on the innovative ways in which RTL offers the news with voice-driven services. The flagship is the RTL News. Other popular channels include RTL Z for business news, Bright for tech news and RTL Sport. "The central question is: how can we serve out the content in such a way that it becomes very easy to consume the RTL brand and to ensure that we stay on the radar? After all, we are a commercial company - we have to make money and that starts with the customer's attention."
"We started making audio updates for Google Home and Alexa. But the general public is not yet really on the smart speaker. More than a year ago, the innovation team came up with the idea of distributing the news via WhatsApp, a channel that people are already busy with on a daily basis. "The updates via WhatsApp turned out to be very accessible and quickly became popular, Jessen told us. "That's how simple an idea can be."
Bringer of the RTL News Update via WhatsApp is newsreader Jan de Hoop. "This way we can get the most personal presenter we have at RTL to tell us the news. The younger people – the largest target group we reach on WhatsApp is between 14 and 25 years of age – are also responding very positively.”
Niche content is popular
According to Jessen, Google's statistics show that niche content does relatively best. "Through Google, we reach between 2,000 and 3,000 daily play-outs with the Z Update. Bright is also doing well; news and sports are lagging behind. Our interpretation is that on news and sports we have some more competition from other channels." On WhatsApp, more than 20,000 people use the RTL news channels on a daily basis. "We just can't see how many people are actually eavesdropping on it," added Jesse.
Just as KRO-NCRV tries to automate the voice of Philip Freriks, RTL is also trying to 'capture' the sound of Jan de Hoop with the help of artificial intelligence. Jessen: "With that we are still testing, with the help of the company Flow.ai."
Jessen emphasized that this is another experiment. "It's a very interesting trajectory, but it also raises ethical questions. After all, what are you going to do with a tool like this if Jan de Hoop stops presenting? We do not yet have an answer to this question. And as a news organisation, you naturally want to continue to radiate reliability. People shouldn't start to doubt: is this real? If that goes wrong once, it can be detrimental to the RTL News brand. We are well aware of that."
Iwan Reuvekamp is Audio & Voice Innovation Manager at DPG Media. DPG Media, formerly De Persgroep, is the publisher of newspapers such as de Volkskrant, AD and Trouw, and owner of online platforms such as Tweakers, Independer, Autotrack and Reclamefolder.nl. Qmusic is also part of this.
About a year ago, Qmusic was one of Google Assistant's launching partners in the Netherlands, Reuvekamp said. "We had a voice-app: Talk to Qmusic. You could ask your smart speaker about all sorts of things about us: what's the name of the DJ? What's your address? But when we put it live, 95 percent of the users said: but I want to listen to Qmusic. And that wasn’t possible. You have to learn from that. So we started again and now you can listen to Qmusic through the smart speaker."
But a voice-command 'Talk to Qmusic' is actually quite a strange assignment, Reuvekamp said. "Better is: Play Qmusic, or Listen to Qmusic. But if you say that, you end up at TuneIn as a user. That's a smart company in the U.S. that streams all the radio stations in the world and enters into partnerships with advertisers. So you don't end up with Qmusic. ”
In addition, Google is the only one to give TuneIn the 'Media Action', Reuvekamp continued. “If you listen to our station via 'Talk to Qmusic', and you say to Google: turn on the lights, Google will not understand you. Such Media Actions will not work. We've been talking to Google about this for some time now, and they keep promising that such commands will work for us, but I hope it will happen someday."
A voice-app that DPG Media is also developing, but of which Reuvekamp – "I'm very transparent" – said that it's not yet certain that this will really come about, is 'Talk to de Volkskrant'. This application should make it possible to navigate through the newspaper with your voice, he explained. "If you say 'play the latest news' to a smart speaker so far, it's still secretly a linear radio product; a voice sums up the news, and after two minutes it's over. We've been looking for a way to create a news bulletin where you as a listener can interrupt an item if you want to know more about it – a kind of dialogue with the news source."
Reuvekamp and his colleagues found out in tests that this only works well if you always ask closed questions in the application. "Because people don't know how to navigate on a smart speaker." Users can answer 'yes' or 'no' to questions such as 'do you want to hear more about this?’ and ‘do you want to hear the next part?’ He added that no thought had yet been given to a voice representative of the de Volkskrant brand. "For the time being, we're still working with the voices of Google." Both the Qmusic app and the app for de Volkskrant were created in collaboration with Mirabeau, A Cognizant Digital Business.
Het Geluid: built in one day
A voice-app that is already live is Het Geluid van Qmusic. You can now also play the well-known radio game when you say to Google Home: 'Hey Google, Talk to The Sound of Qmusic'. Qmusic dj's Mattie & Marieke will take you on a quizmasters tour through the history of almost 15 years of Het Geluid. Reuvekamp explained how he almost single-handedly brought the radio format to the smart speaker. "I made a setup in Excel. It's actually just a structure as we know it from a voicemail system. Only now we call it 'dialogue flow'. In a day it was finished."
‘Talk to the Sound of Qmusic' is played about 30,000 times a month, Reuvekamp said. "We have yet to decide whether to proceed with this."
Reuvekamp mentioned some of the things he noticed about voice:
• Voice is not internet; you don't have the freedom to build what you want;
• Consumers still see the smart speaker mainly as a loudspeaker. If there were control buttons on it, they'd use them;
• Navigating with your voice is a quest. Don't make the actions too complicated;
• How can you easily discover useful applications? For example, how do I know if my energy company is on the smart speaker?
Voice ecosystem for the media
One of Reuvekamp's wishes for the near future is that there should be a voice ecosystem for the media. "In one conversation you can end up with the different actions. If, for example, the voice declares in a news item that the energy bills of the Dutch are rising, you could be given the choice: 'Do you want to look at your energy bill?' And then you should be able to go to Independer in one go to compare providers."
Reuvekamp gave us a tip not to focus to much on the short term. "Voice and audio are now the hype. In two years' time, it'll be a hologram sitting at the breakfast table with you. And who's coming in the car with you to inform you about the latest news. We're all gonna put squads on it to design that."
What is the voice of women in Voice? That's what Maaike Groenewege was wondering. She is an Information Analyst at NS and the initiator and ambassador of Women in Voice Nederland, a chapter of the international organization. Women in Voice connects women who work in voice and conversational technology.
The community in the Netherlands was founded in August and now has over 40 members. "We stand for empowerment of women in voice," said Groenewege. Women in Voice aims to help each other achieve their ambitions.
Groenewege explained that as a conversational designer she is actually indebted to two linguists we all know: the Brothers Grimm. The Germans discovered structural sound changes that occur everywhere. They were the founders of comparative linguistics, which now enables us to make computers understand natural language. Of course we know the brothers mainly from the fairy tale books; they collected folk tales and fairy tales. "They're a great example to me of how conversational AI revolves around technology and stories."
Podcasts Open Voice #07
Listen to the podcasts of these presentations. For example on Spotify or via Google Podcasts.
Stay up to date: follow the voice specialists of Mirabeau, A Cognizant Digital Business on their blog.
