You have probably heard about AI-based virtual speech assistants. While being far from perfect... I mean really far... they have some useful features, if just to avoid looking at your phone for inquiring about a semi accurate info about the weather / temperature or for creating a shopping list, you forget to look at it when you are at the grocery store.
But seriously, the idea of having a virtual assistant, like Jarvis, is intriguing. So, I looked into that matter a bit, searching for an alternative character to interact with like e.g. a B1 battle droid. However, while you could get yourself e.g. a Samuel L. Jackson voiceover, the options were pretty limited.
Having looked at text-to-speech conversion earlier for creating an app-based narrator, and having some programming experience, I considered creating my own voiceover. During that time, our working group was asked to contribute towards recruiting additional students for the physical chemistry department by offering insights into the studies in our research field.
While thinking about creating a high school style video with one of our team members talking in front of a blackboard, the idea came up to incorporate my work on my own voiceover into this video. I gathered some contributions from my coworkers to have – in addition to my input – various materials available. And after roughly three weeks of juggling dependencies for Python packages, I finally managed to install the text-to-speech program and started training my first model – and it turned out that it was surprisingly easy to generate an artificial voice from an arbitrary short audio file. On the one hand, having such features readily available is fascinating. On the other hand, the possibilities for misuse are concerning.
To make another side story short, my narrator of choice was Joachim Höppner. He was the storyteller of the first Lord of the Rings audio books, in combination with being the voice actor of Gandalf, giving him as a storyteller a special flare. Not being able to generate copyright restricted content, I generated an arbitrary mage using the university’s RUB@GPT AI, which seemingly does not have any similarities with the wandering wizard.
To not just have a static image I considered animating the speaker by using a row of different images altering the speaker. When searching for the best approach for realizing that, I encountered another AI. HeyGen, which allows to generate facial expressions and lately also gestures just based on an image and an audio file.
It was interesting to test out the capabilities and limitations with the controversial opinions on AI. Now that I have used quite a few different types of AI, I would say that all of it would have worked better with a person, e.g. doing the narration himself as well as doing the animation. Former would have been quite a bit less work and latter quite a chunk more.
Besides the overall editing I was focusing on including all the fun and interesting aspects I experienced during my studies with light-matter interaction, a phenomenon that also regularly occurs in nature, e.g. given for polar lights, rainbows or sunsets. If you are interested in hearing about physical chemistry, engaging concepts with light and lasers, or just want to check out the editing I was talking about in this blog, have a look at it. As they say: a picture is worth a thousand words. And a video has a lot of them.
I would recommend the German version.
German video version on YouTube
English video version on YouTube
-----------------------------------------------------
About the author
