Mobility of the Future: How Audiovisual AI Will Improve the Driving Experience

AI – sometimes with simple, sometimes with more complex approaches – helps with creating playlists, navigating, unlocking our phones through facial recognition, automatically correcting our messages and emails, or finding the best restaurants nearby, to name just a few examples. Algorithms are also used in social media to show us content and people considered the most interesting for us based on various data, or – thinking of generative AI – images, speech, or videos that are completely new and individually generated by AI. However, there are also restrictions: The EU AI Act, expected to be adopted in 2024, will regulate the use of AI more strictly and set new standards for the use of AI at the European level. Despite recurring concerns about data protection and security, AI will remain a part of our society in the future. Overregulation should be avoided, however, to maintain Europe's competitiveness in the long term.

Optimized Production and Safer Driving

When we talk about "AI" today, we often refer to deep neural networks, which are modeled after the neurons in the human brain to process information and learn in a human-like way. These can be used in a variety of fields. Diverse applications of AI can be found in the field of mobility. AI can be used during production to make processes more efficient. Through predictive maintenance, for example, maintenance work is optimized, significantly reducing the risk of production downtime. However, the innovations that are directly noticeable and thus particularly interesting for consumers as well as professional drivers are increasingly found in driving or future autonomous driving. The integration of AI in vehicles promises not only increased safety but also a completely new driving experience.

The use of multiple sensors in the vehicle, such as audio, video, and others, allows the generation of relevant data so that appropriately trained AI systems can recognize potential danger situations early and alert drivers or automatically intervene directly. In practice, this means: adjusting driving style, recognizing health-related safety impairments, adapting route planning, personalizing entertainment and infotainment, or increasing cabin comfort for drivers and passengers. Additionally, vehicle diagnostics can be improved to promptly indicate potential mechanical issues. Overall, this contributes to increasing road safety. Especially the individually AI-optimized route planning – for example, also coordinated with the personal calendar and the driver's respective mood – can generate significant added value. By simultaneously analyzing traffic data in real-time, proactively planning temporal components, and potentially planned daily activities such as shopping, picking up family members, leisure activities, etc., in connection with the driver's or driver's mood, appropriate routes can be suggested, fuel consumption optimized, or driving behavior adjusted through recommendations. This not only contributes to a much more pleasant driving experience but also reduces costs and negative environmental impacts on the road while simultaneously increasing safety through optimized driving behavior.

Language and Voice as Control Elements of the Future

Particularly exciting in the automotive industry is the use of voice and speech-based AI. Modern vehicles have voice control systems that allow the driver to effortlessly activate various functions through spoken commands. From adjusting the temperature to navigating to a specific destination, many applications can be controlled by simple verbal commands. Additionally, AI can learn individual driver preferences and adjust accordingly. For example, by automatically configuring the lighting, preferred music, temperature, or optimal vehicle settings.

But that's not all: AI-supported technology can not only recognize what is said but also analyze how it is said. If the vehicle is equipped with technologies using AI-supported voice analysis, various voice biomarkers can be determined that provide insights into characteristics and states such as fatigue or excitement. Based on psychological models, the degree of arousal, valence, and dominance of the voice's acoustic features are determined. This, in turn, allows for direct or indirect interventions in particularly critical moments. Studies show that clinically validated voice biomarkers, such as fatigue, are objectively measurable. In the future, the car could suggest to the driver not only at pre-planned times to take a break or drink a coffee but also when it specifically measures exhaustion biomarkers, if one wishes to use this function. On-board entertainment could also be automatically adjusted to the (emotional) state of the occupants by optimizing interior lighting, fragrances, or musical performances.

Such innovative technologies aim to increase not only user-friendliness but also the safety and comfort of drivers. As AI technology allows real-time analysis of conditions and thus a continuous approach, signs of distractions, fatigue, stress, and other potential hazards are directly detected. In these situations, an appropriate response can be triggered in the vehicle, such as external warnings or an automated increase in distances. It is also conceivable that the predefined preference of the corresponding reaction, when detecting the voice biomarker, is already defined in advance by the vehicle occupants. In addition, systemic inquiries or feedback mechanisms could be implemented to increase safety in the vehicle and avoid potential accidents.

What does that mean?

The progressive integration of AI into the automotive industry leads to many exciting innovations that we can look forward to in the coming years. All these technologies have a clear mission: to elevate user-friendliness, safety, and comfort for drivers to a new level. AI-supported technologies in the automotive sector are not just a convenient gimmick that automates manual tasks; they represent a significant step toward safer and more user-friendly driving experiences, which can also contribute to climate protection through optimized driving.

The article was written by Dagmar Schuller. As CEO and co-founder, Dagmar Schuller led the innovation leader for AI-based audio analysis, audEERING, from a simple UG to a multimillion-dollar company with over 70 employees. She is an expert in digital strategy and innovation and has been working with artificial intelligence, machine learning, and big data for over 25 years. The benefits of AI-based audio analysis for people are always her focus. Therefore, she and her team develop applications for ever-new areas: from healthcare to robots to AR & VR technologies. Schuller is also deputy regional director of the AI Federal Association e.V. Bavaria, vice president of the IHK Munich and Upper Bavaria, and professor of business informatics & digital entrepreneurship at the Landshut University of Applied Sciences, Bavaria.