Mihup.ai helps cars talk the talk

In India, it’s no secret that consumers are now getting used to the idea of digital voice assistants in premium cars. Digital voice assistants have been around in cars since 2004 in overseas markets, and since then the feature has been liked by users looking for high-quality products and more diverse uses for them. For the uninitiated, think of it as the Alexa of cars, says Tapan Barman, Co-founder of Mihup, a tech start-up that has designed and developed AI-based voice assistants for automotive brands like Tata Motors. The deep-tech company specialises in conversational and generative AI-powered applications for the automotive sector since 2019.

While voice AI technology enables seamless and intuitive communication between the vehicle and its driver, there have been challenges for the Indian ecosystem where you have hundreds of languages and thousands of dialects to deal with. For successful application in a country like India, a robust natural language understanding system is essential for interpreting driver commands accurately to help reduce the likelihood of misunderstandings or misinterpretations that could lead to unsafe situations.

This was a challenge that prompted Tapan Barman and Biplab Chakraborty, Co-founders of Mihup to establish their start-up in 2016, with the mission to bridge the gap between human and technology, with a Make in India AI voice assistant applications.

Barman says that it took the company a few years to commercially develop a product that’s vocabulary-based and has a phonetic imprint approach. Mihup’s software is able to interpret not just the words used in communication, but the emotions and context behind them as well, claims Barman.

This approach to AI has been path-breaking and has helped Mihup carve a niche in the voice application bolstered by a focus on creating a product that is both multilingual and vernacular in a diverse language/dialect set-up. Barman claims that the system that they have developed successfully recognises 50 Indian dialects like Hindi and Hinglish; Tamil and Tamilish, Bengali and Benglish, Telugu, Marathi, Punjabi, Haryanvi, Kannada and Marwari. Barman claims that the latest product iteration has factored in many of the limitations that ChatGPT and other large-language model (LLM) applications have been known to offer, including fabricated or fake information. Likewise, OpenAI is being targeted these days in a number of defamation and copyright infringement lawsuits. 

Hybrid nature

One of the key features of the Mihup Virtual Agent is its hybrid nature, allowing operation even in the absence of an internet connection. The bot is integrated into the car's infotainment system and data is secured as it is not saved anywhere. Unlike Alexa or Siri, which necessitate internet connectivity, Mihup functions independently of the internet and it can execute all tasks that do not rely on data retrieval.

That’s not all, by understanding and adapting to individual preferences and driving habits, these systems can provide tailored recommendations, route suggestions, and operational guidance, optimising driving efficiency and minimising downtime.

New market opportunities

The Automotive Voice Recognition System market size which is roughly USD 3.06 billion in 2024, is expected to reach USD 6.09 billion by 2029, growing at a CAGR of 14.77% during the forecast period (2024-2029), according to market analyst Mordor Intelligence. 

According to Barman, several major truck makers in India are also considering the idea of including a voice assistant in their vehicles. While specific names aren't disclosed yet, he suggests that this trend isn't limited to a single manufacturer but is a growing consideration across the industry.  On advancements in application of conversational AI, Barman says it can also be adapted for communication in commercial vehicles (CVs) and can be extended to 2-wheelers also. 

Like in cars, voice AI based technology in CVs will enable seamless and intuitive communication between the vehicle and its driver. Equipped with advanced speech recognition and natural language processing capabilities, these systems allow drivers to interact with their trucks using simple voice commands, eliminating the need for manual inputs and reducing distractions on the road. One of the key advantages in conversational AI for trucks is the ability to provide real-time updates and safety alerts to drivers. Through voice prompts and responses, the vehicle can alert drivers to potential hazards, upcoming road conditions, and maintenance requirements, enabling them to take proactive measures and make informed decisions while on the road, says Barman. 

Barman says the journey has been nothing short of transformative, upon partnering with Tata Motors in 2019. Since then, the collaboration has grown from initial deployments to widespread integration across numerous models and applications. “Initially, we launched the Mihup AVA voice assistant on edge in just a few models which has now expanded across vehicle models and various use cases. The journey has been marked by continuous innovation and adaptation to meet the evolving needs of drivers and passengers alike,” says Barman.  Recently, the company says it has taken a step forward with the launch of the hybrid model of Mihup AVA. This new offering combines the best of cloud-based and on-edge technologies, enhancing the voice assistant's capabilities and expanding its applications and hence considered as a significant development. The model allows for seamless updates and integration of the latest AI advancements and further expands the scope of the voice assistant. “We are excited about the introduction of custom LLM and Gen AI use cases in our voice assistant which will revolutionise the conversational experience inside vehicles, across the entire automotive landscape of India,” says Chakraborty.

Software upgrades possible

As with any software application, the versions get outdated or obsolete with new versions and keeping that aspect in mind, the Mihup software can also be upgraded. Barman says this is possible due to its advanced learnability model that means Mihup can also learn the typical usage pattern and preferences of the user from their day-to-day use and push the relevant updates for a customised user experience.

Safety approach

Mihup claims that its voice assistant system aims to reduce driver distractions by allowing users to control vehicle functions through voice commands and minimising the need for touchscreens. This approach aligns with safety directives from organisations like NCAP, emphasising the importance of reducing distractions to enhance road safety.  Explains Barman: “Our approach at Mihup.ai focuses on minimising distractions by integrating advanced conversational AI into vehicle systems. Having to press multiple screens and buttons to get even the most basic tasks done is inherently distracting. Voice is the most natural form of communication and when you have a technology that can understand your spoken words almost as well as another human, any debate over its applicability becomes redundant. This technology allows drivers to manage vehicle functions through voice commands, significantly reducing the need to interact with touchscreens for basic functions.”

Integration of technologies 

The possibility of integration of technologies like biometric recognition and gaze detection into automotive solutions, further makes them the go-to choice as safety enhancers. Barman says this technology ensures that vehicle functions are accessible only to authorised users, enhancing security. Biometrics can adjust vehicle settings such as seat position, mirror angles, and driving preferences automatically, based on the identified driver, creating a safer and more personalised driving experience. By monitoring the driver’s gaze, this technology can alert if the driver is not paying attention to the road. If the system detects that the driver is distracted or drowsy, it can issue alerts or take precautionary actions, such as reducing the vehicle’s speed or even safely pulling over added Chakraborty. "We are dedicated to pushing the boundaries of what conversational AI can achieve in automotive safety. By incorporating these technologies, we not only meet but also exceed safety standards, ensuring our vehicles are safer, smarter, and more connected. We are proud to be at the forefront of this technological evolution, delivering solutions that improve safety outcomes and driving experiences for all road users,” Barman said. 

This feature was first published in Autocar Professional's June 1, 2024 issue.