Towards a Conversational LLM-Based Voice Assistant for Transportation Applications

I’m excited to share our latest project in the rapidly evolving world of automotive technology. As researchers at the University of Luxembourg, we’ve been working on integrating conversational AI into vehicles, and we’re happy to introduce KITT (Knowledge-based Intelligence for Transportation Technologies). KITT is an advanced voice assistant we’ve designed specifically for in-vehicle communication. Unlike general-purpose AI assistants, we’ve built KITT from the ground up to understand and assist with transportation-related tasks. Our system combines the power of large language models (LLMs) with real-time data from external sources to create a truly intelligent in-car companion. Key Features of KITT:

• Natural Conversation: We’ve implemented advanced speech-to-text and text-to-speech models, allowing KITT to engage in natural, voice-based interactions with drivers and passengers.
• Geospatial Awareness: We’ve integrated KITT with mapping and routing services to provide location-based assistance, from finding nearby points of interest to calculating optimal routes.
• Real-Time Information: By connecting to external APIs, we’ve enabled KITT to access up-to-date information on traffic conditions, weather forecasts, and more.
• Customizable Voice: We’ve incorporated XTTS-v2 model, allowing users to customize KITT’s voice and even emulate different emotions.
• Extensible Architecture: We’ve built KITT on a flexible framework that allows for easy addition of new skills and capabilities.

What sets our work apart is KITT’s ability to understand the context of a moving vehicle. It’s not just answering questions; it’s providing relevant, timely information based on your location, destination, and current driving conditions. Imagine asking your car, “Is there a good pizza place on our route?” and getting a response that takes into account your current location, traffic conditions, and even user ratings of nearby restaurants. In developing KITT, we’ve addressed some key challenges in bringing LLM technology to the automotive world. We’ve developed methods to integrate real-time data sources, crucial for providing accurate, up-to-date information. We’ve also focused on reducing latency, ensuring that interactions with KITT feel natural and responsive. While KITT is currently a research project, we believe it points to an exciting future for in-car AI. As automakers race to integrate more advanced AI capabilities into their vehicles, our work with KITT is paving the way for truly intelligent, context-aware automotive assistants. We see vast potential applications for this technology, from enhancing driver safety by providing hands-free information access, to improving the overall driving experience with personalized recommendations and assistance. As we move towards more connected and autonomous vehicles, we believe systems like KITT will play a crucial role in how we interact with our cars and the world around us while on the road. We’re thrilled to be a part of this exciting field of automotive AI, and we look forward to sharing more developments as our research progresses!

Please checkout the relevant publication and project pages.