Hi — I'm Akshay, a Human-Centered AI researcher working at the intersection of multimodal interaction, intelligent interfaces, and usability engineering. My core question is deceptively simple: when people speak to, gesture at, or glance toward an AI system, does it actually understand them? And if not — why not, and how do we measure the gap?
My research builds and evaluates AI systems that integrate speech, gesture, and visual input as first-class modalities — not as afterthoughts bolted onto existing interfaces. I develop psychometric scales, run empirical studies in both controlled and real-world settings, and model the cognitive costs that accumulate when people must switch between interaction modes. This work has produced two validated evaluation instruments: a usability scale purpose-built for voice user interfaces, and a context-aware scale for multimodal automotive interfaces.
Over the past decade I have moved fluidly between academia and industry — stress-testing large language models and foundation models at Fraunhofer IAIS, building a standards-aligned voice evaluation lab at Deutsche Telekom, and conducting early HCI research at Fraunhofer IGD. That back-and-forth between rigorous evaluation and real deployment is the lens through which I approach every research question.
"In the world of voice interfaces, the best designs are those that don't just respond, but truly listen to the human heart."
— Akshay Deshmukh, Ph.D. Dissertation (2025)
A bit more about me
I was born and raised in the city of Bengaluru, India — the Garden City, where the air smells of jasmine and filter coffee, and where I spent my early years equally fascinated by how things worked and why people behaved the way they did around them. That curiosity never left.
Since then I have been, professionally speaking, something of a nomad — moving from India to Germany and staying put just long enough in each city to develop opinions about its public transport. Fun fact: if you plot the coordinates of every city I have lived in for an extended period on a globe, they trace a surprisingly coherent arc from the Indian subcontinent to central Europe.
Outside of research, I trek whenever mountains are within reasonable striking distance, grow cherry tomatoes on my terrace in Darmstadt (mangoes back home in Bengaluru, naturally), and have been known to represent my home state in table tennis — though I now primarily use that skill to win arguments about reaction time in HCI experiments.
I also practice yoga and meditation, which I find essential for anyone who spends their days thinking about cognitive load.
My career trajectory follows a Fitts' Law curve — the further the target (tenure), the longer the movement time, but the larger the target gets as you approach it.
I have collected more usability questionnaire responses than I have eaten dosas. Both numbers are in the thousands.
Scientific Employee / Assistant Professor at TU Bergakademie Freiberg, Germany (Nov 2024 – Present). I teach Human-Centered AI, Intelligent User Interfaces, and Multimodal Systems across B.Sc. and M.Sc. programmes, supervise theses on topics ranging from driver emotion detection to Human-in-the-Loop AI design, and pursue active research in ubiquitous computing and XR-based interaction.
As Senior HMI-UX Researcher at Fraunhofer IAIS (2022–2024), I led the human-centered evaluation of OpenGPT-X and the Foundation Model Playground — defining what "usable" means for large-scale AI systems when the users aren't engineers. Before that, as UX Research Engineer at Deutsche Telekom (2019–2022), I established a Voice Lab from the ground up, aligning evaluation protocols with ETSI and Alexa standards and applying mixed-method assessment to voice assistants deployed at scale.
I hold a Ph.D. in Human-Computer Interaction (2020–2025) from the Universitat Jaume I / Fraunhofer IAIS, and an M.Sc. in Distributed Software Systems from TU Darmstadt.
Multimodal AI Systems
Building and evaluating AI systems that treat speech, gesture, and visual input as equal, interchangeable modalities. A key focus is the Modality-Switch Cognitive Cost — quantifying what users actually lose when they are forced to change interaction channels mid-task.
Usability & UX Evaluation
Designing evaluation instruments that go beyond generic questionnaires. This includes the VUIQ (voice usability) and CAUS (context-aware automotive usability) — purpose-built, empirically validated scales for AI-driven interfaces where standard tools fall short.
Voice User Interfaces
Investigating how conversational AI performs across real users, real accents, and real conditions — not just clean speech in ideal environments. Research spans multilingual VUIs, speaker personalization, and deployment robustness in healthcare and smart-home contexts.
Human-AI Interaction
Designing AI systems that respond not just to what users say, but to how they feel. This includes affect-adaptive dialogue agents, Human-in-the-Loop (HITL) learning pipelines, and XR interfaces that adapt based on detected cognitive load and emotional state.
→ Looking to pursue a Master's thesis on multimodal interaction, VUI evaluation, or Human-AI collaboration? I supervise motivated students working on empirically grounded, publishable research. Reach out with a brief statement of interest.
→ Add screenshots or demo images to each card to bring these projects to life visually.
- 2025Paper Towards Context-Aware Usability Assessments in Vehicles: CAUS Scale accepted at AutoUI 2025 — DOI
- 2025Dissertation published: Measuring User Experience in Voice and Multimodal User Interfaces — Universitat Jaume I
- 2025Paper Mind the Switch: Measuring Modality-Switch Cognitive Costs (MSCC) published in MUM '25 — DOI
- 2025Paper AI-Based Framework for Usability in Digital Health accepted in Information Research
- Nov 2024Joined TU Bergakademie Freiberg as Scientific Employee / Assistant Professor
- 2024Paper User Experience and Usability of Voice User Interfaces: A Systematic Literature Review in Information — DOI
- 2024Paper Validation of System Usability Scale as a Usability Metric to Evaluate Voice User Interfaces in PeerJ Computer Science
- 2022–24Led user-centered design & AI evaluation for OpenGPT-X at Fraunhofer IAIS
- 2019–22Established Voice Lab for AI system testing at Deutsche Telekom, aligned with ETSI and Alexa standards
I teach research-driven, project-oriented courses at TU Bergakademie Freiberg, with a focus on getting students building and evaluating real systems — not just reading about them.
- Interactive Ubiquitous Systems & Intelligent User Interfaces (full course — B.Sc./M.Sc.)
- User-Centered Design (full course — B.Sc./M.Sc.)
- Data Structures in C (lectures, exercises & practicals)
Thesis Topics Supervised
- Evaluating UX in Multimodal Voice Interfaces — studying how combining voice with touch reshapes the interaction experience (M.Sc.)
- Personalization in Voice User Interfaces — measuring the real impact of tailored responses on user satisfaction and engagement (B.Sc.)
- Designing VUIs for Non-Native German Speakers — addressing usability and comprehension gaps often overlooked in mainstream VUI design (M.Sc.)
- Driver Emotion Detection via In-Cabin Sensors — building an AI system that adapts the vehicle interface in real time based on detected stress or fatigue (M.Sc., Ongoing)
- Human-in-the-Loop Learning for AI-Powered Design Systems — exploring how structured human feedback improves generative design quality and usability (M.Sc., Ongoing)
Grants & Collaborations
- DAAD-DST-PPP — International research collaboration between BITS Pilani (India) and TUBAF (Germany) on emotionally intelligent voice assistants
- ESF-Nachwuchsforschergruppen — SharedBots: designing and evaluating multi-user autonomous robotics for public and semi-public spaces