Efficient text entries without a real keyboard remain an industry-wide challenge for unlocking XR headset productivity usage cases. Researchers codified various methods and created a comprehensive catalog of existing text input techniques to analyze their strengths and weaknesses. By making catalogs freely available, researchers hope to get a head start for others in creating new, improved techniques.
Guest article by Max Di Luka
Massimiliano di Luca heads the VR lab at the University of Birmingham, UK and is an associate professor in the school of psychology and the Faculty of Computer Science. He previously worked for Meta, where he pioneered the manual input and tactile work in VR. His recent collaboration with the industry was recently recognized by the ACM Sigchi 2025 Award for establishing basic input methods and interaction guidelines for the XR operating system and pioneering the Android XR interaction framework through an exemplary industrial assademia collaboration.
As immersive experiences become increasingly refined, the challenges of efficient text entry remain an important barrier to seamless interactions in virtual and augmented reality (VR/AR). From creating emails for virtual workspaces to logging and socializing in metaverse, the ability to efficiently type text is essential to the ease of use of all Augmented Reality (XR) applications.
To address this challenge, my team at the VR Lab at the University of Birmingham (UK) developed a comprehensive research initiative cataloguing comprehensive research initiatives through XR techniques by Copenhagen, Arizona State University, Max Planck Institute for Intelligent Systems, Northwest University, and Google. Text Trove is a structured repository of text input technology, a set of filters aimed at selecting and highlighting the breadth and disadvantages of text input methods developed for XR in both academia and industry.
These techniques are categorized using a range of 32 codes, including 13 interaction attributes such as input devices (for input), simultaneity, and haptic feedback modalities, as well as 14 performance metrics such as word per word (WPM) and total error rate (TER). Overall, the number of attribute techniques and scalability provides a comprehensive overview of the state of XR text input techniques.
https://www.youtube.com/watch?v=skcixlg3r30
Some important points can be inferred from our research. First and foremost, the performance of text input is essentially limited by the number of input elements (either a finger, a controller, or other character selector). Only multi-pinger typing can lead to performance comparable to touch typing speeds using a keyboard on a regular PC. As visualized in the plot below, additional input elements (or fingers) add a speed of about 5 wpm to the top user.
Words per minute using multiple fingers and different input devices. (Each dot represents one technique analyzed in the study).
Our research also shows that tactile feedback, presence of external surfaces, and visualization of fingertips alone are preferred methods for improving typing performance. For example, input on a surface (rather than in the air) contributes to a more comfortable and potentially efficient typing experience. The exterior also minimizes sustained muscle tone, making interaction more comfortable and reduces the onset of gorilla arm syndrome.
Finally, even more interestingly, as of today, there is no alternative that completely replaces the keyboard format, as it probably still offers the best words per minute. Perhaps it also requires a high learning curve. I believe that the main path for typing faster in VR than in PC may be considered as a need to reduce the travel distance of multi-finger keyboards via machine learning and AI. The XR requires its own “swipe typing” moment, making one-finger typing on a smartphone much more efficient.
In that respect, deep diving from XR text trobes represents an important step towards a more comprehensive understanding of text input in virtual and augmented reality. By providing a structured, searchable database, it aims to provide researchers and developers with resources alike, paving the way for more efficient and user-friendly text input solutions in an immersive future.
As explained in the paper, this work can have significant benefits to the XR community. “To support XR research and design in this field, we will make databases and related tools available on the XR Text Trove website.
Several authors on our team are mobile co-creators, as well as cataloging VR mobile technologies to provide researchers and designers with a head start on identifying and improving different methods.
