Abstract:

Virtual Reality (VR) environments present significant obstacles for users due to the sheer diversity of input devices, numerous interaction modalities, and varying user interface designs, which contribute to a steep learning curve. To address these complexities, feedforward is essential as it informs the user about the anticipated result of their actions, easing the learning process through contextualized previews of required interactions. However, designing and implementing effective direct feedforward, particularly without dedicated tools, can be tedious. We introduce BeatriXR, a comprehensive, reusable, and adaptive toolkit that provides extensive support for creating all possible configurations of direct feedforward to enhance user understanding and performance in VR. BeatriXR is an integrated system combining a modular VR toolkit with intelligent support derived from Large Language Models (LLMs). It supports the creation, visualization, and customization of direct feedforward using virtual avatars and two visualizations: an in-world representation and an on-screen comparison of interaction alternatives. This toolkit is mapped onto an established feedforward design space, covering phases such as Triggering, Previewing, and Exiting. To overcome the challenge designers face in determining optimal configurations, BeatriXR integrates an LLM-based adaptive decision support layer that proposes context-sensitive configuration alternatives. This guidance can be used both at design time to help domain experts select optimal configurations, and at runtime to adapt to the user's context and environment, such as recommending a change from a default partial avatar preview to a full ghosted avatar when a trainee shows uncertainty. We conducted an exploratory review with XR domain experts who rated the UI interface the final user can use to modify the feedforward settings, and the output of four LLM models customised for use in BeatriXR that would interact with procedure creators, providing insights on how to improve the UI experience and LLM answers. The results indicated that none of the evaluated models consistently outperformed the others, suggesting that the tested LLMs can be used interchangeably. Additionally, participants' feedback provided valuable insights for improving both the user interface, generally perceived positively, and the quality of the LLM-generated responses.

Cite (BibTeX):

@article{artizzubeatrixr2026,
  author    = {Artizzu, Valentino and Luyten, Kris and Rovelo Ruiz, Gustavo and Spano, Lucio Davide},
  title     = {BeatriXR: Comprehensive and adaptive feedforward support for guidance in virtual reality},
  journal = {Proc. ACM Hum. Comput. Interact.},
  year      = {2026},
  number    = {EICS}
}