Based on my experiences with spatial audio and HMD headtracking from working on Blind Archer, my goal became to explore the limits of audio-only. Specifically, to tackle an area which was previously exempt from the project since it required a new solution – User Interface (menu navigation, voice-over, meta feedback).
The creation of an immersive Sonic User Interface led me to look around for inspiration and what had already been achieved in related fields. Grounded in my research, the development of an immersive and exclusively audio-haptic feedback system in virtual reality was realized. The VR prototype takes place in a digital auditory reinforcement chamber, hence its name “DARC”. The setting was chosen because of its potential to train specific hearing skills and the vast potential for immersive variety enabled in a sci-fi context To use the whole bandwidth of human hearing, without creating an indiscernible noise, I established some design rules for players to unconsciously recognize different sounds by their sonic qualities. The following contrasting feedbacks are experienced:

• Environment (360° Simulated Real-world Ambience) | Natural, Unintrusive, Constant
• Object (Interactables, Animated Ornaments) | Spatialized, Ethereal, Realistic, Dynamic
• Meta (Score, AI Narration, Teleportation) | Non-spatialized Stereo, Synthetic, Acute
• Controller (Hand Position, Input) | Haptic Feedback + Interpolated Radar Sound

Several other interesting design decisions lead to the creation of new and unique interaction systems such as ‘dynamic orb feedback’ or ‘key lock gesture input’, which are further explained in my thesis paper. Hopefully it can be useful to you for further VR or AR development. Based on my prolific playtest results, I believe that more complex sonic user interfaces will soon be the expected norm, as interactive audio is overhauled. An audio revolution is inevitable!

Abstract:
This paper represents an assimilation of effective practices from the fields of user interface design, audio games, and technology for the visually impaired. The intention to translate traditional user interface elements for primarily sound-based applications is attempted by a combination of existing approaches with a special focus on immersion-building experiences. Following the findings is a proposal for a sonic user interface in a virtual reality context. The effectiveness of said practices is judged in an isolated audio-only prototype to ensure accuracy.
Keywords: VR audio, spatialization, UI design, audio games, visually impaired