Authors: Zirui Yu, Yanze Gao, Hongyu Ji, Cancan Jin, Yiping Zhang, Ningning Xu*

Conference: CHI EA '26: Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

Abstract: Selecting perfume is often a challenging task that relies on fleeting impressions, making it difficult for customers to compare scents or articulate their preferences. In this paper, we present a data-driven co-creation system that integrates virtual reality (VR), scent-delivery installations, electroencephalography (EEG) data, and artificial intelligence (AI) generated cards with augmented reality (AR) support to transform the concrete experience into a quantified assessment for perfume-making. In this Fragrance Journey, users explore the scenario-based VR environment to sample scents, providing explicit ratings while the system simultaneously captures EEG-derived features. These physiological indicators are converted into weighted metrics, which an algorithm uses to synthesize high-scoring components across fragrance notes into a personalized formula, and then visualized as a unique AI-generated card to get perfume profiles for follow-up customization. The proposed design integrates digital contexts with physical olfactory stimuli, offering a novel paradigm that would contribute to the customization of scent products.

Authors: Yanze Gao, Zirui Yu, Cancan Jin, Hongyu Ji, Guanyu Chen, Kaiwen Li, Ningning Xu*, Yiping Zhang*

Conference:CHI EA '26: Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems

Abstract:University freshmen often face changes in their studies, life, and social adaptation during the transition from high school to university. It is crucial to facilitate familiarity with campus regulations and ensure easy access to relevant information. This study was conducted at a Chinese university to design a campus information service application with augmented reality (AR) technology. Based on preliminary investigation from user interviews and online surveys, we identify the user requirements, which subsequently guide the prototype design. We present an overview of the proposed system, CIService AR, and evaluate its usability and effectiveness through questionnaires and in-depth interviews. Our findings suggest that the prototype has great potential to improve the university experience and the effectiveness of information acquisition and transmission.

Authors: Kai Yao, Cancan Jin, Haoran Nie, Shuyan Chen, Kaiwen Li, Ningning Xu*, Yiping Zhang*

Conference: 2026 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)

Abstract: This paper introduces an innovative technical form to enhance visiting experiences that respect the user perspectives about cultural heritage and the integrity of cultural contents. We curate an interactive art exhibition, CanalWalk, using an autobiographical design approach to reconstruct cultural heritage through co-creation. Our design integrates generative AI and VR sketches to reinterpret cultural landmarks along the Grand Canal. CanalWalk combines a series of interactive mechanisms to foster cultural exploration, self-expression and social communication in visits. This research demonstrates the potential of combining generative design, spatial storytelling, and social VR to represent cultural heritage.

Authors: Jiawen Li, Zheyuan Jiang, Yifei Chen, Yuwei Ren, Jingji Li, Kai Yao, Xu Sun, Bin Jia*, Ningning Xu*

Conference: IEEE International Conference on Educational and Information Technology (ICEIT 2026)

Abstract: Virtual Reality (VR) is an emerging technique that alters the way users interact with music content. In this paper, we introduce VRhythmSpace, a VR musical game prototype that enables users to engage, learn, and play with music. We first conducted a preliminary investigation with seven volunteers in three different VR musical games to examine how users engage in these games and identify key differences in game features and preferable in specific contexts. Drawing from our findings, we formulate five user requirements, followed by design and user test with eight participants. The results show that users have high expectations for rich, diverse and immersive game experience with different motivations and purposes. This work provides valuable insights for researchers and practitioners, laying the foundation for more engaging and creative music experiences.

Authors: Ningning Xu, Yifei Chen, Teng Ma, Qinling Mao, Xinlie Huang, Haibo Zhou, Xu Sun*

Journal: ACM Journal on Computing and Cultural Heritage (JOCCH)

Abstract: Yue Opera is a traditional Chinese through-sung storytelling style that conveys Chinese cultural ideologies and historical depth. In this context, Augmented Reality (AR) has been adopted in cultural venues to break the passive watching experience, turning the audience into active participants. In this article, we present an exploratory case study of LanternOperAR, a hybrid cultural gift to promote Yue Opera with the Chinese philosophy of Yangmingism. We incorporate the tangible interface and virtual content to create an interactive cultural experience. Specifically, we conducted a fieldwork study to investigate the promises and challenges of promoting Yue Opera. Based on expert interviews and an online survey with the public, we derived a set of design goals to guide the design and development process. Our approach incorporates the application of AI image processing, generative AI, information visualization, and gamification with AR technology to generate and visualize cultural knowledge in AR-based narratives and interactions. We evaluated LanternOperAR with 20 participants and acquired some positive feedback on system usability and overall experience, as well as some suggestions for further improvements. Our findings provide insights into the design of cultural products and cultural applications as guidance for cultural researchers and practitioners.

Authors: Hongyu Ji, Cancan Jin, Ningning Xu*, Yiping Zhang*

Conference: International Symposium on Visual Information Communication and Interaction (VINCI 2025)

Abstract: As a hot topic in the 21 century, cultural digitization is an important research spot in both the cultural industry and educational field, providing new opportunities to engage in cultural heritage. A successful cultural visit must allow visitors to make rich interpretations of the underlying cultural information while also paying appropriate attention to the surrounding visitors. In this context, the hybrid cultural gift, with the integration of tangible interfaces and digital technologies, is a promising step toward a more engaging, personalized, and socialized experience for cultural inheritance. In this paper, we investigate the continuous intention to use hybrid cultural gift in collaborative learning considered about emotional design and technology acceptance. We introduce our preliminary research plan, outlining the proposed research model and hypotheses, as well as the hybrid gift design and planned user study. The research will provide insight on hybrid gifts and collaborative learning for cultural researchers and practitioners.

Authors: Ningning Xu, Lingyun Yu*, Qinling Mao, Kaiwen Li, Yifei Chen, Haibo Zhou, Xu Sun*

Conference: ICMI Companion '25: Companion Proceedings of the 27th International Conference on Multimodal Interaction

Abstract: Metaverse has emerged as a prominent research direction, to realize human interactions unfold in virtual environment, considering both intent and the underlying physical mechanisms. However, user actions in the metaverse are driven by user mental processes with indirect mapping between user behavior and interaction, which differs from those in the real world. In this paper, we aim to emphasize the importance of interactive intention, bringing a sense of purpose in metaverse. Through an analysis of existing research, we present a conceptual framework that connects the metaverse with perspectives from human-computer interaction (HCI). This work provides actionable insights for advancing environment construction, interactive intention, and user factors in metaverse, bridging HCI principles with emerging paradigms.

Authors: Cancan Jin, Yanze Gao, Zirui Yu, Ningning Xu*

Conference: ICMI Companion '25: Companion Proceedings of the 27th International Conference on Multimodal Interaction

Abstract: Memory palace, also known as the method of loci, has been widely acknowledged as a strategy for memory learning. However, constructing memory palaces is complex and time-consuming, which requires a range of cognitive abilities. In this paper, we present LociVR, a virtual reality (VR) prototype for memory training. Drawing on expert interviews and online surveys, we identify key challenges and user needs in VR memory palace construction and practice. We formulate the design concepts through a creative workshop, and present the overview of LociVR. In this work, we invite twelve volunteers to participate in VR memory training. Our findings show the promising potential of LociVR to enhance the learning experience and memory effectiveness.

Authors: Zirui Yu, Yanze Gao, Hongyu Ji, Kewei Xu, Ningning Xu*

Conference: 2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

Abstract: The Grand Canal has played an important role in Chinese history, facilitating trade and cultural exchange. Cultural institutions have endeavored to enhance public awareness of the cultural protection of the Grand Canal through digital means, but there are no effective means to stimulate public interest and communicate cultural knowledge yet. In this paper, we introduce Trajectory of Relics, a virtual reality (VR) exploration game for cultural learning and visits with the natural and cultural heritage, "Grand Canal", through interactive narratives in time travel. User test result demonstrates that our design has significantly enhanced the cultural engagement, experience and dissemination through interactions and narratives, shifting into a new paradigm for cultural inheritance.

Authors: Cancan Jin, Yanze Gao, Zirui Yu, Ningning Xu*

Conference: 2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

Abstract: Memory is an essential part of life. A comprehensive grasp of mnemonic techniques enables the application of effective memory strategies and will enhance educational and professional outcomes. The Method of Loci is widely recognized as an effective scientific mnemonic technique, though it has certain limitations due to its high demand for spatial imagination. In this paper, we conducted an exploratory comparative study (N = 36) to investigate the relationship between three mnemonic methods (i.e., Loci Booklet, Personal Adapted Loci, LociVR) and memory performance and experience. We recorded subjective ratings of user experience, measured times, test scores, and memory retention in this study. The quantitative and qualitative analyzes provide design directions for memory training based on mnemonic methods. We further identified the design goals from the preliminary investigation, and proposed key design features, as well as interactive mechanisms, through a participatory workshop (N = 28). Based on these findings, we derived a set of design implications. In future, the design of LociVR would be optimized and contribute to a more personalized and interactive experience for memory training.

Authors: Ningning Xu, Yu Liu, Yifei Chen, Zheyuan Jiang, Yuwei Ren, Zhichao Zhang, Bin Jia, Linyun Yu*

Conference: CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Abstract: We propose ArtifactShow, an interactive system that engages users in cultural experiences through a series of digital shows powered by generative AI. Current cultural digitization processes rely heavily on labor-intensive efforts to create complex multimedia scenes from vast amounts of cultural data. To address this challenge, we integrate generative AI into narrative visualization, enhancing public understanding of historical and cultural narratives through navigation, exploration, and visual analysis. To evaluate the effectiveness of our system, we conducted user studies with three domain experts and seven volunteers. Our findings suggest that ArtifactShow has significant potential to enhance public knowledge and interest in cultural heritage. Through our exploration, we developed a workflow and design guidelines for using generative AI to construct narrative visualizations for cultural heritage.

Authors: Kaiwen Li, Shuyan Chen, Yifei Cai, Yifei Chen, Ningning Xu*

Conference: CHI EA '25: Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Abstract: Gender equality is a fundamental human right, essential for building a peaceful, prosperous, and sustainable community. Despite the progress made over the last decades, old-fashioned sexist attitudes are still common in many professional fields, including DJing. Virtual Reality offers a potential solution for reducing bias by hiding the biological gender identity, thereby shifting the public focus from human features and stereotypes to DJ’s quality. In this paper, we design and curate an interactive live house event in VRChat to explore the role of avatar gender in shaping social interactions and performer identity in DJ performances. Additionally, we integrate a rich array of features to support seamless interaction between the real and virtual worlds in live house events. Preliminary findings from user feedback indicated that our design solution has promising potential to promote gender equality and inclusive interaction. This contributes to a more equitable DJing community aligned with sustainable development goals.

Authors: Ningning Xu, Yue Li, Jiachen Liang, Kexiang Shuai, Yuwen Li, Jiaqi Yan, Cheng Zhang, Yiping Dong

Journal: ACM Journal on Computing and Cultural Heritage (JOCCH)

Abstract: This article explores the use of a mobile Augmented Reality (AR) exploration game to enhance immersive storytelling and enrich cultural experiences. Specifically, we present the prototype design and evaluation of HeritageSite AR, an AR exploration game for a Chinese heritage site known as the Relics of Arhat Monastery and Twin Pagoda, or Shuangta. To develop the AR game for use in heritage sites, we employed a holistic approach, beginning with a review of technical means for cultural application development. We then conducted semi-structured interviews with domain experts and administered an online survey to identify user requirements and design goals, which informed our prototype design. An evaluation study showed positive feedback regarding the impact of game design on meaningful and playful cultural heritage site experience and identified areas for potential refinements in future iterations. We discuss the implications and lessons learned for our future work, which may also interest researchers and practitioners exploring the use of AR technologies and game design in heritage site contexts.

Authors: Yifei Chen, Chenming Ye, Zheyuan Jiang, Jingji Li, Yunqi Wang, Teng Ma, Ningning Xu*

Conference: 2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

Abstract: Mental health is an increasing concern for both individuals and society. In recent years, the prevalence of mental health disorders such as anxiety, depression, and stress has risen significantly, impacting personal well-being and productivity. In this context, Virtual Reality (VR) is an innovative approach to promoting mental health. Thus, we proposed EmoVR, an emotional VR therapy application. Based on preliminary findings derived from the literature and expert interviews, we conducted an ideation workshop to form the design requirements and show the conceptual design using personas and scenarios. Specifically, users can narrate their problematic situations to an AI-driven therapist and be enlightened, via individualized VR psychological treatments, to recognize and correctly face their emotions in a visualized healing space. The prototype is designed with regard to five common factors (i.e., Client Characteristics, therapist qualities, therapeutic relationship, change processes, treatment structures) and in three stages (i.e., consultation, treatment process, consolidation). We conclude with its potential use and our future work.

Authors: Ningning Xu, Yifei Chen, Teng Ma, Qinling Mao, Xinlie Huang, Haibo Zhou, Xu Sun

Conference: EUROGRAPHICS Workshop on Graphics and Cultural Heritage 2024

Abstract: LanternOperAR is a hybrid gift designed to engage with Yue Opera in a handheld experience. With a physical installation and a digital application, users can interact with cultural content through meaningful visualizations and gamified interaction. In this paper, we detail the design process, including the methodologies for processing, analyzing, and presenting cultural data, as well as the presenting the physical prototype. Our evaluation supplies empirical evidence and practical guidelines on designing hybrid gifts that transcend traditional cultural products to promote cultural heritage in interactive experiences. It provides valuable insights to researchers and practitioners in the cultural application area.

Authors: Yifei Chen, Qinling Mao, Xinlie Huang, Ningning Xu*

Conference: CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Abstract: Quality education and well-being have always been the focus of sustainable development as social concerns, with an increasing demand for enriched cultural activities. Shared experiences, especially in family setting, bring about a better understanding of culture and foster public interest. In this context, our research endeavors to promote the philosophies and historical insights of Yangmingism while concurrently seeking to uplift family well-being through the immersive cultural exposure offered by Yue Opera, a national opera in China. To intertwine the cultural content for a consistent and continued experience, we design LanternOperAR, a hybrid gift for cultural appreciation, learning and recreation. We fully consider and respect user interests and requirements among different target groups to provide playful interaction in cultural activity, thereby promoting cultural inheritance and strengthening family ties.

Authors: Ningning Xu, Yu Liu, Zhichao Zhang, Lingyun Yu*

Conference: 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

Abstract: Digitization is becoming a significant trend in cultural learning. In this context, we present TimeQuestAR, an interactive board game to foster public interest and understanding of culture within the ancient capital of China, Jinling. The design grounded in the principles of situated visualization, offers users immersive, contextually rich experiences that deepen their understanding of historical and cultural narratives. We provide an overview of TimeQuestAR and conduct a user evaluation through in-depth interviews. Our findings indicate that the prototype shows promising potential to promote cultural learning through playful experiences. Drawing from these findings, we propose design suggestions and delve into the prospective value of the design in education, business, and culture.

Authors: Ningning Xu, Jiachen Liang, Kexiang Shuai, Yuwen Li, Jiaqi Yan

Conference: CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Abstract: Cultural heritage (CH) plays an important role in realizing the Sustainable Development Goals (SDGs). In this paper, we focus on emerging technologies such as Augmented Reality (AR) and gamified learning to foster public understanding of cultural values in historical contexts. We design HeritageSite AR, an exploration game for onsite CH learning and visits with publics in Relics of Arhat Monastery and Twin Pagoda (also known as Shuangta). Based on research investigation of technical means, expert semi-structured interviews and online survey, we distill and incorporate four design goals using user journey map. The implemented game design is evaluated with respect to three design components (i.e., reality, meaning, play) and four stages (i.e., trigger, engage, consolidate, relate) in CH visits. We conclude our work with a discussion of contributions to SDGs.

Authors: Ningning Xu, Yue Li*, Xingbo Wei, Letian Xie, Linyung Yu, Hai-Ning Liang

Journal: International Journal of Human–Computer Interaction (IJHCI)

Abstract: Museum artifacts are the main way for visitors to experience and learn about cultural heritage. Augmented reality (AR) allows for high interactivity and is increasingly applied in museums to improve tourists’ experience and learning. It also supports the extension of museum experience to outside of the physical museum space, contributing to the visiting trajectory and takeaway experience. In this paper, we present our design of two tangible AR interfaces for cultural artifacts: Postcard AR and CubeMuseum AR, followed by three user studies that evaluate and optimize the design. In Study 1, we conducted a within-subjects study (N = 24) that compares the two AR interfaces with a baseline condition (Leaflet). Our results demonstrate the positive effects of tangible AR interfaces on users’ motivation and engagement in learning cultural heritage. In Study 2, we further explored how to optimize CubeMuseum AR by adopting a user-centered design approach. Through the analysis of expert interviews (N = 7) and an online survey (N = 207), the results specify a series of requirements and design guidelines for tangible AR interfaces to be used as a learning tool and a hybrid gift. Based on the findings, the design of the CubeMuseum AR was optimized and evaluated in Study 3. A between-subjects user study was conducted (N = 32) to compare the optimized design with the initial design. The results verified the positive effects of gamified tangible AR interfaces on users’ motivation, engagement, and performance in learning cultural heritage. We present our design and evaluation results, and discuss the implications of designing tangible AR interfaces for cultural heritage learning and museum gifting.

Authors: Ningning Xu, Yue Li*, Jie Lin, Lingyun Yu, Hai-Ning Liang

Conference: 2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)

Abstract: Mobile Augmented Reality (AR) is becoming increasingly affordable and popular with the constantly improving computing power of mobile devices and the popularity of smartphones and tablets. In this paper, we present a user study that investigates user retention of mobile AR in cultural heritage learning. We developed a mobile AR application that allows users to observe 3D models of museum artifacts and learn about their culture and history. Participants achieved a knowledge retention rate of 78.21%, indicating the positive effects of mobile AR on cultural heritage learning. We performed a structural equation modeling analysis (N=50) to investigate the effects of usability, satisfaction, emotional attachment, focus of attention, and flow experience on user retention of mobile AR. The analysis results confirmed that user satisfaction and flow experience positively affect user retention. Usability and focus of attention contribute positively to user satisfaction and flow experience respectively.