An Agent-Based Intelligent HCI Information System in Mixed Reality

An Agent-Based Intelligent HCI Information System in Mixed Reality Hamed Alqahtani
hsqahtani@kku.edu.sa VISOR, Faculty of Science and Engineering, Macquarie University, Sydney, Australia

Charles Z. Liu charles.liu@mq.edu.au VISOR, Faculty of Science and Engineering, Macquaire University, Sydney, Australia

Manolya Kavakli-Thorne manolya.kavakli@mq.edu.au VISOR, Faculty of Science and Engineering, Macquarie University, Sydney, Australia

Yuzhi Kang y.kang@psismartsys.com PSI Smartsys, Sydney, Australia

Abstract This paper presents a design of agent-based intelligent HCI (iHCI) system using collaborative information for MR to improve user experience and information security based on context-aware computing. In order to implement target awareness system, we propose the use of non-parameter stochastic adaptive learning and a kernel learning strategy for improving the adaptivity of the recognition. The proposed design involves the use of a context-aware computing strategy to recognize patterns for simulating human awareness and processing of stereo pattern analysis. It provides a flexible customization method for scene creation and manipulation. It also enables several types of awareness related to the interactive target, user-experience, system performance, confidentiality, and agent identification by applying several strategies, such as context pattern analysis, scalable learning, data- aware confidential computing.

Keywords: Agent-Based Collaborative HCI, Intelligent Human-Computer Interaction, Mixed Reality

1. Introduction Virtual Reality (VR) deals with an alternate world filled with simulated environments and objects. Computer-generated images allow the end user to interact with the virtual space. The virtual reality presents the user the experience of a mixture of reality and virtuality [6, 12, 54]. Interaction in VR can be divided into three types; namely, textual VR refers to interaction without visualization; desktop VR deals with interaction with visualization, and immersive VR handles interaction with a high level of immersion [4]. The immersive interaction has serious issues in VR [3, 47]. Augmented Reality (AR) refers to an experience in a direct or indirect view of a physical, real-world environment. AR systems are more widely used than immersive VR systems due to their low cost and easily accessible characteristics [41, 22, 17, 60]. However, a mixture of virtual and real worlds supported with equally physical and augmented elements requires immersive interaction.
   Thereafter, Mixed reality (MR) systems [39, 55, 7] were proposed to perform merging of real, with a cyber world, providing possible improvements for high-level of interaction, such as memory functioning [43], attention enhancement [5] and behaviour therapy [19]. Modern computer systems facilitated with auxiliary functionalities are capable of performing analysis on data and visualization with marked characteristics [37, 2]. However, these devices require some sensors and actuators (as in [13]) for such interaction. The interactions developed still lack the understanding of users and their intention. This paper presents the development of an intelligent HCI (iHCI) system using a piece of collaborative information for MR [32, 25] and allows the system to understand the user’s intention during the interaction. Such interaction requires the systems to be aware of the subject and context in human-computer interaction (HCI). The proposed system enables a better user experience and information security using context-aware computing [16, 10].
   Rest of the paper is organized as follows. Section 2 provides a review of state of the art in the field of MR. Section 3 describes the agent-aware computing. Section 4 describes the system and detailed design of the proposed system. Section 5 presents the implementation details of the proposed system. Finally, Section 6 concludes the paper at the end.
2. Related Work The applications of computational intelligence in virtual environments are a convergence of smart system and HCI systems. Development of intelligent HCI (iHCI) systems remained as a key to this convergence. Several designs have been proposed in the last decade. The agent-based design is one of the most commonly used designs for implementing context- aware iHCI systems. Several designs have been proposed using agents for developing intelligent system design [24, 50, 18, 35, 48]. These designs have different capabilities for intelligence, such as perception, cognition, recognition and instruction. However, there are many issues related to intelligent VR systems that include knowledge representation, autonomous agents, and agent coupling. Those intelligent interactions were based on the results of machine states influenced by

…(Full text truncated)…