Collaborative Telepresence Using Mixed Mediums

Telepresence has been trending and in the eye of the tech industry for more than two decades (Fuchs, 2014); many other industries are constantly utilizing telepresence systems and are impatiently waiting for its breakthroughs. From popular apps like Skype and Hangouts to portable and autonomous robots, telepresence has taken many forms and will continue to do so in the coming years (Neustaedter, 2015).

The aim is to still allow people located in two or more different locations to connect in one space with minimal distortions (Chen, 2000). In industry, the need for these telepresence systems has increased due to the cost per commute as well as the frequency of commuting needed (Edwards, 2011). Some commutes for bigger multinational corporations are overseas with hefty amounts spent on them. Entrepreneurs with startups that are required to travel in order to do business overseas can go through the same cash flow problem with a higher risk of loss. Live one-on-one meetings are essential in business regardless of the medium they are depicted in and according to McDougall (1994), “[The company] has to be over there to find out what the customers want.” One-on-one meetings can occur daily in a business and in some industries like advertising, meetings become the forum for brainstorming and making business decisions on the go. With international agencies, the need for collaboration increases, and with costly air travel, telepresence shines as the prime solution.

Despites their use in business contexts, telepresence systems still have a number of limitations. 2D telepresence systems like Skype and Hangouts are often the most used in corporations. Their main limitation is that they restrict the user’s feelings of being present by limiting the amount of control the user has in the remote environment (Fuchs, 2014; Neustaedter, 2015). With 3D telepresence using an avatar, the user can maneuver around a virtual environment giving them more control than 2D systems. They also limit the user’s identity (Price, 2008). Both user ends are unable to see each other but rather see a fictional character which causes social disruptions during communication (Monahan, 2008). Telepresence robots on the other hand provide more control by allowing the user to navigate through a remote environment.

Their limitation however, is the user’s multitasking ability in navigating and interacting with people (Neustaedter, 2016; Kristoffersson, 2013). Most of the commercial robots do not have a collision prevention system and this pushes the user’s focus more towards the navigation, losing the ability to interact comfortably (Neustaedter, 2016; Desai, 2011; Takayama, 2011). With projection telepresence, the ability to see a remote person in a local environment is more successful, however, the ability to collaborate isn’t. Movements of the user distorts their projected image (Maimone, 2013; Fuchs, 2014). With virtual and augmented reality telepresence systems, the users can feel quite present at the space, but some argue that there is a social disruption caused by the lack in the ability to see the user’s eyes (Beck, 2013). It is claimed that eyes play a vital role in communication (Chen, 2000). By 3D scanning a user and placing them into a virtual environment, the users can interact with each other without seeing headsets on each other (Newcombe, 2011; Tong, 2012). The goal of this MRP is to create a virtual reality telepresence environment that allows users to explore and share their feedback on what they have seen as well as what they would’ve liked to see in a virtual telepresence environment.

The system works with the Unreal Engine 4.11 and uses a blueprint node based system. Building base meshes were bought and designed to look like a creative office. To provide the participant access to the internet, the chromium based web browser widget was created. To run a webpage in virtual reality, the widget was designed and then placed within a null 3D widget where it could then be placed within the environment. A text box was created on top of the webpage to allow the user to change the URL. A button on the bottom was created to close the web browser. Web browsers that were in 3D space were surrounded by volume trigger boxes to track the user’s gaze. A separate browser was created that could be operated by clicking the ‘1’ button found on the left side of the mouse. This web browser is static and pops out whenever it is called.

TV screens were created by adding a media player component and attaching a video to them (see Figure 5). A null 3D widget was created and the media player was added to it in order to allow the video to be placed anywhere within the environment. A volume trigger box was added around the media player in the widget to allow the user to control the video. Once the user approached and was inside the trigger box, the video would play. The video would also close once the user stepped out of the box. The parameters of the volume trigger box were invisible to the user. Some of the lights within the environment were also surrounded by volume trigger boxes to light an area as the user walked by.

For the active collaborative task, a building system was created to allow the user to create their own primitive buildings while they were in their virtual experience. These building tools were created by adding instances of meshes that would respawn whenever the user gazed through a trigger box. Each mesh had different restrictions and logic that would not allow the user from spawning them. The floor for instance, is the essential mesh required to be able to spawn any other mesh. Next a user had to spawn two poles before they could spawn any walls, doors, or roofs. Once there was a minimum of two floor meshes, a user could spawn a staircase. For the user interface, the mesh would turn green to signal a correct placement in the environment and would signal red if it cannot place the mesh. This system was built for multiplayer and uses a listen server so users would be able to collaborate and build at the same time.

The design of the 3D space, objects in the space, and the types of collaboration tools all went through informal design critiques. Once the system was ready, it was tested on two groups of participants. The participants recruited for this study were industry professionals who work in Horizon FCB Kuwait, an advertising agency and are between the ages of 22 to 46. Four participants were above 35 and four were below 35. These volunteers were familiar with one another and are constantly present in a creative and professional environment where they are required to collaborate on a daily basis. The first group from this participant pool were four employees from the creative department with equal amounts of both genders aged between 23 and 45. and the second were four employees from the account servicing department. The participants of this user study have never had a virtual reality experience before.  A non active participant was also recruited to participate and collaborate with users in the second scenario. The participants were asked to use online tools in the virtual space to go through two scenarios, the first was a navigation task, and the second was an active collaborative task with the non active participant. The experimenter observed the user interactions and took notes. The user interactions were also screen recorded. After the users were done with their collaborative experience, they were asked to complete a quick survey querying their experience. The time taken of each participant’s virtual experience was also measured with a time limit of 10 minutes per session. The usage differences of the telepresence system between the groups were analyzed, their feedback was documented, and the results determined. The results are documented in this final paper and an additional video was submitted with this paper showcasing highlights from the user’s screen recorded experience as well as an overview of the project.

The study included two separate tasks. Two users participated in the tasks at a time. For the user study’s effectiveness, a non active user was oriented with the system and given instructions on the process of conducting the study. The same non active user conducted all the tasks with every active participant. All the participants first went through an orientation task as none of them have had a past virtual reality experience before. This task involved a walkthrough around the environment with the non active participant in order to orient the active participants. The second scenario was an active building task where both participants were asked to create a primitive building that would take them to the roof of the  main building. This task did not relate to advertising itself and the reason behind this was to not restrict the participants and lead them to use features and determine what needs to be done to the features presented. Instead the participants were active in a building task and were asked to think of possible features they would want that would help them if they were to use such a system at work and what kind of things would they be using it for.

Each participant was guided through a designed path around the virtual area while maintaining a conversation. The environment provided all the possible user interactions through objects in the space such as sliding doors, and screens that showcase work as well as the ability for the user to connect online. The participants held a conversation about the work environment in an advertising agency and how the different areas within the virtual space caters to segments within the agency (see Figure 7). After the virtual experience was over, they reported their feedback through a survey. The purpose of this task was to identify if a user was able to navigate through the virtual environment while maintaining the ability to hold a conversation throughout half the experience as well as to orient them on using the system before an active task. Each participant had a maximum of 10 minutes to explore the environment. The participants were asked to complete a survey questioning their experience.

Participants also went through a collaborative task in the virtual space. For the purpose of the user study’s effectiveness, the participants were limited to a space where the collaborative building tools are located (see Figure 8). The participants had to create a building that would take them to the roof of the main building (see Figure 9) using the tools operated through the programmed buttons on the mouse. They were given a maximum of 15 minutes to complete the task.

The purpose of this task was to identify how participants collaborate in a virtual space, the effectiveness of virtual collaboration, and its failures. After the virtual experience was over, they reported their feedback through a survey where they were asked if collaboration would be more efficient if they were able to see a 3D interactive representation of themselves as well as others in the virtual environment.