Title: A new content format for immersive experiences
ArXiv ID: 1610.07577
Date: 2016-10-25
Authors: Joan Llobera
📝 Abstract
The arrival of head-mounted displays (HMDs) to the consumer market requires a novel content format that is, first, adapted to the specificities of immersive displays and, second, that takes into account the current reality of multi-display media consumption. We review the requirements for such content format and report on existing initiatives, some of our own, towards implementing such content format.
💡 Deep Analysis
Deep Dive into A new content format for immersive experiences.
The arrival of head-mounted displays (HMDs) to the consumer market requires a novel content format that is, first, adapted to the specificities of immersive displays and, second, that takes into account the current reality of multi-display media consumption. We review the requirements for such content format and report on existing initiatives, some of our own, towards implementing such content format.
The arrival of head-mounted displays (HMDs) to the
consumer market requires a novel content format that is,
first, adapted to the specificities of immersive displays and,
second, that takes into account the current reality of multi-
display media consumption. We review the requirements for
such content format and report on existing initiatives, some
of our own, towards implementing such content format.
Index Terms—Virtual Reality, Content Production
INTRODUCTION
The arrival of head-mounted displays (HMDs) to the
consumer market requires a novel content format that is,
first, adapted to the specificities of immersive displays and,
second, that takes into account the current reality of multi-
display media consumption. Immersive displays impose
radically different audience requirements compared to
traditional broadcast TV and social media. For example,
cuts between shots, which constitute the very basic fabric of
traditional cinematic language, do not work well in
immersive displays. Immersive displays also need to
integrate with the consumer habits already in place within
the contemporary living room, where TV consumers often
use second screens -mostly smartphones, tablets or laptops.
This paper summarizes, first, the requirements for such
content format and, second, review current initiatives to
deliver such content.
FORMAT REQUIREMENTS
2.1. Immersive display requirements
Presence in Immersive Virtual Environments (IVEs) is
based upon two factors: place illusion and plausibility [1].
Place illusion is realized through the affordance of
sensorimotor correlations. Research in this area has also
shown the extreme importance of having a virtual body to
feel place illusion.
Plausibility is a less understood cognitive factor that
requires implementing interaction mechanisms similar to
our everyday social conventions. Creating compelling
immersive
content
requires
addressing
both
these
requirements.
This work has been funded by European Union’s Horizon
2020 program under grant agreement nº688619.
2.2. Audio-visual conventions
One reason cuts and camera movements do not work well in
IVEs is because they override sensorimotor affordances.
One option is to avoid using these by creating content closer
to the conventions of theatre. Otherwise, a suitable
alternative to cuts and camera movements, preserving place
illusion, needs to be introduced.
2.3. Multi-platform delivery
Immersive content needs to shareable online through social
media. It also needs to be possible to visualize and edit it
with common media consumption devices (TV, tablets and
smartphones).
2.4 Interaction paradigms
Interaction with the overall IVEs and particularly with
virtual characters needs to be intuitive. In addition, the
implementation of interactive content needs to be done cost-
effectively, close to standards in audiovisual and videogame
industry.
EMERGING PRODUCTION TOOLS
3.1. Delivering place illusion
Traditionally, IVEs were created using three-dimensional
(3D) Computer Graphics Imagery (CGI). 3D Modeling and
character animation were combined with real time sensing
data to allow sensorimotor affordances (turn your head, look
under the table…). This remains a viable option to produce
content. More recently, omnidirectional video production tools
(for example, www.video-stitch.com/
) have delivered
simpler methods to deliver photo-realistic content allowing
for head rotations, shortcutting some of the technical burden
associated with traditional CGI content production.
Emerging commercial solutions, such as 8i (www.8i.com
)
and Presenz (www.nozon.com/)
, which build upon research
in free viewpoint video and point cloud rendering, also
allow for head displacements. Despite these solutions only allows for head movements,
simple methods and the peripherals of a new generation of
HMDs (HTC vive, Oculus Touch, Microsoft Kinect…)
allow to integrate easily animated virtual bodies linked to
the movements of users immersed in such formats. Place
illusion is therefore achievable in current consumer HMDs.
3.2 Replacing cuts
In terms of audiovisual conventions, one option is to
introduce portals, i. e., inserts within a 3D scene that allow
displaying either directional content, either another 3D
scene, similar to how it was implemented in the classic
videogame Portal, by Valve Corporation. Such simple mechanism allows reintroducing traditional
content, with cuts and camera movements, appearing like a
floating panel in the main 3D scene. It also allows rendering
other 3D scenes to introduce flashbacks, close-ups, another
character’s perspective, etc., while preserving sensorimotor
affordances.
In other terms: the adoption of portals as a language
convention seems to allow reintroducing in I
