Immersive media represents the methods for displaying and interacting with content in new and exciting ways for the consumer. This includes changing the dynamics of content with more colors, precision and dynamic range.  It also includes changing mediums for the display from flexible, transparent and large displays. Finally, it represents a new level of personalization, including first person, fully immersed by 360 degrees of content and personalized vantage points.

Samsung and Huawei showcased their flexible display phones in 2019. Beyond phones, bendable displays will enable new types of wearables and other yet to be imagined devices and experiences. Corning is working on a flexible glass product, called Willow, which is bendable, non-porous, and as thin as paper.

Ultra-high-resolution head mounted displays (HMDs) have very high frame rates to reduce the nausea experienced with earlier VR prototypes.  In addition to pixel count, Pixel Density  can be used to describe resolution of a head mounted display and has historically been the factor separating consumer grade and professional grade VR headsets. The 8K Pimax headset and  HP's high-def VR headset, Copper, are both examples of high resolution HMDs. . Finnish startup Varjo, founded by former Microsoft and Nokia Research employees, is creating a product which is committed to delivering human eye-resolution and fidelity.

The migration of mobile AR from your phone at arm’s length to “glasses based display” is often described as standalone, but this is not really the case;  the consensus is that to meet full MR requirements you will need the latency that 10G or 5G can provide as well as edge computing. The computational and requisite power requirements are driving things to edge cloud compute architectures requiring “split rendering” with only essential “lightweight” processing onboard the device and the rest distributed at the edge. Adding cameras, integrating voice assistants, moving smart watch applications into heads-up display and other experiments are giving traction to new glasses:  Everysight Raptor,   Vuzix Blade,  , as well as SNAPchat’ s 3rd generation Spectacles.  

Interacting with applications and data through vision will be a game changer. Using emerging eye-tracking technologies, visual interfaces will be designed for navigation, input, and interaction tasks, as well as, hands free environments. AR and VR headset manufacturers integrate Tobili VR biomechanics and eye tracking technologies to create vision driven user interfaces. Vision oriented interaction models enable a new class of services for the disabled.

8K displays are available from Samsung, LG, Sharp, TCL, and Toshiba. In preparation for the 2020 Olympics in Tokyo, consumers began to purchase 8K TVs. But content delivered in 8K is still sparse and developed for very specialized events. Japan’s NHK launched the first 8K TV channel in 2019.  The increased resolution will change the way we interact with TVs. Virtual windows of content from multiple sources will blend in a compelling story telling experience while maintaining the intended distribution resolution of each source. Consumers will also leverage the display resolution to combine Internet, social and broadcast content in exciting new ways.  Virtual windows will allow dynamic viewing experiences while maintaining very high resolution. As more content is developed, 8K adoption will increase.

Volumetric displays are autostereoscopic and use some physical mechanism to display points of light within a volume without the use of wearables.   Such displays often use voxels instead of pixels as reference. Volumetric displays include lenticular displays (where a coating displays stereo images), multiplanar displays (which have multiple display planes stacked up), and rotating panel displays (where a rotating panel sweeps out a volume).   These provide a  limited set of views to provide some depth perception – like the  Looking Glass display or the Red Hydrogen Phone. Although many of these use the marketing term Holographic, most are not true Light Fields that project radiance images supporting 1000’s of views for extreme realism and large fields of view. 

Every surface will become a display, including mirrors and displays in new shapes with flexibility. These mirrored displays enable customized and interactive solutions well suited to fashion and retail environments. While commercially available smart mirrors were few and far between in the past, the DIY community forged ahead to provide blueprints to close the gap.

Consumers will encounter content on see-through transparent displays, providing a unique experience in their homes and workplaces.  These displays could look like glass, shelving, doors, or windows in your home, which transform from a tinted see through glass-like finish to an immersive display delivering personalized information.

This technology allows the skin to be used as a finger input surface. When a finger taps on the skin, the impact creates acoustic signals, which can be captured by a bio-acoustic sensing device. Variations in bone density, size and the different filtering effects created by soft tissues and joints create distinct acoustic locations of signals, which are sensed, processed and classified by software. Interactive capabilities can be linked to different locations on the body.

Light Field displays are truly holographic, producing 3D aerial imagery without head tracking and/or eyewear by reproducing the light that would reflect off an object as if it were physically present in front of the viewers.  Light Fields displays are early in development and require large amounts of processing power and complex content generation to deliver these visual stunning 3D images. Early deployments target big venues and entertainment experiences focused on delivering new immersive experiences that consumers cannot get at home; just like in real life, viewers will have natural shared viewing experiences with the realism of parallax, occlusion and spectral highlights. This will be no fake Tupac’s Hologram, so be prepared to walk into the holodeck at your favorite theme park.

Free-space (contactless) haptics is a technology that enables users to receive tactile feedback without needing to wear or touch anything, enabling new levels of immersion and experiences. The technology uses acoustic waves to create the effect of touching physical objects or substances through the air. Users can ‘feel’ touchless buttons to get feedback for mid-air gestures or interact with virtual objects. Companies, like UltraHaptics are focusing on early product use cases for automotive, desktop, appliances use cases. As immersive 3D and holographic displays for VR, MR and Lightfields gain market traction, the realization of free-space haptics is critical for the market to live up to lofty expectations for immersive experiences. Light Field Lab plans to integrate volumetric haptics into their roadmap to simulate touch into their panels allowing not just for visual immersion, but tactile immersion as well.

Creating body-language datasets is an unusually difficult task since there is no pre-existing documentation like written language to start from. Developers are at ground zero, deciphering a non-verbal language that was never possible to record until Plenoptic stages allowed multi-view motion tracking: spatially recording full expressions of the body, face, and hands in all directions. This behavioral data generation is currently underway at select universities and companies like Facebook, but the work is scaling worldwide, and several Body-Language Datasets will soon become large enough to train from, and will likely be used for smart, volumetric social applications.

Biometrics will continue to evolve to include brain-computer interfaces that enable the next generation wireless control for devices, applications and services. Emotiv Systems is an example of a company already producing a headset that detects and measures electrical activity along the scalp using Electroencephalography (EEG) technology. This headset is used in human-computer research and additionally as a platform for gaming applications. 

The future of smart contact lenses will enable smart vision. Innovega  has already designed near eye micro displays and DARPA has developed a bionic contact lens to help those with macular degeneration focus better. Samsung has also filed a patent for a contact lens that acts as a remote camera and can transmit video to a nearby smartphone. The medical applications are also numerous. Contact lenses used as medical monitoring devices for diabetes have been developed by The Korea Institute of Science and Technology (KIST).

With companies like NVIDIA building GPUs that outperform Moore’s Law, Industry consortiums of technologists, and content creators like The Immersive Digital Experiences Alliance (IDEA) setting standards for interchange and distribution of immersive media, the likelihood of home Light Field Displays replacing traditional TVs sometime in the future is high. As cost structures around backend electronics, content toolchains, and network readiness for distribution are improved, Light Field Displays will become commonplace.

Traditional human-computer interfacing requires some kind of input device that modifies natural human behavior: fingers touching a screen, pushing a joystick, moving a mouse, etc. 3DoF and 6DoF headsets are the first input devices that allow for physical actions that are used in natural human behavior: looking around by moving the head (3DoF) or moving to a different position in space by physically walking or moving otherwise (6DoF). Natural-Human Interfacing will become more important as holographic media becomes mainstream: using a hand to grab a holographic object, such as a coffee cup, will be an easier and more compelling way to interact with that media. Vision interfaces and free space haptics are earlier enablers.

Content will be available for AR/VR and other immersive experiences requiring higher resolution displays in the 8K, 10K, and 16K ranges, as well as, for light field displays.

Persistent displays offer many benefits, including the ability to display high levels of image content using ultra-low power reflective displays with image storage that only consume power with changes to the image. This technology enables flexible plastic displays capable of showing color images.