Innovators at Alcatel-Lucent’s Bell Labs have developed a new means by which people can hold video meetings and share documents over PCs, tablets and smartphones. Thomas Kallstenius reports on this pioneering work. Co-sponsored feature: Alcatel-Lucent 

                            

                             
Thomas Kallstenius: With Bell Labs’ immersive
communications, people look on the screen as if they are
actually in the same space 
                                        
After decades of a promise unfulfilled, video communication has finally reached the point of widespread usability.
Various solutions have entered the marketplace with varying degrees of success and adoption — but have been limited by issues of broad availability, or expense, or not being able to simulate a group of people in the same space in a way that is easy and intuitive to follow.
Thanks to pioneering work by Bell Labs, new technologies are emerging which allow people to use any video enabled device to enter a shared virtual space, and discuss and share information in a way that is almost like being together. That is the foundation of immersive communications.
                                        
                                        
A new conversation experience
                                           
Immersive communications is about stimulating the human senses to such an extent that a perception of “actually being together” is created. To achieve this, certain psychological barriers — which normally hamper remote conversation and collaboration — should be unlocked. For this, new immersive technologies are needed that:
• properly convey feelings and emotions;
• leverage both verbal and nonverbal communication — such as gestures, body language or posture, facial expression and eye contact;
• hide underlying technical complexity from users;
• include a contagious user interface; ease of use is not enough, joy of use must be the aim.
The foundation for immersive communications (see tinyurl.com/GTB-AL1) is mixed reality — that is, inserting real people into a virtual environment that can be customized to accommodate particular needs and requirements. By enabling this new degree of proximity, immersive communications makes human interaction much more effective, productive and enjoyable — so people can focus more on the conversation experience and less on the supporting software and hardware.
The business case for immersive communications is initially best understood within the context of the enterprise need for cost-effective video communications to reduce travel. Additionally, video collaboration needs to be taken to the next level for a number of other — equally important — reasons: improved group collaboration, engaging an ever more nomadic workforce, and addressing meeting types which are not typically done through video today, such as trust-building kick-offs and creative brainstorms.
In this context, immersive communications:
• enables people to intuitively come together in one virtual environment to attend scheduled video conferences from anywhere — and from any device;
• accommodates instantaneous, ad-hoc video conferencing and — to address a growing market need;
• facilitates the set-up of multi-party, interactive presentations.
                                 
                       
Way beyond today’s video conferencing
                
Today’s video communication experience suffers from some major limitations. The experience isn’t as rich or engaging as it is in related media, such as TV, movies and video games; video communication applications lack the human touch, not taking into account the human factor and emotional values.
Devices dominate and hamper the experience; today, only very high-end systems — based on dedicated telepresence rooms — give the end-user somewhat of an immersive experience.
Users complain about complex user interfaces and feature overkill; too many features inevitably confuse the user and create redundancy and overlap with other applications.
Access barriers often prevent users from easily connecting to the service. Finally, video communication feels intrusive, not addressing the need for end-user privacy.
Innovations such as IPTV, 3D movies and interactive gaming have brought people engaging and interactive visual media experiences. In comparison, video communications seems flat.
All video communications systems have the same basic limitation: they simply render the video stream on the screen as it is sent by the other camera. If there are multiple participants, the video is divided into squares — a tiled window approach. Each square, or window, shows a participant or document. It’s just a matter of how many windows there are and how they are tiled across the screen.
Moreover, from an experience point of view, each window is separate. The systems don’t understand that there’s a person in one window and a document in another so they can’t put them together. This makes the experience less meaningful for participants.
Device-centric video communications limit potential for a number of reasons as well:
• Different levels of processing power, battery life and bandwidth capacity make “any device to any device” video communications inconsistent and difficult.
• Different device types and software clients are often incompatible and can’t communicate with one another using video.
• While sophisticated video communications systems such as room-to-room telepresence solutions offer an immersive experience, they require very expensive endpoint devices that most people don’t have access to. And when they are accessible, these systems typically require IT assistance to operate.
• When more accessible devices, such as computers and smartphones, are used for video communications, the experience is often far from stellar.
The network is key to removing these limitations and delivering the best of both worlds — an experience comparable to that of a telepresence system but accessible from anywhere, on any video-capable device. Welcome to the world of immersive communications!
How does it work? Three innovative concepts are at the basis of any successful immersive communications experience.
                             
Background extraction and mixed reality: participants to an immersive communications session are visually removed from their live environment, and staged in a virtual setting.
Background extraction is similar to the concept of green screening in movie making. It cuts out each participant and object in a video call from their background and inserts them into a common virtual space.
Each participant enjoys their own personalized view of the virtual meeting. And all participants can communicate and collaborate in the natural and unstructured way they would if they were in the same room. Also, participants are ensured the privacy of not showing their actual physical environment.
Mixed reality is the technology that lets people blend the human environment with a 3D virtual environment in a realistic way by incorporating technology advances from the world of gaming. With immersive communications, on the screen people look like they are actually in the same space.
Virtual director: to allow people to extract and mix participants and objects in virtual environments, the system needs to recognize and understand participants’ movements and react accordingly. It needs to behave as a virtual director. By following head and hand movements, for example, the system can determine when to zoom in on a participant, change the viewpoint or trigger an action.
The software in the network has the intelligence to analyse both audio and visual cues and make the decisions. This requires bringing together a number of sources of real-time metadata to recognize and interpret subtle changes in facial expression, gestures, postures and gazes.
Gesture control: gestures are a major form of non-verbal communications. When collaborating, it’s often much easier and faster to point to an illustration or a sentence in a document than it is to refer to it verbally.
The gesture control capabilities in immersive communications let participants in a video meeting display and interact with objects in the virtual space. They can use common gestures and move their hands in natural ways to share images and text or call for attention.
                         
                
Underlying technologies
                       
A number of underlying technologies enable immersive communications to achieve its full potential. They include immersive technologies, multi-mode/multi-device access, and network/cloud-based operations.
The immersive communications experience is largely defined by means of innovations such as background extraction, mixed reality, virtual director and gesture control.
The immersive communications application is multi-modal, meaning that it is accessible through any device — including PC, smartphone and tablet — eliminating the need to reserve designated videoconference rooms. No matter where they are, participants can log in, set their own environment and be staged in a common virtual space.
Running immersive applications in the network eliminates processing capacity constraints on clients and provides both scalability and elasticity. Moreover, this approach makes sure that content can be delivered to any device at any time — through the use of new routing mechanisms that are embedded into core network elements. In today’s video communications systems, the network is used mainly for transporting video streams. But it can play a much greater role.
If we put more advanced video communications technology and all of the intense video processing in the network, device limitations disappear. As such, the experience becomes richer and more engaging. The device needs only a thin client and the ability to get a video stream up and down to a nearby location in the network to access a new world of video communications capabilities.
Putting the video intelligence in the network makes sense from both technology and business perspectives. Service providers’ networks are already engineered for low latency and low jitter. That means video quality is essentially built into the network.
Service providers also already understand how to efficiently manage routing of real-time, high-bandwidth traffic across the network. This expertise means service providers are the only ones that can offer enterprises service level agreements for high-quality, secure and highly available video communications services as well as further service enhancements such as call handling, presence and location.
Because immersive communications is essentially a software approach deployed on cloud hardware, service providers have an easy-to-understand and separate layer of video capabilities. Combining network capabilities with network-hosted services allows service providers to offer video communications services that work across all levels of the stack. These capabilities can also be integrated with other assets, such as customer profiles and billing systems.
Immersive communications thus becomes an extension of the things service providers already do today. And it helps them turn video consumption into video communications.
                  
                
Not just for enterprises
                   
The immersive communications concept can break out of the business-only use scenario and enter the much broader sphere of all electronic interpersonal based communication.
In combination with social media, immersive communications can generate new interactive experiences such as personal picture sharing, joint video watching, holiday planning, language learning, e-commerce, health care, entertainment, interactions, gaming, dating, etc. It will make for a completely different, non-anonymous, way of engaging in group conversations.
While enterprises are the most likely first adopters of immersive communications, everyday more people have video camera enabled devices at their fingertips. They are an eager audience that — once they have access — will readily bring the advantages of immersive communications to all aspects of their lives. GTB
Thomas Kallstenius is director of immersive communications at Alcatel-Lucent’s Bell Labs