Digital & ICT

Immersive lighting for an authentic mixed reality experience

MR360 Project 1

The lighting and shadows created by the MR360 toolkit contribute to the realism of this mixed reality scene.

Lighting and shadows are what sell realism and believability in the world of Mixed Reality (MR).

MR experiences create an environment that allows both physical and virtual objects to exist and interact in real-time.

For the artists and content creators of MR applications—who must successfully blend their digital creations in with physical surroundings—lighting is paramount. The more accurately they can replicate real-world lighting conditions in a 3D digital space, the more realistic and immersive the experience becomes for the end users of their products.

Content creators can spend hours adjusting the lights and reflection maps in their scenes to achieve the best realism but, until now, have not been able to achieve realistic lighting in real-time.

Victoria University of Wellington researchers have developed the MR360 toolkit to give content creators the power to build perfectly lit—and therefore incredibly realistic—mixed reality experiences using real-time360° video footage.

The tool automatically analyses live streamed 360° video to detect lighting conditions, allowing content developers to apply accurate, ambient and directional lights, reflections and shadows—and seamlessly blend the synthetic and real worlds together for a truly immersive, interactive experience. 

Next steps

Wellington UniVentures is currently working with the project team to explore opportunities with mixed reality hardware and development tool businesses who may benefit from incorporating the technology into their software development kits.

The team is also seeking companies to test their light estimation method and discuss other MR innovations such as materials estimation.

 

For more information, please contact the Commercialisation Manager below.

Liam Sutton
Liam Sutton

Commercialisation Manager

View Profile

Features and benefits

Real-time light detection and estimation

The MR360 toolkit automatically analyses 360° video and detects lighting conditions to apply accurate ambient and directional lights, reflections and shadows in real time.

Adaptable and automatic

Because the tool runs in real time, it adapts to the lighting conditions as the live stream video changes—no manual setup or precomputation required.

Cross-platform software

The MR360 toolkit has been developed to work with two of the most popular content creation engines on the market today—Unreal Engine 4 (UE4) and Unity.

Next steps

The ability to interact with both physical and virtual objects gives mixed reality technologies a huge number of potential applications—from film production, architectural design and broadcasting, right through to education, engineering, entertainment, healthcare and e-commerce.

The advent of 5G looks set to bring even more applications for the technology, which Wellington UniVentures expects to stimulate a corresponding demand for high-quality light estimation tools such as the MR360 toolkit.

Targeted at mixed reality content creation studios, the MR360 toolkit is being developed for two of the most popular content creation engines for MR developers—Unreal Engine 4 (UE4) and Unity—both of which are currently in the first phase of software testing.

Wellington UniVentures has been talking with a global leader in the immersive computing industry on behalf of the researchers, to ensure the product is developed to meet industry needs. And while a software development company in the United States is currently exploring the tool’s potential, the research team is keen to work with other creative studios to help them implement their virtual or mixed reality projects.

 

For more information, please contact the Commercialisation Manager below.

Liam Sutton
Liam Sutton

Commercialisation Manager

View Profile