Visualizing Spatial Mapping of a Real Space in your Holographic App using HoloLens Emulator

Here is the 8th post in the series of Holographic App Development Using Microsoft HoloLens.  In this post we are going to learn about Spatial Mapping and how to render Spatial Mapping Mesh of a real space in your holographic app . Spatial Mapping is one of the most important aspects of the HoloLens and Holographic app development. The Spatial mapping is to know where it is in the real space, how is the real space surface, where are different objects  and how you can make holograms interact with the real space. Spatial mapping provides a detailed representation of real-world surfaces in the environment around HoloLens.  This allows developers to mix holograms into the world around you.

Getting Started

Holographic App Development Using Microsoft HoloLens.

The Spatial Mapping information can be accessible by Unity3D API’s or using Universal Windows Holographic API’s.  Following are the main purposes of using Spatial Mapping in our app.

  • Navigation
  • Occlusion
  • Physics
  • Placement
  • Visualization
Visualization & Placement - Using Spatial Mapping
Visualization & Placement – Using Spatial Mapping

Refer to the following video to know more details of Spatial mapping

Following are the two primary object types used for spatial mapping for the real space

  • Spatial Surface Observer
  • Spatial Surface

SurfaceObserver  in the application defined region of space that we need spatial mapping data for.  Depends on the regions and volume the spatial apping will be provided with information set of Spatial surface.

Read more about Spatial Mapping here :

https://developer.microsoft.com/en-us/windows/holographic/spatial_mapping

Rendering the Spatial Mesh

Consider your Unity Solution has HoloLens Unity Toolkit Included and have some object placed in your Unity Solution.

Adding the Spatial Mapping Prefab

Add the Spatial Mapping Prefab (Located in HoloToolkit –> SpatialMapping –>Prefabs) into the root of your Unity Hierarchy.

Adding the Spatial Mapping Prefab
Adding the Spatial Mapping Prefab

The SpatialMapping prefabs has three different scripts associated it.

SpatialMapping with associated scripts
SpatialMapping with associated scripts

SpatialMappingManager class allows app to use a SurfaceObserver  (defined region of space that we need spatial mapping data for) and The SpatialMappingObserver class represent the SurfaceObserver into an easy to use object for the application to get the mapping coordinates to use with holographic apps.

Tip :

You can use the different set of 2D texture to represent different set of empty objects or background elements from the Inspector tool.

image

Quick Unity 3D Tips

Unity 3D – How to rotate an object around and also along with a specific point ?

Enabling the Capabilities

From the Build dialog window, navigate to Player Settings. Under Publishing Setting options, navigate to Capabilities, and then enable the SpatialPerception by selecting the checkbox.

Enable the SpatialPerception
Enable the SpatialPerception

Build and Run

Once done, build and run the application on HoloLens emulator.  You should be able to see the rendering spatial mesh for the selected room.

Spatial Mesh
Spatial Mesh

Side by side you can open the device portal to see the projection in real space along with the spatial mesh inside HoloLens Emulator.

3D Projection of Apps in Room Space
3D Projection of Apps in Room Space

 

Explore More with Spatial Mesh along with Room View 

Projecting your Holographic apps in a Real Space Surroundings using HoloLens Emulator

This scripts which we have used here, are part of HoloLensToolkit-Unity. Incase you want to explore them by your own and want to use the Unity Specific API, you need to include following two namespaces in your script,

using UnityEngine;
using UnityEngine.VR.WSA;

Refer this article for more details API implementations

https://developer.microsoft.com/en-us/windows/holographic/spatial_mapping_in_unity

image

Hope this helps ! Feel free to share your feedback and don’t hesitate to reach out incase any questions.

 

Thanks

Abhijit

 

Advertisements

Written by Abhijit Jana

I am a Modern and Cloud App Development Consultant working at Microsoft. Presently I am focusing on adopting and delivering world class solution on N+1 Technologies, where I mainly drive engagements on Universal Windows Platforms, Microsoft Azure, IoT and Cross Platform and Device related application Development. I am a former Microsoft MVP, Author of Kinect for Windows SDK and HoloLens BluePrints book and Run Daily .NET Tips . To know more about me click here.

One comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s