Hololens入门之空间映射

Hololens入门之空间映射

本文主要讲述使用HoloToolkit项目中提供的空间映射组件，便捷快速的开始使用空间映射特性，本文示例在 Hololens入门之凝视射线的基础上进行修改。

空间映射提供了Hololens周边真实世界物体表面的详细表示，允许开发人员创建一个具有说服力的混合现实。通过将真实世界和虚拟世界的合并，一个应用程序可以使得全息图像显得更加真实。

空间映射(Spatial mapping)有以下常见的应用场景

1、Occlusion
2、Visualization
3、Placement
4、Physics
5、Navigation

以下通过一个例子来说明空间映射的使用

1、在HoloToolkit->SpatialMapping->Prefabs 中找到并添加SpatialMapping Prefabs

在SpatialMapping中能够看到已经存在了三个脚本组件

SpatialMappingObserver.cs 主要是用于定期进行对周围环境进行扫描并更新Surface数据，具体可参考代码

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.using System.Collections.Generic;
using UnityEngine;
using UnityEngine.VR.WSA;namespace HoloToolkit.Unity
{/// <summary>/// Spatial Mapping Observer states./// </summary>public enum ObserverStates{/// <summary>/// The SurfaceObserver is currently running./// </summary>Running = 0,/// <summary>/// The SurfaceObserver is currently idle./// </summary>Stopped = 1}/// <summary>/// The SpatialMappingObserver class encapsulates the SurfaceObserver into an easy to use/// object that handles managing the observed surfaces and the rendering of surface geometry./// </summary>public class SpatialMappingObserver : SpatialMappingSource{[Tooltip("The number of triangles to calculate per cubic meter.")]public float TrianglesPerCubicMeter = 500f;[Tooltip("The extents of the observation volume.")]public Vector3 Extents = Vector3.one * 10.0f;[Tooltip("How long to wait (in sec) between Spatial Mapping updates.")]public float TimeBetweenUpdates = 3.5f;/// <summary>/// Our Surface Observer object for generating/updating Spatial Mapping data./// </summary>private SurfaceObserver observer;/// <summary>/// A dictionary of surfaces that our Surface Observer knows about./// Key: surface id/// Value: GameObject containing a Mesh, a MeshRenderer and a Material/// </summary>private Dictionary<int, GameObject> surfaces = new Dictionary<int, GameObject>();/// <summary>/// A queue of SurfaceData objects. SurfaceData objects are sent to the/// SurfaceObserver to generate meshes of the environment./// </summary>private Queue<SurfaceData> surfaceWorkQueue = new Queue<SurfaceData>();/// <summary>/// 防止不同surface的网格同时生成，置标记位，只允许一次性生成一个surface的网格。/// </summary>private bool surfaceWorkOutstanding = false;/// <summary>/// Used to track when the Observer was last updated./// </summary>private float updateTime;/// <summary>/// Indicates the current state of the Surface Observer./// </summary>public ObserverStates ObserverState { get; private set; }private void Awake(){//为需要空间映射数据的空间区域在应用中初始化一个SurfaceObserver对象。observer = new SurfaceObserver();ObserverState = ObserverStates.Stopped;}private void Start(){//通过调用SetVolumeAsSphere、SetVolumeAsAxisAlignedBox、//SetVolumeAsOrientedBox、 或 SetVolumeAsFrustum方法可以为每个SurfaceObserver对象//指定它们需要获取数据的空间范围。以后你还可以通过再次调用它们来重新设定检测的空间范围。observer.SetVolumeAsAxisAlignedBox(Vector3.zero, Extents);}/// <summary>/// Called once per frame./// </summary>private void Update(){// 只有在SurfaceObserver处于运行状态时进行处理if (ObserverState == ObserverStates.Running){if (surfaceWorkOutstanding == false && surfaceWorkQueue.Count > 0){//将SurfaceData从队列中取出SurfaceData surfaceData = surfaceWorkQueue.Dequeue();//当RequestMeshAsync调用成功，置surfaceWorkOutstanding为true，等待回调函数SurfaceObserver_OnDataReady//处理完成，再将surfaceWorkOutstanding置为falsesurfaceWorkOutstanding = observer.RequestMeshAsync(surfaceData, SurfaceObserver_OnDataReady);}//每隔一段时间进行刷新，查看是否发生变化else if (surfaceWorkOutstanding == false && (Time.time - updateTime) >= TimeBetweenUpdates){observer.Update(SurfaceObserver_OnSurfaceChanged);updateTime = Time.time;}}}/// <summary>/// Starts the Surface Observer./// </summary>public void StartObserving(){if (ObserverState != ObserverStates.Running){Debug.Log("Starting the observer.");ObserverState = ObserverStates.Running;// We want the first update immediately.updateTime = 0;}}/// <summary>/// Stops the Surface Observer./// </summary>/// <remarks>Sets the Surface Observer state to ObserverStates.Stopped.</remarks>public void StopObserving(){if (ObserverState == ObserverStates.Running){Debug.Log("Stopping the observer.");ObserverState = ObserverStates.Stopped;}}/// <summary>/// Handles the SurfaceObserver's OnDataReady event./// </summary>/// <param name="cookedData">Struct containing output data.</param>/// <param name="outputWritten">Set to true if output has been written.</param>/// <param name="elapsedCookTimeSeconds">Seconds between mesh cook request and propagation of this event.</param>private void SurfaceObserver_OnDataReady(SurfaceData cookedData, bool outputWritten, float elapsedCookTimeSeconds){GameObject surface;if (surfaces.TryGetValue(cookedData.id.handle, out surface)){// 设置材质MeshRenderer renderer = surface.GetComponent<MeshRenderer>();renderer.sharedMaterial = SpatialMappingManager.Instance.SurfaceMaterial;renderer.enabled = SpatialMappingManager.Instance.DrawVisualMeshes;if (SpatialMappingManager.Instance.CastShadows == false){renderer.shadowCastingMode = UnityEngine.Rendering.ShadowCastingMode.Off;}}surfaceWorkOutstanding = false;}/// <summary>/// Handles the SurfaceObserver's OnSurfaceChanged event./// </summary>/// <param name="id">The identifier assigned to the surface which has changed.</param>/// <param name="changeType">The type of change that occurred on the surface.</param>/// <param name="bounds">The bounds of the surface.</param>/// <param name="updateTime">The date and time at which the change occurred.</param>//处理空间表面变化private void SurfaceObserver_OnSurfaceChanged(SurfaceId id, SurfaceChange changeType, Bounds bounds, System.DateTime updateTime){if (ObserverState != ObserverStates.Running){return;}GameObject surface;//关于空间表面变化，有几个典型情形需要处理。Added状态和Updated状态可以使用相同的代//码处理，Removed状态则使用另一种代码来处理。switch (changeType){//在Added和Updated情形下，我们从字典中添加或者获取代码当前网格的对象，使用必要//的组件来创建一个SurfaceData结构体，然后调用RequestMeshDataAsync方法在场景中//使用网格数据和位置来填充对象。case SurfaceChange.Added:case SurfaceChange.Updated:if (!surfaces.TryGetValue(id.handle, out surface)){surface = AddSurfaceObject(null, string.Format("Surface-{0}", id.handle), transform);surface.AddComponent<WorldAnchor>();surfaces.Add(id.handle, surface);}//将surface添加到队列中，等待处理QueueSurfaceDataRequest(id, surface);break;//在Removed情形下，我们从字典中移除当前网格代表的对象并销毁它。case SurfaceChange.Removed:if (surfaces.TryGetValue(id.handle, out surface)){RemoveSurfaceObject(surface);surfaces.Remove(id.handle);}break;}}/// <summary>/// Calls GetMeshAsync to update the SurfaceData and re-activate the surface object when ready./// </summary>/// <param name="id">Identifier of the SurfaceData object to update.</param>/// <param name="surface">The SurfaceData object to update.</param>private void QueueSurfaceDataRequest(SurfaceId id, GameObject surface){SurfaceData surfaceData = new SurfaceData(id,//当前对象的MeshFilter组件surface.GetComponent<MeshFilter>(),//用于在空间中定位对象的空间锚surface.GetComponent<WorldAnchor>(),//当前网格对象的MeshCollider组件surface.GetComponent<MeshCollider>(),//每立方米网格三角形的数量TrianglesPerCubicMeter,true);surfaceWorkQueue.Enqueue(surfaceData);}/// <summary>/// Called when the GameObject is unloaded./// </summary>private void OnDestroy(){// Stop the observer.StopObserving();observer.Dispose();observer = null;// Clear our surface mesh collection.surfaces.Clear();}}
}

SpatialMappingManager.cs 主要是对Surface的材质，是否显示网格等一些参数进行配置获取管理

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.using System.Collections.Generic;
using UnityEngine;namespace HoloToolkit.Unity
{/// <summary>/// The SpatialMappingManager class allows applications to use a SurfaceObserver or a stored /// Spatial Mapping mesh (loaded from a file)./// When an application loads a mesh file, the SurfaceObserver is stopped./// Calling StartObserver() clears the stored mesh and enables real-time SpatialMapping updates./// </summary>[RequireComponent(typeof(SpatialMappingObserver))]public partial class SpatialMappingManager : Singleton<SpatialMappingManager>{[Tooltip("The physics layer for spatial mapping objects to be set to.")]public int PhysicsLayer = 31;[Tooltip("The material to use for rendering spatial mapping data.")]public Material surfaceMaterial;[Tooltip("Determines if the surface observer should be automatically started.")]public bool autoStartObserver = true;[Tooltip("Determines if spatial mapping data will be rendered.")]public bool drawVisualMeshes = false;[Tooltip("Determines if spatial mapping data will cast shadows.")]public bool castShadows = false;/// <summary>/// Used for gathering real-time Spatial Mapping data on the HoloLens./// </summary>private SpatialMappingObserver surfaceObserver;/// <summary>/// Time when StartObserver() was called./// </summary>[HideInInspector]public float StartTime { get; private set; }/// <summary>/// The current source of spatial mapping data./// </summary>public SpatialMappingSource Source { get; private set; }// Called when the GameObject is first created.private void Awake(){surfaceObserver = gameObject.GetComponent<SpatialMappingObserver>();Source = surfaceObserver;}// Use for initialization.private void Start(){if (autoStartObserver){StartObserver();}}/// <summary>/// Returns the layer as a bit mask./// </summary>public int LayerMask{get { return (1 << PhysicsLayer); }}/// <summary>/// The material to use when rendering surfaces./// </summary>public Material SurfaceMaterial{get{return surfaceMaterial;}set{if (value != surfaceMaterial){surfaceMaterial = value;SetSurfaceMaterial(surfaceMaterial);}}}/// <summary>/// Specifies whether or not the SpatialMapping meshes are to be rendered./// </summary>public bool DrawVisualMeshes{get{return drawVisualMeshes;}set{if (value != drawVisualMeshes){drawVisualMeshes = value;UpdateRendering(drawVisualMeshes);}}}/// <summary>/// Specifies whether or not the SpatialMapping meshes can cast shadows./// </summary>public bool CastShadows{get{return castShadows;}set{if (value != castShadows){castShadows = value;SetShadowCasting(castShadows);}}}/// <summary>/// Sets the source of surface information./// </summary>/// <param name="mappingSource">The source to switch to. Null means return to the live stream if possible.</param>public void SetSpatialMappingSource(SpatialMappingSource mappingSource){UpdateRendering(false);if (mappingSource == null){Source = surfaceObserver;}else{Source = mappingSource;}UpdateRendering(DrawVisualMeshes);}/// <summary>/// Sets the material used by all Spatial Mapping meshes./// </summary>/// <param name="surfaceMaterial">New material to apply.</param>public void SetSurfaceMaterial(Material surfaceMaterial){SurfaceMaterial = surfaceMaterial;if (DrawVisualMeshes){foreach (Renderer renderer in Source.GetMeshRenderers()){if (renderer != null){renderer.sharedMaterial = surfaceMaterial;}}}}/// <summary>/// Checks to see if the SurfaceObserver is currently running./// </summary>/// <returns>True, if the observer state is running.</returns>public bool IsObserverRunning(){return surfaceObserver.ObserverState == ObserverStates.Running;}/// <summary>/// Instructs the SurfaceObserver to start updating the SpatialMapping mesh./// </summary>public void StartObserver(){
#if !UNITY_EDITORif (!IsObserverRunning()){surfaceObserver.StartObserving();StartTime = Time.time;}
#endif}/// <summary>/// Instructs the SurfaceObserver to stop updating the SpatialMapping mesh./// </summary>public void StopObserver(){
#if !UNITY_EDITORif (IsObserverRunning()){surfaceObserver.StopObserving();}
#endif}/// <summary>/// Gets all meshes that are associated with the SpatialMapping mesh./// </summary>/// <returns>/// Collection of Mesh objects representing the SpatialMapping mesh./// </returns>public List<Mesh> GetMeshes(){List<Mesh> meshes = new List<Mesh>();List<MeshFilter> meshFilters = GetMeshFilters();// Get all valid mesh filters for observed surfaces.foreach (MeshFilter filter in meshFilters){// GetMeshFilters ensures that both filter and filter.sharedMesh are not null.meshes.Add(filter.sharedMesh);}return meshes;}/// <summary>/// Gets all Mesh Filter objects associated with the Spatial Mapping mesh./// </summary>/// <returns>Collection of Mesh Filter objects.</returns>public List<MeshFilter> GetMeshFilters(){return Source.GetMeshFilters();}/// <summary>/// Sets the Cast Shadows property for each Spatial Mapping mesh renderer./// </summary>private void SetShadowCasting(bool castShadows){CastShadows = castShadows;foreach (Renderer renderer in Source.GetMeshRenderers()){if (renderer != null){if (castShadows){renderer.shadowCastingMode = UnityEngine.Rendering.ShadowCastingMode.On;}else{renderer.shadowCastingMode = UnityEngine.Rendering.ShadowCastingMode.Off;}}}}/// <summary>/// Updates the rendering state on the currently enabled surfaces./// Updates the material and shadow casting mode for each renderer./// </summary>/// <param name="Enable">True, if meshes should be rendered.</param>private void UpdateRendering(bool Enable){List<MeshRenderer> renderers = Source.GetMeshRenderers();for (int index = 0; index < renderers.Count; index++){if (renderers[index] != null){renderers[index].enabled = Enable;if (Enable){renderers[index].sharedMaterial = SurfaceMaterial;}}}}}
}

ObjectSurfaceObserver.cs主要用于当处于Unity编辑环境下时，加载房间模型数据，来进行测试，真机环境下该脚本没啥用处

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.using UnityEngine;namespace HoloToolkit.Unity
{public class ObjectSurfaceObserver : SpatialMappingSource{[Tooltip("The room model to use when loading meshes in Unity.")]public GameObject roomModel;// Use this for initialization.private void Start(){
#if UNITY_EDITOR// When in the Unity editor, try loading saved meshes from a model.Load(roomModel);if (GetMeshFilters().Count > 0){SpatialMappingManager.Instance.SetSpatialMappingSource(this);}
#endif}/// <summary>/// Loads the SpatialMapping mesh from the specified room object./// </summary>/// <param name="roomModel">The room model to load meshes from.</param>public void Load(GameObject roomModel){if (roomModel == null){Debug.Log("No room model specified.");return;}GameObject roomObject = GameObject.Instantiate(roomModel);Cleanup();try{MeshFilter[] roomFilters = roomObject.GetComponentsInChildren<MeshFilter>();foreach (MeshFilter filter in roomFilters){GameObject surface = AddSurfaceObject(filter.sharedMesh, "roomMesh-" + surfaceObjects.Count, transform);Renderer renderer = surface.GetComponent<MeshRenderer>();if (SpatialMappingManager.Instance.DrawVisualMeshes == false){renderer.enabled = false;}if (SpatialMappingManager.Instance.CastShadows == false){renderer.shadowCastingMode = UnityEngine.Rendering.ShadowCastingMode.Off;}// Reset the surface mesh collider to fit the updated mesh. // Unity tribal knowledge indicates that to change the mesh assigned to a// mesh collider, the mesh must first be set to null.  Presumably there// is a side effect in the setter when setting the shared mesh to null.MeshCollider collider = surface.GetComponent<MeshCollider>();collider.sharedMesh = null;collider.sharedMesh = surface.GetComponent<MeshFilter>().sharedMesh;}}catch{Debug.Log("Failed to load object " + roomModel.name);}finally{if (roomModel != null && roomObject != null){GameObject.DestroyImmediate(roomObject);}}}}
}

2、新增一个Cube Prefab，并且添加脚本组件CubeScript.cs

using UnityEngine;
using System.Collections;public class CubeScript : MonoBehaviour {// Use this for initializationvoid Start () {}// Update is called once per frame//刚启动应用时，空间映射还没准备好，将创建的Cube释放掉void Update () {if (transform.position.y < -3){Destroy(gameObject);}}
}

3、在MainCamera上新增脚本组件CubeCreator.cs，用于每隔一秒钟产生一个Cube,用于测试

using UnityEngine;
using System.Collections;public class CubeCreator : MonoBehaviour {public GameObject cubePrefab;// Use this for initializationvoid Start () {StartCoroutine(CreateCube());}// Update is called once per framevoid Update () {}private IEnumerator CreateCube(){while (true){float r = 1.5f;var theta = transform.rotation.eulerAngles.y * Mathf.Deg2Rad;var x = r * Mathf.Sin(theta);var z = r * Mathf.Cos(theta);Instantiate(cubePrefab,new Vector3(x, 1, z),Quaternion.Euler(0, transform.rotation.eulerAngles.y, z));yield return new WaitForSeconds(1);}}
}

4、在SpatialMapping上添加DrawMeshChanger.cs脚本组件，用于改变Surface的材质，一个是带网格的，一个是没有网格线的

using UnityEngine;
using System.Collections;
using HoloToolkit.Unity;
using UnityEngine.VR.WSA.Input;
using System;public class DrawMeshChanger : MonoBehaviour {GestureRecognizer recognizer;public bool isWireframe = true;public Material Wireframe;public Material Occlusion;// Use this for initializationvoid Start () {recognizer = new GestureRecognizer();recognizer.TappedEvent += Recognizer_TappedEvent;recognizer.StartCapturingGestures();}private void Recognizer_TappedEvent(InteractionSourceKind source, int tapCount, Ray headRay){SpatialMappingManager.Instance.SetSurfaceMaterial(isWireframe ? Occlusion : Wireframe);isWireframe = !isWireframe;}// Update is called once per framevoid Update () {}
}

5、为了使用空间映射数据，SpatialPerception能力必须被启用

6、运行测试

能够看到在现实世界的物体表面附着一层网格线，落下的Cube可以在桌子表面，或者落到了地面上。

当在环境中进行点击，可以进行切换Surface的材质。当点击后可以看到网格消失了。

Hololens入门之空间映射相关推荐

HoloLens开发手记 - 空间映射(SpatialMapping)
空间映射提供了HoloLens周围环境中真实世界表面的详细表示,允许开发人员创建令人信服的混合现实体验.通过将真实世界与虚拟世界合并,应用可以使全息图看起来是真实的.通过提供熟悉的现实世界行为和交互, ...
[洪流学堂]Hololens开发高级篇5：空间映射（Spatial mapping）
本教程基于Unity2017.2及Visual Studio 2017 本教程编写时间:2017年12月16日本文内容提要空间映射让holograms了解周围环境,将真实世界和虚拟世界更好地结合在 ...
Hololens官方教程精简版 - 07. Spatial mapping（空间映射）
前言注意:本文已更新到5.5.1f1版本个人建议,学习Holograms 230之前,一定完成<Hololens官方教程精简版 - 02. Introduction with Device& ...
基于Hololens开发---本地化空间锚点
基于Hololens开发-本地化空间锚点本地化空间锚基于Hololens的空间映射,本项目本章内容主要是对Hololens端的离线瞄点进行保存,当再次启用项目时将数据进行读取重置当前位置.具体过程见 ...
使用Unity 3D开发Hololens入门教程
Microsoft已经发布了官方的Hololens SDK,本文将深入介绍使用Emulator(模拟器)开发Hololens,教大家如何使用Visual Studio 和 Unity 3D打造你的第一 ...
【camera】自动驾驶感知系统实现（车道线检测和拟合、目标检测与跟踪、道路可行驶区域分割、深度估计、图像视野到BEV空间映射、像平面到地平面映射）
自动驾驶感知系统实现(车道线检测和拟合.目标检测与跟踪.道路可行驶区域分割.深度估计.图像视野到BEV空间映射.像平面到地平面映射) 项目下载地址:项目下载地址推理引擎下载地址:推理引擎下载地址支 ...
基于BERT的化学空间映射
目录背景介绍与方法概述 Related Work 方法概述结果与讨论化学反应分类可视化注意力分布化学空间映射(Mapping the chemical reaction space) 背景介 ...
空间映射网络--Spatial Transformer Networks
Spatial Transformer Networks 主要对目标在特征空间做不变性归一化解决角度.尺度等变形引入的影响 Code: https://github.com/skaae/trans ...
如何将网盘空间映射为本地磁盘，可以看看这些方案
前言相信大部分的朋友都会在日常工作中使用到网盘服务,比如我们日常几乎天天打交道的操作系统 Windows 就内置了 OneDrive.Apple 生态系统中最为核心的云服务 iCloud 中也包含了 ...

Hololens入门之空间映射

Hololens入门之空间映射相关推荐

最新文章

热门文章