PyRealsense开发参考资料汇总和总结（VScode+pyrs+PyQt5+OpenCV+Open3d）

前言
一、代码环境
二、VScode使用技巧
- 1.调试
- 2.VScode使用Git连接Gitee和GitHub
- 3.VScode常用快捷键
- 4.VScode添加右键菜单
三、PyRealsense与OpenCV
- 1.显示彩色图像和深度图像（colormap）
- 2.多相机管理device_manager
- 3.OpenCV双目视觉检测
四、PyRealsense和Open3d
- 1.显示3d点云
五、PyQt5
- 1.创建基本窗口
- 2.基础Pyqt5：Realsense点云显示与保存
- 3.线程管理与Pyqt5：实时显示多Realsense相机获取RGB图像
总结

前言

记录开发pyrs过程常用的代码结构，包括基础的pipeline和frame处理，结合PyQt和OpenCV开发交互界面显示彩色图像和深度图像，基于PyQt和Open3d开发点云显示，以及开发c++的函数库和应用。
如果未特别标明系统环境，则默认是在Windows环境下开发。
本文在作者毕业前长期更新，面向萌新，希望抛砖引玉，欢迎大佬批评指正。

一、代码环境

绝大部分代码是在VScode中完成，由于对VScode使用不精，少部分代码借鉴他人在Ubuntu18.04使用Cmake进行编译。

Windows10下：
VScode（C++）设置参考https://blog.csdn.net/qq_43041976/article/details/100542557
VScode配置Anaconda（Python）参考https://blog.csdn.net/zzly1126/article/details/84927785，注意Pyqt5必须使用版本大于3.5的Python
VScode配置PyQt5参考https://blog.csdn.net/weixin_40014984/article/details/104531359

Linux下：
由于需求和环境千差万别，还请灵活使用百度选择合适自己的搭配方案，自行研究（逃
以供参考：
linux安装anacondahttps://blog.csdn.net/qq_53564294/article/details/120535377

二、VScode使用技巧

1.调试

VScode调试python代码时在launch.json中添加参数（args）（c++添加参数同理）：https://blog.csdn.net/zk0272/article/details/83105574

2.VScode使用Git连接Gitee和GitHub

参考：https://blog.csdn.net/qq_38981614/article/details/115013188

git教程可以参考廖雪峰的网站：https://www.liaoxuefeng.com/wiki/896043488029600

3.VScode常用快捷键

Windows与Ubuntu通用常用指令：
Ctrl+K Ctrl+C 快速注释
Ctrl+K Ctrl+U 快速取消注释
Ctrl+K Ctrl+F 快速整理代码格式（需要IntelliCode）
F1 万能键，打开命令面板
Ctrl+F 查找

更多可以参考https://www.cnblogs.com/schut/p/10461840.html

4.VScode添加右键菜单

1）在桌面鼠标右键新建一个文本文件（txt文件），双击打开复制下面的代码

Windows Registry Editor Version 5.00[HKEY_CLASSES_ROOT\*\shell\VSCode]
@="Open with Code"
"Icon"="C:\\Program Files\\Microsoft VS Code\\Code.exe"[HKEY_CLASSES_ROOT\*\shell\VSCode\command]
@="\"C:\\Program Files\\Microsoft VS Code\\Code.exe\" \"%1\""Windows Registry Editor Version 5.00[HKEY_CLASSES_ROOT\Directory\shell\VSCode]
@="Open with Code"
"Icon"="C:\\Program Files\\Microsoft VS Code\\Code.exe"[HKEY_CLASSES_ROOT\Directory\shell\VSCode\command]
@="\"C:\\Program Files\\Microsoft VS Code\\Code.exe\" \"%V\""Windows Registry Editor Version 5.00[HKEY_CLASSES_ROOT\Directory\Background\shell\VSCode]
@="Open with Code"
"Icon"="C:\\Program Files\\Microsoft VS Code\\Code.exe"[HKEY_CLASSES_ROOT\Directory\Background\shell\VSCode\command]
@="\"C:\\Program Files\\Microsoft VS Code\\Code.exe\" \"%V\""

2）将里面所有VScode的路径改成自己的VScode安装目录（注意路径中的双反斜杠），保存，然后将这个文件后缀改为: .reg，文件名任意，例如：0.reg
3）双击运行，全部选是

参考：https://yiliang.blog.csdn.net/article/details/82194913

三、PyRealsense与OpenCV

PyRealsense官方网站：https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python
这里面有PyRealsense的安装以及一些examples，方便入手

要用到OpenCV-python和Numpy，请提前安装

pip install opencv-python==3.4.2.17
pip install numpy

想了解OpenCV的详细安装过程以及一些常用操作可以参考：https://blog.csdn.net/qq_45066628/article/details/119342697

1.显示彩色图像和深度图像（colormap）

## License: Apache 2.0. See LICENSE file in root directory.
## Copyright(c) 2015-2017 Intel Corporation. All Rights Reserved.###############################################
##      Open CV and Numpy integration        ##
###############################################import pyrealsense2 as rs
import numpy as np
import cv2# Configure depth and color streams
pipeline = rs.pipeline()  # 该管道简化了用户与设备和计算机视觉处理模块的交互。
config = rs.config()  # 配置允许管道用户请求管道流和设备选择和配置的过滤器。# Get device product line for setting a supporting resolution
pipeline_wrapper = rs.pipeline_wrapper(pipeline)
pipeline_profile = config.resolve(pipeline_wrapper)
device = pipeline_profile.get_device()
device_product_line = str(device.get_info(rs.camera_info.product_line))# 提取设备产品线信息# 判断是否有RGB camera
found_rgb = False
for s in device.sensors:if s.get_info(rs.camera_info.name) == 'RGB Camera':found_rgb = Truebreak
if not found_rgb:print("The demo requires Depth camera with Color sensor")exit(0)config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)# 启动深度图像流if device_product_line == 'L500':config.enable_stream(rs.stream.color, 960, 540, rs.format.bgr8, 30)
else:config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)# 启动彩色图像流，注意分辨率为640*480与深度图保持一致，注意是bgr8# Start streaming
pipeline.start(config)
try:while True:# Wait for a coherent pair of frames: depth and colorframes = pipeline.wait_for_frames()depth_frame = frames.get_depth_frame() # 变量类型为pyrealsense2.depth_framecolor_frame = frames.get_color_frame() # 变量类型为pyrealsense2.video_frame，get_infrared_frame获得的变量类型也是pyrealsense2.video_frameif not depth_frame or not color_frame:continue# Convert images to numpy arraysdepth_image = np.asanyarray(depth_frame.get_data())color_image = np.asanyarray(color_frame.get_data())# Apply colormap on depth image (image must be converted to 8-bit per pixel first)depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET)depth_colormap_dim = depth_colormap.shapecolor_colormap_dim = color_image.shape# If depth and color resolutions are different, resize color image to match depth image for displayif depth_colormap_dim != color_colormap_dim:resized_color_image = cv2.resize(color_image, dsize=(depth_colormap_dim[1], depth_colormap_dim[0]), interpolation=cv2.INTER_AREA)images = np.hstack((resized_color_image, depth_colormap))else:# images = np.vstack((color_image, depth_colormap))# 将彩色图和深度图沿着第一轴(高度)进行连接images = np.hstack((color_image, depth_colormap))# 将彩色图和深度图沿着第二轴(宽度)进行连接# images = np.dstack((color_image, depth_colormap))# 将彩色图和深度图沿着第三轴(rgb通道)进行连接（此处无效）# Show imagescv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)cv2.imshow('RealSense', images)cv2.waitKey(1)finally:# Stop streamingpipeline.stop()

对pipeline.start(config)的补充说明
根据配置启动管道流。管道流以循环的方式从设备中捕获样本，并根据每个模块的需求和线程模型将它们发送到附加的计算机视觉模块和处理块。在循环执行期间，应用程序可以通过调用wait_for_frames()或poll_for_frames()来访问摄像机流。流循环一直运行，直到管道停止。只有当管道未启动时，才可能启动它。如果管道已启动，则会引发一个异常。管道在启动时根据配置或默认配置选择并激活设备。当rs2::config被提供给该方法时，管道会尝试激活config.resolve()结果。如果应用程序请求与管道计算机视觉模块冲突，或者平台上没有匹配的设备，则该方法失败。在config.resolve()调用和管道启动之间，可用的配置和设备可能会发生变化，如果设备被连接或断开，或者另一个应用程序获得了设备的所有权。
该函数返回一个pyrealsense2.pipeline_profile

两种获取帧（frame）的方式，帧的格式：pyrealsense2.composite_frame
1）poll_for_frames()详解
检查是否有一组新的帧可用，并检索最新未交付的帧集。帧集包括管道中每个启用的流的时间同步帧。该方法返回时不会阻塞调用线程，并且返回新帧的状态是否可用。如果可用，则获取最新的帧集。在函数未被调用时产生的设备帧被删除。为了避免帧下降，调用该方法的速度应该与设备的帧速率一样快。应用程序可以维护框架句柄以延迟处理。但是，如果应用程序的历史保存时间过长，设备可能会缺少内存资源来产生新的帧，接下来调用这个方法将不会返回新的帧，直到资源可用为止。
2）wait_for_frames()详解
等待一组新的帧可用。帧集包括管道中每个启用的流的时间同步帧。在流的帧率不同的情况下，帧集包含慢流的匹配帧，该帧可能已经包含在前面的帧集中。该方法阻塞调用线程，并获取最新的未读帧集。在函数未被调用时产生的设备帧被删除。为了避免帧下降，调用该方法的速度应该与设备的帧速率一样快。应用程序可以维护框架句柄以延迟处理。但是，如果应用程序的历史保存时间过长，设备可能会缺少产生新帧的内存资源，接下来调用这个方法将无法获取新帧，直到资源可用为止。

获取图像的简要代码块如下：

pipeline = rs.pipeline()  # 该管道简化了用户与设备和计算机视觉处理模块的交互。
config = rs.config()  # 配置允许管道用户请求管道流和设备选择和配置的过滤器。
config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)# 启动深度图像流
config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)# 启动彩色图像流，注意分辨率为640*480与深度图保持一致，注意是bgr8
pipeline.start(config) # Start streaming
try:while True:# Wait for a coherent pair of frames: depth and colorframes = pipeline.wait_for_frames()depth_frame = frames.get_depth_frame() # 变量类型为pyrealsense2.depth_framecolor_frame = frames.get_color_frame() # 变量类型为pyrealsense2.video_frame# Convert images to numpy arraysdepth_image = np.asanyarray(depth_frame.get_data())color_image = np.asanyarray(color_frame.get_data())# Apply colormap on depth image (image must be converted to 8-bit per pixel first)depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET)images = np.hstack((color_image, depth_colormap))# 将彩色图和深度图沿着第二轴(宽度)进行连接# Show imagescv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE)cv2.imshow('RealSense', images)cv2.waitKey(1)
finally:pipeline.stop() # Stop streaming

需要注意的是，在ubuntu环境下使用VScode运行这段代码可能会遇到qt.qpa.plugin: Could not find the Qt platform plugin “xcb” 的问题，这是vscode和imshow出现了兼容问题
为了查看更具体的原因，需要在Terminal中输入

export QT_DEBUG_PLUGINS=1

然后再运行一次原代码，得到的报错信息中查看具体的问题，包括是否缺失库文件或者是远程连接没有正确设置显示器
可以在代码中添加

if platform.system()=="Linux":envpath = '/home/lirq/anaconda3/envs/Pyrealsense-py3-8/lib/python3.8/site-packages/cv2/qt/plugins/platforms'os.environ['QT_QPA_PLATFORM_PLUGIN_PATH'] = envpath

其中envpath换成自己的anaconda环境中qt的platforms文件夹路径即可
作者遇到的问题是could not connect to display:0，可以通过重新安装opencv解决问题

pip uninstall opencv-python
pip install opencv-contrib-python
sudo apt-get install opencv-python

重启Ubuntu之后就可以顺利使用VSCode进行imshow

2.多相机管理device_manager

import pyrealsense2 as rs
import cv2
import numpy as np
from Calibration.realsense_device_manager import DeviceManager # 多相机管理
# Define some constants
resolution_width = 1280 # pixels
resolution_height = 720 # pixels
frame_rate = 30  # fps
dispose_frames_for_stablisation = 30  # frames the auto-exposure controller to stablise
cv2.namedWindow('RealSense', cv2.WINDOW_AUTOSIZE) #窗口设定
try:# Enable the streams from all the intel realsense devicesrs_config = rs.config()rs_config.enable_stream(rs.stream.depth, resolution_width, resolution_height, rs.format.z16, frame_rate)rs_config.enable_stream(rs.stream.infrared, 1, resolution_width, resolution_height, rs.format.y8, frame_rate)rs_config.enable_stream(rs.stream.color, resolution_width, resolution_height, rs.format.bgr8, frame_rate)# Use the device manager class to enable the devices and get the framesdevice_manager = DeviceManager(rs.context(), rs_config)device_manager.enable_all_devices()# Allow some frames for the auto-exposure controller to stablisefor frame in range(dispose_frames_for_stablisation):frames = device_manager.poll_frames()assert( len(device_manager._available_devices) > 0 )# Continue acquisition until terminated with Ctrl+C by the userwhile 1:'''#################原版################### Get the frames from all the devicesframes_devices = device_manager.poll_frames()# TODO: 对每个相机流进行操作for (device_info, frame) in frames_devices.items():#dev_info = (serial, device.product_line)device = device_info[0] #serial number color_image = np.asarray(frame[rs.stream.color].get_data())#frame[rs.stream.color]的类型为pyrealsense2.video_frame# Visualise the resultscv2.imshow('Color image from RealSense Device Nr: ' + device, color_image)cv2.waitKey(1)#################原版##################'''# 对每个相机流进行操作for (serial, device) in device_manager._enabled_devices.items():#device_manager._enabled_devices[device_serial] = (Device(pipeline, pipeline_profile, product_line))# device.pipeline与pipeline = rs.pipeline()等价# serial: 相机编号# device: Device(pipeline, pipeline_profile, product_line) #见realSense_device_manager# TODO：对每个相机流进行操作 streams = device.pipeline_profile.get_streams()frames = device.pipeline.wait_for_frames() #frameset will be a pyrealsense2.composite_frame objectcolor_frame = frames.get_color_frame() color_image = np.asanyarray(color_frame.get_data())# Visualise the resultscv2.imshow('Color image from RealSense Device Nr: ' + serial, color_image)cv2.waitKey(1)except KeyboardInterrupt:print("The program was interupted by the user. Closing the program...")finally:device_manager.disable_streams()cv2.destroyAllWindows()

与原版相比，新版API与第一小节更加统一，需要注意的是，device_manager与多线程thread在Ubuntu下出现过不兼容的情况，但。
多相机管理核心代码如下：

from Calibration.realsense_device_manager import DeviceManager # 多相机管理
try:# Enable the streams from all the intel realsense devicesrs_config = rs.config()rs_config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)rs_config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)# Use the device manager class to enable the devices and get the framesdevice_manager = DeviceManager(rs.context(), rs_config)device_manager.enable_all_devices()assert( len(device_manager._available_devices) > 0 )# Continue acquisition until terminated with Ctrl+C by the userwhile 1:# 对每个相机流进行操作for (serial, device) in device_manager._enabled_devices.items()# TODO：对每个相机流分别进行操作 streams = device.pipeline_profile.get_streams()frames = device.pipeline.wait_for_frames() #frameset will be a pyrealsense2.composite_frame objectcolor_frame = frames.get_color_frame() color_image = np.asanyarray(color_frame.get_data())# Visualise the resultscv2.imshow('Color image from RealSense Device Nr: ' + serial, color_image)cv2.waitKey(1)
except KeyboardInterrupt:print("The program was interupted by the user. Closing the program...")
finally:device_manager.disable_streams()cv2.destroyAllWindows()

3.OpenCV双目视觉检测

四、PyRealsense和Open3d

由于PCL安装过于繁琐，而无论是Open3d-python还是Open3d（C++）安装都较为方便，因此选用Open3d
Open3d官方网站：http://www.open3d.org/docs/release/introduction.html

1.显示3d点云

五、PyQt5

有关在VSvode中PyQt5的安装详见：https://blog.csdn.net/weixin_40014984/article/details/104531359
PyQt5中文网：http://www.pyqt5.cn/
参考的超详细基础教程（内含视频链接）：https://blog.csdn.net/qq_40243295/article/details/105633221

1.创建基本窗口

在VScode左侧Explore中右键选择PYQT: New Form
为了方便本节选择Dialog，但平时开发最好选择MainWindow
做一个界面后保存为.ui文件，例如：

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0"><class>Dialog</class><widget class="QDialog" name="Dialog"><property name="geometry"><rect><x>0</x><y>0</y><width>400</width><height>300</height></rect></property><property name="windowTitle"><string>Dialog</string></property><widget class="QDialogButtonBox" name="buttonBox"><property name="geometry"><rect><x>30</x><y>240</y><width>341</width><height>32</height></rect></property><property name="orientation"><enum>Qt::Horizontal</enum></property><property name="standardButtons"><set>QDialogButtonBox::Cancel|QDialogButtonBox::Ok</set></property></widget><widget class="QLabel" name="label"><property name="geometry"><rect><x>110</x><y>100</y><width>54</width><height>12</height></rect></property><property name="text"><string>TextLabel</string></property></widget></widget><resources/><connections><connection><sender>buttonBox</sender><signal>accepted()</signal><receiver>Dialog</receiver><slot>accept()</slot><hints><hint type="sourcelabel"><x>248</x><y>254</y></hint><hint type="destinationlabel"><x>157</x><y>274</y></hint></hints></connection><connection><sender>buttonBox</sender><signal>rejected()</signal><receiver>Dialog</receiver><slot>reject()</slot><hints><hint type="sourcelabel"><x>316</x><y>260</y></hint><hint type="destinationlabel"><x>286</x><y>274</y></hint></hints></connection></connections>
</ui>

右键选中这个.ui文件，选择PYQT: Compile Form
会得到一个Ui_开头的.py文件，但我们不要修改内容，这个文件只是方便查看各个QT部件的属性，具体的功能关联要另建一个新的.py文件，实现前端与后端独立开发，互不影响
新建一个.py文件：

import sys
from PyQt5.QtWidgets import QApplication, QDialog
from Ui_Untitled import Ui_Dialog # 导入QtDesigner中绘制好的UI
if __name__ == '__main__':app = QApplication(sys.argv)MainWindow = QDialog()ui = Ui_Dialog()ui.setupUi(MainWindow)MainWindow.show()sys.exit(app.exec_())

保存即可直接运行
上面这种只是一个简单的Dialog，但这种形式不方便进行扩展，因此多用MainWindow。

2.基础Pyqt5：Realsense点云显示与保存

Ui_Mainwindow.py和Main.py代码如下（只保留基础代码）

# Ui_Mainwindow.py
from PyQt5 import QtCore, QtGui, QtWidgets
class Ui_MainWindow(object):def setupUi(self, MainWindow):MainWindow.setObjectName("MainWindow")MainWindow.resize(1312, 571)self.centralwidget = QtWidgets.QWidget(MainWindow)self.centralwidget.setObjectName("centralwidget")self.pushButton = QtWidgets.QPushButton(self.centralwidget)self.pushButton.setGeometry(QtCore.QRect(20, 20, 101, 25))self.pushButton.setObjectName("pushButton")self.label = QtWidgets.QLabel(self.centralwidget)self.label.setGeometry(QtCore.QRect(130, 20, 431, 17))self.label.setObjectName("label")self.label_show = QtWidgets.QLabel(self.centralwidget)self.label_show.setGeometry(QtCore.QRect(20, 60, 1280, 480))self.label_show.setObjectName("label_show")MainWindow.setCentralWidget(self.centralwidget)self.menubar = QtWidgets.QMenuBar(MainWindow)self.menubar.setGeometry(QtCore.QRect(0, 0, 1312, 22))self.menubar.setObjectName("menubar")MainWindow.setMenuBar(self.menubar)self.statusbar = QtWidgets.QStatusBar(MainWindow)self.statusbar.setObjectName("statusbar")MainWindow.setStatusBar(self.statusbar)self.retranslateUi(MainWindow)QtCore.QMetaObject.connectSlotsByName(MainWindow)def retranslateUi(self, MainWindow):_translate = QtCore.QCoreApplication.translateMainWindow.setWindowTitle(_translate("MainWindow", "MainWindow"))self.pushButton.setText(_translate("MainWindow", "获取点云"))self.label.setText(_translate("MainWindow", "export path"))self.label_show.setText(_translate("MainWindow", "TextLabel"))

# main.py
import sys
from PyQt5.QtWidgets import QApplication, QMainWindow, QMessageBox
from PyQt5.QtGui import QImage, QPixmap
from Ui_UI_GetPointCloud import Ui_MainWindow
from PyQt5.QtCore import QObject, pyqtSignalimport pyrealsense2 as rs
import cv2
import numpy as np
import open3d as o3dimport threading as th
import datetime as dt
import ctypes
import inspect
import re
import os
class MainWindow(QMainWindow, Ui_MainWindow):def __init__(self):super(MainWindow, self).__init__()# Set up the user interface from Designer.self.setupUi(self)self.pushButton.clicked.connect(self.pushButton_clicked)self.dis_update.connect(self.camera_view)self.savePointCLoud = Falseself.id = 0# 在对应的页面类的内部，与def定义的函数同级dis_update = pyqtSignal(QPixmap)def pushButton_clicked(self):self.savePointCLoud = True# 添加一个退出的提示事件def closeEvent(self, event):"""我们创建了一个消息框，上面有俩按钮：Yes和No.第一个字符串显示在消息框的标题栏，第二个字符串显示在对话框，第三个参数是消息框的俩按钮，最后一个参数是默认按钮，这个按钮是默认选中的。返回值在变量reply里。"""reply = QMessageBox.question(self, 'Message', "Are you sure to quit?",QMessageBox.Yes | QMessageBox.No, QMessageBox.No)# 判断返回值，如果点击的是Yes按钮，我们就关闭组件和应用，否则就忽略关闭事件if reply == QMessageBox.Yes:self.stop_thread(self.thread_camera)event.accept()else:event.ignore()def open_camera(self):# target选择开启摄像头的函数self.thread_camera = th.Thread(target=self.open_realsense)self.thread_camera.start()# 多线程有兼容性bugprint('Open Camera')def camera_view(self, c):# 调用setPixmap函数设置显示Pixmapself.label_show.setPixmap(c)# 调用setScaledContents将图像比例化显示在QLabel上self.label_show.setScaledContents(True)def _async_raise(self, tid, exctype):"""raises the exception, performs cleanup if needed"""tid = ctypes.c_long(tid)if not inspect.isclass(exctype):exctype = type(exctype)res = ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, ctypes.py_object(exctype))if res == 0:raise ValueError("invalid thread id")elif res != 1:# """if it returns a number greater than one, you're in trouble,# and you should call it again with exc=NULL to revert the effect"""ctypes.pythonapi.PyThreadState_SetAsyncExc(tid, None)raise SystemError("PyThreadState_SetAsyncExc failed")def stop_thread(self, thread):self._async_raise(thread.ident, SystemExit)def open_realsense(self):print('open_realsense')# Create a pipelinepipeline = rs.pipeline()# Create a config and configure the pipeline to stream#  different resolutions of color and depth streamsconfig = rs.config()config.enable_stream(rs.stream.depth, 640, 480, rs.format.z16, 30)config.enable_stream(rs.stream.color, 640, 480, rs.format.bgr8, 30)# Start streamingprofile = pipeline.start(config)# Getting the depth sensor's depth scale (see rs-align example for explanation)depth_sensor = profile.get_device().first_depth_sensor()depth_scale = depth_sensor.get_depth_scale()print("Depth Scale is: ", depth_scale)# We will be removing the background of objects more than#  clipping_distance_in_meters meters awayclipping_distance_in_meters = 4  # 4 meterclipping_distance = clipping_distance_in_meters / depth_scale# Color Intrinsics# intr = color_frame.profile.as_video_stream_profile().intrinsics# Create an align object# rs.align allows us to perform alignment of depth frames to others frames# The "align_to" is the stream type to which we plan to align depth frames.align_to = rs.stream.coloralign = rs.align(align_to)# Streaming looptry:while True:# Wait for a coherent pair of frames: depth and colorframes = pipeline.wait_for_frames()frames  = align.process(frames) # 帧对齐depth_frame = frames.get_depth_frame()color_frame = frames.get_color_frame()profile = frames.get_profile()if not depth_frame or not color_frame:continue# Convert images to numpy arraysdepth_image = np.asanyarray(depth_frame.get_data())color_image = np.asanyarray(color_frame.get_data())#实时显示当前深度图和彩色图depth_colormap = cv2.applyColorMap(cv2.convertScaleAbs(depth_image, alpha=0.03), cv2.COLORMAP_JET)# Remove background - Set pixels further than clipping_distance to greyRemoveBackground = Truegrey_color = 153depth_image_3d = np.dstack((depth_image, depth_image, depth_image))  # depth image is 1 channel, color is 3 channelsif RemoveBackground == True:depth_image_3d = np.where((depth_image_3d > clipping_distance) | (depth_image_3d < 0), grey_color,color_image)else:depth_image_3d = color_imageimages = np.hstack((depth_image_3d, depth_colormap))qimage = QImage(images, 1280, 480, QImage.Format_BGR888)pixmap = QPixmap.fromImage(qimage)self.dis_update.emit(pixmap)#获得点云数据o3d_color = o3d.geometry.Image(color_image)o3d_depth = o3d.geometry.Image(depth_image)rgbd_image = o3d.geometry.RGBDImage.create_from_color_and_depth(o3d_color, o3d_depth,depth_scale=1000.0,depth_trunc=3.0,convert_rgb_to_intensity=False)intrinsics = profile.as_video_stream_profile().get_intrinsics() #此处profile指向color_frame# intrinsics = color_frame.profile.as_video_stream_profile().intrinsics# 转换为open3d中的相机参数pinhole_camera_intrinsic = o3d.camera.PinholeCameraIntrinsic(intrinsics.width, intrinsics.height,intrinsics.fx, intrinsics.fy,intrinsics.ppx, intrinsics.ppy)# 获取pcdo3d_result = o3d.geometry.PointCloud.create_from_rgbd_image(rgbd_image,pinhole_camera_intrinsic)if(self.savePointCLoud == True):now_date = dt.datetime.now().strftime('%F')now_time = dt.datetime.now().strftime('%F_%H%M%S')path_ok = os.path.exists(now_date)if(path_ok == False):os.mkdir(now_date)if(os.path.isdir(now_date)):  # 在以当前日期命名的文件夹下id = str(self.id)self.id = self.id+1pc_full_path_ply = os.path.join('./', now_date, id + '.ply')pc_full_path_stl = os.path.join('./', now_date, id + '.stl')pc_full_path_pcd = os.path.join('./', now_date, id + '.pcd')print(pc_full_path_ply)self.label.setText(pc_full_path_ply)#输出.plyo3d.io.write_point_cloud(pc_full_path_ply, o3d_result)#read plymesh_ply = o3d.io.read_triangle_mesh(pc_full_path_ply)mesh_ply.compute_vertex_normals()# V_mesh 为ply网格的顶点坐标序列，shape=(n,3)，这里n为此网格的顶点总数，其实就是浮点型的x,y,z三个浮点值组成的三维坐标V_mesh = np.asarray(mesh_ply.vertices)# print("ply info:", mesh_ply)# print("ply vertices shape:", V_mesh.shape)# o3d.visualization.draw_geometries([mesh_ply], window_name="ply", mesh_show_wireframe=True)# ply -> stlmesh_stl = o3d.geometry.TriangleMesh()mesh_stl.vertices = o3d.utility.Vector3dVector(V_mesh)mesh_stl.compute_vertex_normals()# print("stl info:", mesh_stl)# o3d.visualization.draw_geometries([mesh_stl], window_name="stl")# 输出.stlo3d.io.write_triangle_mesh(pc_full_path_stl, mesh_stl)o3d_result.transform([[1,0,0,0],[0,-1,0,0],[0,0,-1,0],[0,0,0,1]])o3d.visualization.draw_geometries([o3d_result], window_name="pcd")# save pcdo3d.io.write_point_cloud(pc_full_path_pcd, o3d_result)print('PointCloud Saved ply stl pcd')self.savePointCLoud = False# time.sleep(DELAY)finally:pipeline.stop()if __name__ == "__main__":app = QApplication(sys.argv)w = MainWindow()w.show()w.open_camera()print('Hello World!')sys.exit(app.exec_())

3.线程管理与Pyqt5：实时显示多Realsense相机获取RGB图像