Abstract.

Conributions

Conclusion

[1] Avetisyan A , Khanova T , Choy C , et al. SceneCAD: Predicting Object Alignments and Layouts in RGB-D Scans[J]. 2020.

Fig. 1. Our method takes as input a 3D scan and a set of CAD models. We jointly detect
objects and layout elements in the scene. Each detected object or layout component
then forms a node in a graph neural network which estimates object-object relationships
and object-layout relationships. This holistic understanding of the scene enables results
in a lightweight CAD-based representation of the scene.

Abstract.

We present a novel approach to reconstructing lightweight, CAD-based representations of scanned 3D environments from commodity RGB-D sensors. Our key idea is to jointly optimize for both CAD model alignments as well as layout estimations of the scanned scene, explicitly modeling inter-relationships between objects-to-objects and objects-to-layout. Since object arrangement and scene layout are intrinsically coupled, we show that treating the problem jointly signicantly helps to produce globally-consistent representations of a scene. Object CAD
models are aligned to the scene by establishing dense correspondences between geometry, and we introduce a hierarchical layout prediction approach to estimate layout planes from corners and edges of the scene. To this end, we propose a message-passing graph neural network to model
the inter-relationships between objects and layout, guiding generation of a globally object alignment in a scene. By considering the global scene layout, we achieve signicantly improved CAD alignments compared to state-of-the-art methods, improving from 41.83% to 58.41% alignment accuracy on SUNCG and from 50.05% to 61.24% on ScanNet, respectively. The resulting CAD-based representations makes our method well-suited for applications in content creation such as augmented- or virtual reality.

Conributions

Conclusion

In this work we formulated a method to digitize 3D scans that goes beyond the
focus of objects in the scene. We propose a novel method that estimates the
layout of the scene by sequentially predicting corners, then edges and nally
quads in a fully differentiable way. The estimated layout is used in conjunction
with an object detector to predict contact relationships between objects and
the layout and ultimately to predict a CAD arrangement of the scene. We can
show that objects and the surrounding (scene layout) go hand in hand and are a
crucial factor towards full scene digitization and scene understanding. Objects
in the scene are often not arbitrarily arranged, for instance often cabinets are
leaned at walls or a table is surrounded by chairs in a dining room, hence
we leverage the inherent coupling between objects and layout structure in the
learning process. Our approach improves global CAD alignment accuracy by

learning those patterns on both real and synthetic scans. We hope that we can
encourage further research towards this avenue, and see as next immediate steps
for future work the necessity of texturing digitized shapes in order to enhance
the immersive experience in VR environments.

Fig. 7. Qualitative CAD alignment and layout estimation results on ScanNet [9] scans
(zoomed in views on the bottom). Our approach incorporating object and layout
relationships produces globally consistent alignments along with the room layout.

SceneCAD: Predicting Object Alignmentsand Layouts in RGB-D Scans相关推荐

【ECCV2020】完整论文集part2
ECCV2020将于2020年8月23-28日在线上举行,今年共接受了1361篇论文,本文是接收论列表的第二部分,第一部见链接 Paper ID Paper Title Category 2515 T ...
基于视觉的机器人抓取: 论文及代码(Vision-based Robotic Grasping: Papers and Codes)
本文同步于微信公众号:3D视觉前沿,欢迎大家关注. 本文总结了基于视觉的机器人抓取的相关论文及代码,同步于 GitHub. 机器人抓取必需的信息是相机系下抓取器的6DoF位姿,包括抓取器的3D位置和抓 ...
深度学习论文阅读目标检测篇（三）：Faster R-CNN《 Towards Real-Time Object Detection with Region Proposal Networks》
深度学习论文阅读目标检测篇(三):Faster R-CNN< Towards Real-Time Object Detection with Region Proposal Networks&g ...
目标检测经典论文——Faster R-CNN论文翻译：Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Net
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Faster R-CNN:通过Region ...
《Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks》论文翻译
原文:https://arxiv.org/abs/1506.01497 Faster R-CNN: Towards Real-Time ObjectDetection with Region Prop ...
python rgba_使用PIL将RGBA PNG转换为RGB
我正在使用PIL将使用Django上传的透明PNG图片转换为JPG文件.输出看起来坏了. 源文件透明源文件码 Image.open(object.logo.path).save('/tmp/out ...
基于深度学习的视频预测研究综述
原址:http://html.rhhz.net/tis/html/201707032.htm (收集材料ing,为论文做准备)[综述性文章,,,可以做背景资料] 莫凌飞, 蒋红亮, 李煊鹏摘要:近年 ...
ECCV2022论文列表（中英对照）
Paper ID Paper Title 论文标题 8 Learning Uncoupled-Modulation CVAE for 3D Action-Conditioned Human Motio ...
（十四：2020.08.28）CVPR 2014 追踪之论文纲要（译）
CVPR 2020 追踪之论文纲要(修正于2020.08.28) 讲在前面论文目录讲在前面论坛很多博客都对论文做了总结和分类,但就医学领域而言,对这些论文的筛选信息显然需要更加精细的把控,所以自 ...
手把手教你实现一个 Vue 进度条组件！
最近在个人的项目中,想对页面之间跳转的过程进行优化,想到了很多文档或 npm 等都用到的页面跳转进度条,于是便想自己去实现一个,特此记录. 来看下 npm 搜索组件时候的效果: so 下面咱们一起动手 ...

SceneCAD: Predicting Object Alignmentsand Layouts in RGB-D Scans

Abstract.

Conributions

Conclusion

SceneCAD: Predicting Object Alignmentsand Layouts in RGB-D Scans相关推荐

最新文章

热门文章