使用openMVG+openMVS对自制数据集三维重建(单相机图片序列)

1 对于单个相机拍摄的图像序列

很简单，将自己的图片放入一个文件夹，命令输入指向这个文件夹就好了：

1.1 Sequential & Incremental SfM pipeline

$ cd openMVG_Build/software/SfM/
$ python SfM_SequentialPipeline.py [full path image directory] [resulting directory]
$ python SfM_SequentialPipeline.py ~/home/user/data/ImageDataset_SceauxCastle/images ~/home/user/data/ImageDataset_SceauxCastle/Castle_Incremental_Reconstruction

1.2 Global SfM pipeline

$ cd openMVG_Build/software/SfM/
$ python SfM_GlobalPipeline.py [full path image directory] [resulting directory]
$ python SfM_GlobalPipeline.py ~/home/user/data/ImageDataset_SceauxCastle/images ~/home/user/data/ImageDataset_SceauxCastle/Castle_Global_Reconstruction

注意：一般拍摄图像中会有相机的参数信息，OpenMVG将尝试检索图像的像素焦距。如果没有，在 Scene Initialization stage我们可以使用-f X * 1.2 提供一个近似焦距，其中X = Max(image <Width, Height>)

1.3 DIY

在sfm过程中可能有一些额外的参数需要设置，可以分步骤，下面是自己用的.bash文件：

cd reconstruction
mkdir reconstruction_work
cd reconstruction_work
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_SfMInit_ImageListing.exe -i ..\images\ -d D:\Reconstruction\openmvg_v20r\share\openMVG\sensor_width_camera_database.txt -o .\matches
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_ComputeFeatures.exe -i .\matches\sfm_data.json -o .\matches
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_PairGenerator.exe -i .\matches\sfm_data.json -o .\matches\pairs.bin
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_ComputeMatches.exe -i .\matches\sfm_data.json -p .\matches\pairs.bin -o .\matches\matches.putative.bin
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_GeometricFilter.exe -i .\matches\sfm_data.json -m .\matches\matches.putative.bin -g f -o .\matches\matches.f.bin
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_SfM.exe -s INCREMENTAL -i .\matches\sfm_data.json -M .\matches\matches.f.bin -o .\output
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_ComputeSfM_DataColor.exe -i .\output\sfm_data.bin -o .\output\sfm_data_colorized.ply
mkdir .\mvs
D:\Reconstruction\openmvg_v20r\bin\openMVG_main_openMVG2openMVS.exe -i .\output\sfm_data.bin -d .\mvs\undistortedImages -o .\mvs\scene.mvs

本人使用的是某车载摄像头的数据，图片上没有内参信息，运行上指令会在重建部分报错：

ERROR: [sequential_SfM.cpp:110] Unable to choose an initial pair, since there is no defined intrinsic data.
INFO: [sequential_SfM.cpp:173] Cannot find a valid initial pair - stop reconstruction.

因此需要在第一步openMVG_main_SfMInit_ImageListing生成sfm_data.json文件前手动添加的额外参数：

[-f|–focal] (value in pixels)
[-k|–intrinsics] Kmatrix: “f;0;ppx;0;f;ppy;0;0;1”
[-c|–camera_model] Camera model type:
- 1: Pinhole
- 2: Pinhole radial 1
- 3: Pinhole radial 3 (default)
[-g|–group_camera_model]
- 0-> each view have it’s own camera intrinsic parameters
- 1-> (default) view can share some camera intrinsic parameters

查看图像的参数：

设置上述内参，因为是同一个相机的图像设置g为1注意内参用分号隔开：

D:\Reconstruction\openmvg_v20r\bin\openMVG_main_SfMInit_ImageListing.exe -i ..\images\ -d D:\Reconstruction\openmvg_v20r\share\openMVG\sensor_width_camera_database.txt -o .\matches -k "3656.311177; 0.0; 1225.812961; 0.0; 3671.211662; 993.655164; 0.0; 0.0; 1.0" -g 1

格式输入错误运行bash参数会报错：

ERROR: [main_SfMInit_ImageListing.cpp:45]Missing ';' character
ERROR: [main_SfMInit_ImageListing.cpp:238] Invalid K matrix input

正确格式运行应该会完成重建，数据集有问题可能无法重建：

ERROR: [sequential_SfM.cpp:534]  /!\ Robust estimation failed to compute E for this pair: {4,5}

这里应该是车载数据集的视差太小了，连续帧前后估计有困难。自制数据集应尽量保证在重复区域足够多的前提下扩大视角。

openMVG的sfm pipeline成功运行后，输出三个子文件夹：

分别包含特征匹配部分、重建部分和mvs格式转换的内容
output中文件sfm_data.bin就是重建的数据

1.4 openMVS

在进行OpenMVS步骤之前还需要将上一步生成的sfm_data.bin转化成mvs格式，该步在1.3bash最后一行已经执行，这一步会产生两个输出

一个就是scene.mvs这个文件
另一个就是一个undistorted_images文件夹，里面是经过畸变校正的图像

下面使用MVS进行网格化和纹理贴图：
为了方便，直接把mvs文件夹复制到MVS的exe所在文件夹下

可以先查看mvs文件中的稀疏点云：

Viewer.exe mvs\scene.mvs

生成稠密点云：

DensifyPointCloud.exe mvs\scene.mvs

查看稠密点云：

Viewer.exe mvs\scene_dense.mvs

建立粗网格：

ReconstructMesh.exe mvs\scene_dense.mvs

查看粗网格：

Viewer.exe mvs\scene_dense_mesh.mvs

细化网格（可选，比较耗时：

RefineMesh.exe mvs\scene_dense_mesh.mvs

查看细化网格：

Viewer.exe mvs\scene_dense_mesh_refine.mvs

加纹理：

TextureMesh.exe mvs\scene_dense_mesh_refine.mvs

查看结果：

Viewer.exe mvs\scene_dense_mesh_refine_texture.mvs

1.5 汇总：

完整的流程，设置好输入输出就可以了：

#!/usr/bin/python3
# -*- encoding: utf-8 -*-
#
# Created by @FlachyJoe
"""
This script is for an easy use of OpenMVG and OpenMVSusage: MvgMvs_Pipeline.py [-h] [--steps STEPS [STEPS ...]] [--preset PRESET][--0 0 [0 ...]] [--1 1 [1 ...]] [--2 2 [2 ...]][--3 3 [3 ...]] [--4 4 [4 ...]] [--5 5 [5 ...]][--6 6 [6 ...]] [--7 7 [7 ...]] [--8 8 [8 ...]][--9 9 [9 ...]] [--10 10 [10 ...]] [--11 11 [11 ...]][--12 12 [12 ...]] [--13 13 [13 ...]][--14 14 [14 ...]] [--15 15 [15 ...]][--16 16 [16 ...]] [--17 17 [17 ...]]input_dir output_dirPhotogrammetry reconstruction with these steps:0. Intrinsics analysis             openMVG_main_SfMInit_ImageListing1. Compute features                openMVG_main_ComputeFeatures2. Compute pairs                   openMVG_main_PairGenerator3. Compute matches                 openMVG_main_ComputeMatches4. Filter matches                  openMVG_main_GeometricFilter5. Incremental reconstruction      openMVG_main_IncrementalSfM6. Global reconstruction           openMVG_main_GlobalSfM7. Colorize Structure              openMVG_main_ComputeSfM_DataColor8. Structure from Known Poses      openMVG_main_ComputeStructureFromKnownPoses9. Colorized robust triangulation  openMVG_main_ComputeSfM_DataColor10. Control Points Registration    ui_openMVG_control_points_registration11. Export to openMVS              openMVG_main_openMVG2openMVS12. Densify point-cloud            DensifyPointCloud13. Reconstruct the mesh           ReconstructMesh14. Refine the mesh                RefineMesh15. Texture the mesh               TextureMesh16. Estimate disparity-maps        DensifyPointCloud17. Fuse disparity-maps            DensifyPointCloudpositional arguments:input_dir                 the directory which contains the pictures set.output_dir                the directory which will contain the resulting files.optional arguments:-h, --help                show this help message and exit--steps STEPS [STEPS ...] steps to process--preset PRESET           steps list preset inSEQUENTIAL = [0, 1, 2, 3, 4, 5, 11, 12, 13, 14, 15]GLOBAL = [0, 1, 2, 3, 4, 6, 11, 12, 13, 14, 15]MVG_SEQ = [0, 1, 2, 3, 4, 5, 7, 8, 9]MVG_GLOBAL = [0, 1, 2, 3, 4, 6, 7, 8, 9]MVS = [12, 13, 14, 15]MVS_SGM = [16, 17]default : SEQUENTIALPassthrough:Option to be passed to command lines (remove - in front of option names)e.g. --1 p ULTRA to use the ULTRA preset in openMVG_main_ComputeFeaturesFor example, running the script[MvgMvsPipeline.py input_dir output_dir --steps 0 1 2 3 4 5 11 12 13 15 --1 p HIGH n 8 --3 n HNSWL2][--steps 0 1 2 3 4 5 11 12 13 15] runs only the desired steps[--1 p HIGH n 8] where --1 refer to openMVG_main_ComputeFeatures,p refers to describerPreset option and set to HIGH, and n refersto numThreads and set to 8. The second step (Compute matches),[--3 n HNSWL2] where --3 refer to openMVG_main_ComputeMatches,n refers to nearest_matching_method option and set to HNSWL2
"""import os
import subprocess
import sys
import argparseDEBUG = Falseif sys.platform.startswith('win'):PATH_DELIM = ';'FOLDER_DELIM = '\\'
else:PATH_DELIM = ':'FOLDER_DELIM = '/'# add this script's directory to PATH
os.environ['PATH'] += PATH_DELIM + os.path.dirname(os.path.abspath(__file__))# add current directory to PATH
os.environ['PATH'] += PATH_DELIM + os.getcwd()def whereis(afile):"""return directory in which afile is, None if not found. Look in PATH"""if sys.platform.startswith('win'):cmd = "where"else:cmd = "which"try:ret = subprocess.run([cmd, afile], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, check=True)return os.path.split(ret.stdout.decode())[0]except subprocess.CalledProcessError:return Nonedef find(afile):"""As whereis look only for executable on linux, this find look for all file type"""for d in os.environ['PATH'].split(PATH_DELIM):if os.path.isfile(os.path.join(d, afile)):return dreturn None# Try to find openMVG and openMVS binaries in PATH
OPENMVG_BIN = whereis("openMVG_main_SfMInit_ImageListing")
OPENMVS_BIN = whereis("ReconstructMesh")# Try to find openMVG camera sensor database
CAMERA_SENSOR_DB_FILE = "sensor_width_camera_database.txt"
CAMERA_SENSOR_DB_DIRECTORY = find(CAMERA_SENSOR_DB_FILE)# Ask user for openMVG and openMVS directories if not found
if not OPENMVG_BIN:OPENMVG_BIN = input("openMVG binary folder?\n")
if not OPENMVS_BIN:OPENMVS_BIN = input("openMVS binary folder?\n")
if not CAMERA_SENSOR_DB_DIRECTORY:CAMERA_SENSOR_DB_DIRECTORY = input("openMVG camera database (%s) folder?\n" % CAMERA_SENSOR_DB_FILE)PRESET = {'SEQUENTIAL': [0, 1, 2, 3, 4, 5, 11, 12, 13, 14, 15],'GLOBAL': [0, 1, 2, 3, 4, 6, 11, 12, 13, 14, 15],'MVG_SEQ': [0, 1, 2, 3, 4, 5, 7, 8, 9, 11],'MVG_GLOBAL': [0, 1, 2, 3, 4, 6, 7, 8, 9, 11],'MVS': [12, 13, 14, 15],'MVS_SGM': [16, 17]}PRESET_DEFAULT = 'SEQUENTIAL'# HELPERS for terminal colors
BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8)
NO_EFFECT, BOLD, UNDERLINE, BLINK, INVERSE, HIDDEN = (0, 1, 4, 5, 7, 8)# from Python cookbook, #475186
def has_colours(stream):'''Return stream colours capability'''if not hasattr(stream, "isatty"):return Falseif not stream.isatty():return False  # auto color only on TTYstry:import cursescurses.setupterm()return curses.tigetnum("colors") > 2except Exception:# guess false in case of errorreturn FalseHAS_COLOURS = has_colours(sys.stdout)def printout(text, colour=WHITE, background=BLACK, effect=NO_EFFECT):"""print() with colour"""if HAS_COLOURS:seq = "\x1b[%d;%d;%dm" % (effect, 30+colour, 40+background) + text + "\x1b[0m"sys.stdout.write(seq+'\r\n')else:sys.stdout.write(text+'\r\n')# OBJECTS to store config and data in
class ConfContainer:"""Container for all the config variables"""def __init__(self):passclass AStep:""" Represents a process step to be run """def __init__(self, info, cmd, opt):self.info = infoself.cmd = cmdself.opt = optclass StepsStore:""" List of steps with facilities to configure them """def __init__(self):self.steps_data = [["Intrinsics analysis",          # 0os.path.join(OPENMVG_BIN, "openMVG_main_SfMInit_ImageListing"),["-i", "%input_dir%", "-o", "%matches_dir%", "-d", "%camera_file_params%"]],["Compute features",             # 1os.path.join(OPENMVG_BIN, "openMVG_main_ComputeFeatures"),["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-o", "%matches_dir%", "-m", "SIFT"]],["Compute pairs",                # 2os.path.join(OPENMVG_BIN, "openMVG_main_PairGenerator"),["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-o", "%matches_dir%"+FOLDER_DELIM+"pairs.bin"]],["Compute matches",              # 3os.path.join(OPENMVG_BIN, "openMVG_main_ComputeMatches"),["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-p", "%matches_dir%"+FOLDER_DELIM+"pairs.bin", "-o", "%matches_dir%"+FOLDER_DELIM+"matches.putative.bin", "-n", "AUTO"]],["Filter matches",               # 4os.path.join(OPENMVG_BIN, "openMVG_main_GeometricFilter"),["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-m", "%matches_dir%"+FOLDER_DELIM+"matches.putative.bin", "-o", "%matches_dir%"+FOLDER_DELIM+"matches.f.bin"]],["Incremental reconstruction",   # 5os.path.join(OPENMVG_BIN, "openMVG_main_SfM"),["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-m", "%matches_dir%", "-o", "%reconstruction_dir%", "-s", "INCREMENTAL"]],["Global reconstruction",        # 6os.path.join(OPENMVG_BIN, "openMVG_main_SfM"),["-i", "%matches_dir%"+FOLDER_DELIM+"sfm_data.json", "-m", "%matches_dir%", "-o", "%reconstruction_dir%", "-s", "GLOBAL", "-M", "%matches_dir%"+FOLDER_DELIM+"matches.e.bin"]],["Colorize Structure",           # 7os.path.join(OPENMVG_BIN, "openMVG_main_ComputeSfM_DataColor"),["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin", "-o", "%reconstruction_dir%"+FOLDER_DELIM+"colorized.ply"]],["Structure from Known Poses",   # 8os.path.join(OPENMVG_BIN, "openMVG_main_ComputeStructureFromKnownPoses"),["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin", "-m", "%matches_dir%", "-f", "%matches_dir%"+FOLDER_DELIM+"matches.f.bin", "-o", "%reconstruction_dir%"+FOLDER_DELIM+"robust.bin"]],["Colorized robust triangulation",  # 9os.path.join(OPENMVG_BIN, "openMVG_main_ComputeSfM_DataColor"),["-i", "%reconstruction_dir%"+FOLDER_DELIM+"robust.bin", "-o", "%reconstruction_dir%"+FOLDER_DELIM+"robust_colorized.ply"]],["Control Points Registration",  # 10os.path.join(OPENMVG_BIN, "ui_openMVG_control_points_registration"),["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin"]],["Export to openMVS",            # 11os.path.join(OPENMVG_BIN, "openMVG_main_openMVG2openMVS"),["-i", "%reconstruction_dir%"+FOLDER_DELIM+"sfm_data.bin", "-o", "%mvs_dir%"+FOLDER_DELIM+"scene.mvs", "-d", "%mvs_dir%"+FOLDER_DELIM+"images"]],["Densify point cloud",          # 12os.path.join(OPENMVS_BIN, "DensifyPointCloud"),["scene.mvs", "--dense-config-file", "Densify.ini", "--resolution-level", "1", "--number-views", "8", "-w", "\"%mvs_dir%\""]],["Reconstruct the mesh",         # 13os.path.join(OPENMVS_BIN, "ReconstructMesh"),["scene_dense.mvs", "-w", "\"%mvs_dir%\""]],["Refine the mesh",              # 14os.path.join(OPENMVS_BIN, "RefineMesh"),["scene_dense_mesh.mvs", "--scales", "1", "--gradient-step", "25.05", "-w", "\"%mvs_dir%\""]],["Texture the mesh",             # 15os.path.join(OPENMVS_BIN, "TextureMesh"),["scene_dense_mesh_refine.mvs", "--decimate", "0.5", "-w", "\"%mvs_dir%\""]],["Estimate disparity-maps",      # 16os.path.join(OPENMVS_BIN, "DensifyPointCloud"),["scene.mvs", "--dense-config-file", "Densify.ini", "--fusion-mode", "-1", "-w", "\"%mvs_dir%\""]],["Fuse disparity-maps",          # 17os.path.join(OPENMVS_BIN, "DensifyPointCloud"),["scene.mvs", "--dense-config-file", "Densify.ini", "--fusion-mode", "-2", "-w", "\"%mvs_dir%\""]]]def __getitem__(self, indice):return AStep(*self.steps_data[indice])def length(self):return len(self.steps_data)def apply_conf(self, conf):""" replace each %var% per conf.var value in steps data """for s in self.steps_data:o2 = []for o in s[2]:co = o.replace("%input_dir%", conf.input_dir)co = co.replace("%output_dir%", conf.output_dir)co = co.replace("%matches_dir%", conf.matches_dir)co = co.replace("%reconstruction_dir%", conf.reconstruction_dir)co = co.replace("%mvs_dir%", conf.mvs_dir)co = co.replace("%camera_file_params%", conf.camera_file_params)o2.append(co)s[2] = o2def replace_opt(self, idx, str_exist, str_new):""" replace each existing str_exist with str_new per opt value in step idx data """s = self.steps_data[idx]o2 = []for o in s[2]:co = o.replace(str_exist, str_new)o2.append(co)s[2] = o2CONF = ConfContainer()
STEPS = StepsStore()# ARGS
PARSER = argparse.ArgumentParser(formatter_class=argparse.RawTextHelpFormatter,description="Photogrammetry reconstruction with these steps: \r\n" +"\r\n".join(("\t%i. %s\t %s" % (t, STEPS[t].info, STEPS[t].cmd) for t in range(STEPS.length()))))
PARSER.add_argument('input_dir',help="the directory which contains the pictures set.")
PARSER.add_argument('output_dir',help="the directory which will contain the resulting files.")
PARSER.add_argument('--steps',type=int,nargs="+",help="steps to process")
PARSER.add_argument('--preset',help="steps list preset in \r\n" +" \r\n".join([k + " = " + str(PRESET[k]) for k in PRESET]) +" \r\ndefault : " + PRESET_DEFAULT)GROUP = PARSER.add_argument_group('Passthrough', description="Option to be passed to command lines (remove - in front of option names)\r\ne.g. --1 p ULTRA to use the ULTRA preset in openMVG_main_ComputeFeatures\r\nFor example, running the script as follows,\r\nMvgMvsPipeline.py input_dir output_dir --1 p HIGH n 8 --3 n ANNL2\r\nwhere --1 refer to openMVG_main_ComputeFeatures, p refers to\r\ndescriberPreset option which HIGH was chosen, and n refers to\r\nnumThreads which 8 was used. --3 refer to second step (openMVG_main_ComputeMatches),\r\nn refers to nearest_matching_method option which ANNL2 was chosen")
for n in range(STEPS.length()):GROUP.add_argument('--'+str(n), nargs='+')PARSER.parse_args(namespace=CONF)  # store args in the ConfContainer# FOLDERSdef mkdir_ine(dirname):"""Create the folder if not presents"""if not os.path.exists(dirname):os.mkdir(dirname)# Absolute path for input and output dirs
CONF.input_dir = os.path.abspath(CONF.input_dir)
CONF.output_dir = os.path.abspath(CONF.output_dir)if not os.path.exists(CONF.input_dir):sys.exit("%s: path not found" % CONF.input_dir)CONF.reconstruction_dir = os.path.join(CONF.output_dir, "sfm")
CONF.matches_dir = os.path.join(CONF.reconstruction_dir, "matches")
CONF.mvs_dir = os.path.join(CONF.output_dir, "mvs")
CONF.camera_file_params = os.path.join(CAMERA_SENSOR_DB_DIRECTORY, CAMERA_SENSOR_DB_FILE)mkdir_ine(CONF.output_dir)
mkdir_ine(CONF.reconstruction_dir)
mkdir_ine(CONF.matches_dir)
mkdir_ine(CONF.mvs_dir)# Update directories in steps commandlines
STEPS.apply_conf(CONF)# PRESET
if CONF.steps and CONF.preset:sys.exit("Steps and preset arguments can't be set together.")
elif CONF.preset:try:CONF.steps = PRESET[CONF.preset]except KeyError:sys.exit("Unknown preset %s, choose %s" % (CONF.preset, ' or '.join([s for s in PRESET])))
elif not CONF.steps:CONF.steps = PRESET[PRESET_DEFAULT]# WALK
print("# Using input dir:  %s" % CONF.input_dir)
print("#      output dir:  %s" % CONF.output_dir)
print("# Steps:  %s" % str(CONF.steps))if 4 in CONF.steps:    # GeometricFilterif 6 in CONF.steps:  # GlobalReconstruction# Set the geometric_model of ComputeMatches to EssentialSTEPS.replace_opt(4, FOLDER_DELIM+"matches.f.bin", FOLDER_DELIM+"matches.e.bin")STEPS[4].opt.extend(["-g", "e"])if 15 in CONF.steps:    # TextureMeshif 14 not in CONF.steps:  # RefineMesh# RefineMesh step is not run, use ReconstructMesh outputSTEPS.replace_opt(15, "scene_dense_mesh_refine.mvs", "scene_dense_mesh.mvs")for cstep in CONF.steps:printout("#%i. %s" % (cstep, STEPS[cstep].info), effect=INVERSE)# Retrieve "passthrough" commandline optionsopt = getattr(CONF, str(cstep))if opt:# add - sign to short options and -- to long onesfor o in range(0, len(opt), 2):if len(opt[o]) > 1:opt[o] = '-' + opt[o]opt[o] = '-' + opt[o]else:opt = []# Remove STEPS[cstep].opt options now defined in optfor anOpt in STEPS[cstep].opt:if anOpt in opt:idx = STEPS[cstep].opt.index(anOpt)if DEBUG:print('#\tRemove ' + str(anOpt) + ' from defaults options at id ' + str(idx))del STEPS[cstep].opt[idx:idx+2]# create a commandline for the current stepcmdline = [STEPS[cstep].cmd] + STEPS[cstep].opt + optprint('Cmd: ' + ' '.join(cmdline))if not DEBUG:# Launch the current steptry:pStep = subprocess.Popen(cmdline)pStep.wait()if pStep.returncode != 0:breakexcept KeyboardInterrupt:sys.exit('\r\nProcess canceled by user, all files remains')else:print('\t'.join(cmdline))printout("# Pipeline end #", effect=INVERSE)

reference：

https://openmvg.readthedocs.io/en/latest/software/SfM/SfMInit_ImageListing/