中国荷斯坦牛建模相关技术的研究(Open3D、D435)

文章字数:1751
  • 相关环境
  1. Open3D(https://github.com/IntelVCL/Open3D/)
  2. D435(https://github.com/IntelRealSense/librealsense)
  • Open3D环境的安装

备注:官方版本有更新,遇到问题以官方说明为准

1.Python环境安装

Open3D安装成功导出一个Python库以供调用,网上有Open3D教程基于Python2.7,此处使用的Python版本为3.7,经测试可用。

Python安装可前往官网下载:https://www.python.org/ ,安装时可勾选添加环境变量以免仍需自己配置。

2.Cmake环境安装

Cmake允许开发者编写一种平台无关的 CMakeList.txt 文件来定制整个编译流程,然后再根据目标用户的平台进一步生成所需的本地化 Makefile 和工程文件

在官网下载最新版本:https://cmake.org ,安装时勾选添加环境变量。​

3.Visual Studio环境安装

Visual Studio可前往官网下载:Visual Studio: IDE and Code Editor for Software Developers and Teams

Open3D官方推荐版本为2015,此处使用VS2017编译成功

4.Open3D编译

前往github下载最新的代码:https://github.com/IntelVCL/Open3D

参考官方文档:http://www.open3d.org/docs/getting_started.html

使用Cmake最终可在Open3D文件夹下过的build文件夹,后续研究大部分基于此文件夹。

配置成功在python中输入import open3d应该可以正常运行

  • D435环境的安装

1.MeshLab安装

MeshLab用于打开相机导出的数据。

在官网下载最新版:MeshLab

按照提示安装即可。

2.D435安装SDK

在github上下载官方SDK:

https://github.com/IntelRealSense/librealsense/releases

其中主要是:Intel.RealSense.Viewer.exe

插入D435相机,运行Intel.RealSense.Viewer.exe,可在右方显示相机图像。

打开RGB通道与Depth通道,切换为3D模式,可导出ply文件

  • 运行代码

为了方便调试代码,推荐把需要调试的代码与数据复制到其他文件夹操作。

合成代码:https://paste.ubuntu.com/p/gq2SwfjVss/

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
import numpy as np
import open3d
from open3d import registration_ransac_based_on_feature_matching as RANSAC
from open3d import registration_icp as ICP
from open3d import compute_fpfh_feature as FPFH
from open3d import get_information_matrix_from_point_clouds as GET_GTG


def register(pcd1, pcd2, size):

    kdt_n = open3d.KDTreeSearchParamHybrid(radius=size, max_nn=50)
    kdt_f = open3d.KDTreeSearchParamHybrid(radius=size * 10, max_nn=50)

    pcd1_d = open3d.voxel_down_sample(pcd1, size)
    pcd2_d = open3d.voxel_down_sample(pcd2, size)
    open3d.estimate_normals(pcd1_d, kdt_n)
    open3d.estimate_normals(pcd2_d, kdt_n)

    pcd1_f = FPFH(pcd1_d, kdt_f)
    pcd2_f = FPFH(pcd2_d, kdt_f)

    checker = [open3d.CorrespondenceCheckerBasedOnEdgeLength(0.9),
               open3d.CorrespondenceCheckerBasedOnDistance(size * 2)]

    est_ptp = open3d.TransformationEstimationPointToPoint()
    est_ptpln = open3d.TransformationEstimationPointToPlane()

    criteria = open3d.RANSACConvergenceCriteria(max_iteration=400000,
                                              max_validation=500)

    result1 = RANSAC(pcd1_d, pcd2_d,
                     pcd1_f, pcd2_f,
                     max_correspondence_distance=size * 2,
                     estimation_method=est_ptp,
                     ransac_n=4,
                     checkers=checker,
                     criteria=criteria)

    result2 = ICP(pcd1, pcd2, size, result1.transformation, est_ptpln)

    return result2.transformation

def merge(pcds):

    all_points = []
    for pcd in pcds:
        all_points.append(np.asarray(pcd.points))

    merged_pcd = open3d.PointCloud()
    merged_pcd.points = open3d.Vector3dVector(np.vstack(all_points))

    return merged_pcd


def add_color_normal(pcd): # in-place coloring and adding normal
    pcd.paint_uniform_color(np.random.rand(3))
    size = np.abs((pcd.get_max_bound() - pcd.get_min_bound())).max() / 30
    kdt_n = open3d.KDTreeSearchParamHybrid(radius=size, max_nn=50)
    open3d.estimate_normals(pcd, kdt_n)


def load_pcds(pcd_files):

    pcds = []
    for f in pcd_files:
        pcd = open3d.read_point_cloud(f)
        add_color_normal(pcd)
        pcds.append(pcd)


    return pcds


def align_pcds(pcds, size):

    pose_graph = open3d.PoseGraph()
    accum_pose = np.identity(4)
    pose_graph.nodes.append(open3d.PoseGraphNode(accum_pose))

    n_pcds = len(pcds)
    for source_id in range(n_pcds):
        for target_id in range(source_id + 1, n_pcds):
            source = pcds[source_id]
            target = pcds[target_id]

            trans = register(source, target, size)
            GTG_mat = GET_GTG(source, target, size, trans)

            if target_id == source_id + 1:
                accum_pose = np.matmul(trans, accum_pose)
                pose_graph.nodes.append(open3d.PoseGraphNode(np.linalg.inv(accum_pose)))

            pose_graph.edges.append(open3d.PoseGraphEdge(source_id,
                                                       target_id,
                                                       trans,
                                                       GTG_mat,
                                                       uncertain=True))



    solver = open3d.GlobalOptimizationLevenbergMarquardt()
    criteria = open3d.GlobalOptimizationConvergenceCriteria()
    option = open3d.GlobalOptimizationOption(
             max_correspondence_distance=size / 10,
             edge_prune_threshold=size / 10,
             reference_node=0)


    open3d.global_optimization(pose_graph,
                            method=solver,
                            criteria=criteria,
                            option=option)


    for pcd_id in range(n_pcds):
        trans = pose_graph.nodes[pcd_id].pose
        pcds[pcd_id].transform(trans)


    return pcds


def main():
    pcds = load_pcds(["data/test/bun270.ply",
				  "data/test/bun315.ply",
				  "data/test/chin.ply",
				  "data/test/bun000.ply",
				  "data/test/bun045.ply",
				  "data/test/bun090.ply",
				  "data/test/bun180.ply"])

    open3d.draw_geometries(pcds, "input pcds")

    size = np.abs((pcds[0].get_max_bound() - pcds[0].get_min_bound())).max() / 30

    pcd_aligned = align_pcds(pcds, size)
    open3d.draw_geometries(pcd_aligned, "aligned")

    pcd_merge = merge(pcd_aligned)
    add_color_normal(pcd_merge)
    open3d.draw_geometries([pcd_merge], "merged")

if __name__ == '__main__':
    main()

将测试数据与代码放入对应文件夹内

  1. 打开命令提示符窗口
  2. cd 代码文件夹(如果不在C盘还需要再键入一行盘符,如D:)
  3. python main.py(main.py为代码文件夹名称)

环境配置无误的话将依次获得处理结果

  • 处理数据

1.通过D435导出ply文件

2.导入MeshLab中,通过上方工具去除多余点

常用操作:

鼠标:旋转模型

Ctrl+鼠标:整体位置拖动 选取某一区域的点:

删除所选的点:ctrl+delete

处理完成后在左上角选择保存。

  1. 把数据放入指定文件夹中
  2. 修改代码读取的文件名,即给pcds变量复制的load_pcds函数
  3. 运行代码查看效果
  • 附录

测试数据:

https://download.csdn.net/download/u011493189/10713389

3DCloud基于照片的3D模型构建:

官方网站:http://www.3dcloud.cn

模型展示:

http://www.3dcloud.cn/Member/?m=Models&a=model_view&model_id=10078

所用照片:

https://download.csdn.net/download/u011493189/10713405

备注:

网站功能尚不完善,免费下载额度有限,高精度建模尚未开放,API尚未开放。

正在慢慢完善之中,可观察其后续进展。

该内容采用 CC BY-NC-SA 4.0 许可协议。

如果对您有帮助或存在意见建议,欢迎在下方评论交流。

加载中...