Multimodal Dynamic Graph CNN for 3D Semantic Segmentation

Abstract

In this paper, a semantic segmentation method of point clouds in the form of terrain using a new multimodal convolutional neural network architecture based on a regular dynamic weighted graph, which allows to obtain an accurate solution to the segmentation problem based on a fusion of geometric and color features. The method can be effectively used for sparse, noisy, inhomogeneous and non convex point clouds. The computer modeling of state-of-the-art methods for 3D semantic segmentation was carried out using the reference data collection ModelNet 40 and a data set of archaeological sites of the Bronze Age of the Southern Trans-Urals, namely data obtained as a result of a total station survey (the Trimble 3300 total station) of a complex of archaeological sites in the valley of the Sintashta river. A comparative analysis of the proposed method and state-of-the-art methods for 3D semantic segmentation with different combinations of input features of point clouds was carried out, and the method influence of forming a point cloud on the accuracy of 3D semantic segmentation was also investigated: in the first case, a point cloud from a reference dataset was studied, in the second case, variants using 3D registration based on NICP and FICP algorithms were applied.