Clever Geek Handbook

3D ML。第3部分：3D ML中的数据集和框架



在机器学习框架内的特定主题领域中工作（在我们的例子中为3D），有必要了解主要的数据集是什么，在这些数据集的基础上进行模型的训练和测试，以及考虑到数据的细节而存在哪些库和程序用于舒适的工作。



3D ML 3D .



3D ML :



  1. 3D
  2. 3D ML
  3. 3D ML


GitHub .



IT- “VR/AR & AI” — PHYGITALISM.



Datasets



, , . , . 3D ML .1. github ( geometricdeeplearning.com Tutorials).





.1 3D ML.



, ( , ), . -, : , ( ShapeNet). -, , , , .





.2 3D Geometric Deep Learning SGP 2018.



, , , , , .



3D ML, , , , .



, GDL.



1. ShapeNet (2015) [1]





3 (.obj ), 4 . . : , , . (SHREC, ICCV) 3D ML.



:



  • ShapeNetCore: 51300 55 .
  • ShapeNetSem: 12000 270 .


2D-to-3D , 3D . , , , .



2. ModelNet (2015) [2]





127915 3D 662 ( .off).



:



  • ModelNet10: 4899 10 .
  • ModelNet40: 12311 40 , .


, “ModelNet Benchmark”. ShapeNet , , .



3. Pix3D (2018) [3]





395 3D .obj .mat (.mat Matlab , scipy.io). 9 , . , , 3D .



, 2D-to-3D. .



4. ABC dataset (2019) [4]





CAD . , , , , .



, 3D ML, . , 3D .



, , GDL ADASE . 3D ML ADASE , .



.3 3D [4].



5. VOCASET: Speech-4D Head Scan Dataset (2019) [5]





VOCASET — 4D face dataset 29- , 60 . 12 480 3-4 , , .



6. ScanNet (2017) [6]





RGB-D , 2,5 1500 , 3D-, .



, (indoor scene), , .



7. Semantic3D (2017) [7]





, 3D , (outdoor scene) 4 , .



KITTY, Semantic3D , .



8. Campus3D (2020) [8]





. 3D ( 900 ).





.4 [8].



Frameworks





.5 , Kaolin [9].



, , , , , (.. train/test pipelines). , . .



, , , 3D . Blender, PCL (Point cloud library) MeshLab. , Python, ( Tensorflow Pytorch) .



Unity 3D Unity ML agents , ( RL ). ML agents RL : , , , .



, .





, 3D ML (. Kaolin .5):







.6 . PyTorch3D.



, [9] NVIDIA Kaolin, [10] PyTorch3D, TensorFlow Graphics [11].



, , : , polyscope, , , mesh_to_sdf.





1. PyTorch Geometric (Fey & Lenssen: Department of Computer Graphics TU Dortmund University | 2019) [12]





— PyTorch, , . , . .



, PyTorch Geometric 3D ML, . — . . PyTorch3D.



: Linux; Windows; Mac (CPU only).



.



2. TensorFlow Graphics (Google Brain | 2019) [11]





PyTorch Geometric, TensorFlow Graphics .



, TensorBoard, 3D . ( TensorFlow PyTorch), TensorBoard 3D.



, TensorFlow Graphics . .



: Linux; Windows; Mac.



.





.6 TensorBoard 3D. .



3. NVidia Kaolin (NVidia | 2019) [9]





3D ML, ( ). — , .



, , .



: Linux; Windows (unstable).



.





.7 Kaolin [9].



4. PyTorch 3D (Facebook research | 2019) [10]





Linux , , (. PyTorch Geometric + trimesh + polyscope) 3D ML.



CPU GPU. , 3D ML: chamfer loss, , , , 3D PyTorch .



. .



: Linux; Mac; Windows (unstable).



.



5. Points 3D (Chaton & Chaulet: Principia Labs | 2020)





. SOTA , .

CPU GPU. , .



Medium.



: Linux; Mac; Windows.



.





.8 Jupyter Lab , Points 3D .





1. Kornia (2019) [50]





3D ML , , geometrical deep learning Kornia PyTorch.





.9 [13].



, , , Kornia , 3D ML , RGB-D .



.



2. Polyscope





. trimesh ( ). , . .





.10 Polyscope.



3. trimesh





:



  • ;
  • (ICP .);
  • , .;
  • .


.



.



4. mesh_to_sdf





.11 SDF mesh_to_sdf .



SDF . non-watertight meshes: , , , .. non-manifold .



.



DeepSDF [14].



5. Open3D [15]





trimesh. Python C++. .



[15], .





— deep learning machine learning. 3D ML , , , .



, , . PyTroch3D + , trimesh, PCL Open3D ( C++ ), Python API Blender, .



"SGP 2020 Graduate School: Black Box Geometric Computing with Python", Python, , 3D .



[1] Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012, 2015 [project page]



[2] Z. Wu, S. Song, A. Khosla, F. Yu, L. Zhang, X. Tang and J. Xiao 3D ShapeNets: A Deep Representation for Volumetric Shapes Proceedings of 28th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015) [project page]



[3] X. Sun, J. Wu, X. Zhang, Z. Zhang, C. Zhang, T. Xue, J. B. Tenenbaum, and W. T. Freeman. Pix3d: Dataset and methods for single-image 3d shape modeling. In CVPR, 2018 [project page]



[4] Koch, S., Matveev, A., Jiang, Z., Williams, F., Artemov, A., Burnaev, E., Alexa, M., Zorin, D. and Panozzo, D., 2019. ABC: A Big CAD Model Dataset For Geometric Deep Learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 9601–9611). [project page]



[5] Cudeiro, D., Bolkart, T., Laidlaw, C., Ranjan, A. and Black, M.J., 2019. Capture, learning, and synthesis of 3d speaking styles. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 10101-10111). [project page]



[6] Dai, A., Chang, A.X., Savva, M., Halber, M., Funkhouser, T. and Nießner, M., 2017. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5828-5839). [project page]



[7] Hackel, T., Savinov, N., Ladicky, L., Wegner, J.D., Schindler, K. and Pollefeys, M., 2017. Semantic3d. net: A new large-scale point cloud classification benchmark. arXiv preprint arXiv:1704.03847. [project page]



[8] Li, X., Li, C., Tong, Z., Lim, A., Yuan, J., Wu, Y., Tang, J. and Huang, R., 2020. Campus3D: A Photogrammetry Point Cloud Benchmark for Hierarchical Understanding of Outdoor Scene. arXiv preprint arXiv:2008.04968. [project page]



[9] Jatavallabhula, Krishna Murthy, Edward Smith, Jean-Francois Lafleche, Clement Fuji Tsang, Artem Rozantsev, Wenzheng Chen, Tommy Xiang, Rev Lebaredian and Sanja Fidler. “Kaolin: A PyTorch Library for Accelerating 3D Deep Learning Research.” ArXivabs/1911.05063 (2019): n. pag. [project page]



[10] Ravi, N., Reizenstein, J., Novotny, D., Gordon, T., Lo, W.Y., Johnson, J. and Gkioxari, G., 2020. Accelerating 3D Deep Learning with PyTorch3D. arXiv preprint arXiv:2007.08501. [github page]



[11] Valentin, J., Keskin, C., Pidlypenskyi, P., Makadia, A., Sud, A. and Bouaziz, S., 2019. Tensorflow graphics: Computer graphics meets deep learning. [project page]



[12] Fey, M. and Lenssen, J.E., 2019. Fast graph representation learning with PyTorch Geometric. arXiv preprint arXiv:1903.02428. [project page]



[13] Riba, E., Mishkin, D., Ponsa, D., Rublee, E. and Bradski, G., 2019. Kornia: an open source differentiable computer vision library for pytorch. arXiv preprint arXiv:1910.02190. [project page]



[14] Park, J.J., Florence, P., Straub, J., Newcombe, R. and Lovegrove, S., 2019. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 165-174). [github page]



[15] Zhou, Q.Y., Park, J. and Koltun, V., 2018. Open3D: A modern library for 3D data processing. arXiv preprint arXiv:1801.09847. [project page]



More articles:


All Articles