Review of key technologies for warehouse 3D reconstruction

Authors

  • Y.N. Hao Centre for Modelling and Simulation, Faculty of Engineering, Built Environment and Information Technology, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia. Phone: +60361451777; Fax.:+60361451666
  • Y.C. Tan Centre for Modelling and Simulation, Faculty of Engineering, Built Environment and Information Technology, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia. Phone: +60361451777; Fax.:+60361451666
  • V.C. Tai Centre for Modelling and Simulation, Faculty of Engineering, Built Environment and Information Technology, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia. Phone: +60361451777; Fax.:+60361451666
  • X.D. Zhang Centre for Modelling and Simulation, Faculty of Engineering, Built Environment and Information Technology, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia. Phone: +60361451777; Fax.:+60361451666
  • E.P. Wei Centre for Modelling and Simulation, Faculty of Engineering, Built Environment and Information Technology, SEGI University, 47810 Petaling Jaya, Selangor, Malaysia. Phone: +60361451777; Fax.:+60361451666
  • S.C. Ng Faculty of Arts and Science, International University of Malaya-Wales, 50480 Kuala Lumpur, Malaysia.

DOI:

https://doi.org/10.15282/jmes.16.3.2022.15.0724

Keywords:

3D reconstruction, Warehouse management system, Binocular, Stereo vision, Fusion

Abstract

Most of the current warehouse management system is made out of two-dimensional (2D) plane schematic warehouse, which brings a lot of inconvenience to warehouse management, including the warehouse data, storage of goods, location search, inventory, etc. 3D warehouse model began to attract attention as it can provide more intuitive view of warehouse-related information. This paper aims to review and investigate the current key technologies used for 3D modeling of warehouse system. This paper reviewed the method of 3D view reconstruction of the warehouse management system, including the active and passive, active and passive fusion methods, and makes a detailed comparison between the active and passive methods. It was found that different methods were applied to reconstruct the 3D view of warehouse, each with its own advantages and disadvantages.

References

W. Ding, “Study of smart warehouse management system based on the IOT,” Advances in Intelligent Systems and Computing, vol. 180, pp. 203-207, 2013.

J. O. Strandhagen, L. R. Vallandingham, G. Fragapane, and J. W. Strandhagen, “Logistics 4.0 and emerging sustainable business models,” Advances in Manufacturing, vol. 5, no. 4, pp. 359-369, 2017.

L. Barreto, A. Amaral, and T. Pereira, “Industry 4.0 implications in logistics: an overview,” Procedia Manufacturing, vol. 13, pp. 1245-1252, 2017.

L. Sommer, “Industrial Revolution-Industry 4.0: Are German manufacturing SMEs the first victims of this revolution,” Journal of Industrial Engineering and Management, vol. 8, no. 5, pp. 1512-1532, 2015.

L. D. Gamindo, “The challenges of Logistics 4.0 for the supply chain management and the information technology,” Master thesis, Norwegian University of Science and Technology, Trondheim, Norway, 2016.

W. Zhou, S. Piramuthu, F. Chu, C. Chu, “RFID-enabled flexible warehousing,” Decision Support Systems, vol 98, pp. 99- 112, 2017.

H. Karadag, “The role of SMES and entrepreneurship on economic growth in emerging economies within the post-crisis era: an analysis from Turkey,” Journal of Small Business and Entrepreneurship Development, vol. 4, no. 1, pp. 22-31, 2016.

A. Issa, D. Lucke, and T. Bauernhansl, “Mobilizing SMEs towards Industrie 4.0-enabled smart products,” Procedia CIRP, Vol. 63, pp. 670 -674, 2017.

Z. Taha, V. C. Tai, and P. C. See, “Design and manufacture of a miniature UAV using 3D rapid prototyping”, Advanced Material Research, vol. 308-310, pp. 1425-1435, 2011.

D. Sanders, B. Sanders, D. Ndzi, and N. Bausch, “Using confidence factors to share control between a mobile robot tele-operater and ultrasonic sensors,” IEEE Proceedings of SAI Intelligent Systems Conference (IntelliSys) UK. ISBN 978-1-5090-6435-9, pp: 1026-1033, 2017.

C. Silva, L.M. Ferreira, M. Thurer, and M. Stevenson, “Improving the logistics of a constant order-cycle kanban system,” Production Planning and Control, vol. 27, pp. 650-659,2016.

T. L. Landers, M. H. Cole, B. Walker, and R.W. Kirk, “The virtual warehousing concept,” Transportation Research Part E, vol. 36, pp. 115-126, 2000.

S. H. Fung, C. F. Cheung, W. B. Lee, and S. K. Kwok, “A virtual warehouse system for production logistics,” Production Planning and Control, vol. 16, no. 6, pp. 597- 607, 2005.

W. J. Chen, “Design of a 3-dimensional visual simulation monitoring system for warehouse logistics management,” Network and information. pp. 421-422+453, 2014,

W. C. Tan, and K. M. Yap, “Exploratory research on application of different vision system on warehouse robot using selective algorithm,” In: Advances in Visual Informatics, IVIC2017. Lecture Notes in Computer Science, vol. 10645, pp. 290-296, 2017.

A. Causo, Z. -H. Chong, R. Luxman and I. -M. Chen, "Visual marker-guided mobile robot solution for automated item picking in a warehouse," In: 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 201-206, 2017.

Z. -H. Chong, R. Luxman, W. -C. Pang, Z. Yi, R. Meixuan, H. S. Tju, A. Causo, and I. -M. Chen, "An innovative robotics stowing strategy for inventory replenishment in automated storage and retrieval system,” In: 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV), pp. 305-310, 2018.

S. C. Ng, V. C. Tai, Y. C. Tan, and N. F. A. Rahman, “SFlex-WMS: a novel multi-expert system for flexible logistics and warehouse operation in the context of Industry 4.0,” SHS Web Conf., vol. 124, paper no. 10002, 2021.

Y. Wu, and D. Ge, “Key technologies of warehousing robot for intelligent logistics” Advances in Social Science, Education and Humanities Research, vol. 309, pp. 79 – 82, 2019.

C. K. M. Lee, Y. Lv, K. K. H. Ng, W. Ho, and K. L. Choy, “Design and application of internet of things-based warehouse management system for smart logistics,” International Journal of Production Research, vol. 56, no. 8, pp. 2753-2768, 2018.

Y. Ding, M. Jin, S. Li, and D. Feng, “Smart logistics based on the internet of things technology: an overview,” International Journal of Logistics Research and Applications, vol. 24, no. 4, pp. 323-345, 2021.

S. Winkelhaus, and E.H. Grosse, “Logistics 4.0: a systematic review towards a new logistics system,” International Journal of Production Research, vol.58, no. 1, pp. 18-43, 2020.

J. C. Chen, C. -H. Cheng, P. B. Huang, K. -J. Wang, C. -J. Huang, and T. -C. Ting, “Warehouse management with lean and RFID application: a case study,” International Journal of Advanced Manufacturing Technology, vol. 69, pp. 531-542, 2013.

A. R. Laxmi, and A. Mishra, "RFID based logistic management system using internet of things (IoT)," In: 2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA), pp. 556-559, 2018.

J. Leng, D. Yan, Q. Liu, H. Zhang, G. Zhao, L. Wei, D. Zhang, A. Yu, and X. Chen, “Digital twin-driven joint optimisation of packing and storage assignment in large-scale automated high-rise warehouse product-service system,” International Journal of Computer Integrated Manufacturing, vol. 34, no. 7-8, pp. 783-800, 2021.

X. Li, I. Y. H. Chen, S. Thomas, and B. A. MacDonald, “Using kinect for monitoring warehouse order picking operations” In: Proceedings of Australasian Conference on Robotics and Automation, 3-5 December 2012, Victoria University of Wellington, New Zealand, 2012.

Y. Zou, W. Chen, X. Wu and Z. Liu, “Indoor localization and 3D scene reconstruction for mobile robots using the Microsoft Kinect sensor,” In: IEEE 10th International Conference on Industrial Informatics, pp. 1182-1187, 2012.

X. H. Zhou, Z. M. Yi, K. H. Huang, and H. Huang, “Survey on path and view planning for UAVs”, Virtual Reality & Intelligent Hardware, vol. 2, no. 1, pp. 56-59, 2020.

R. Mahmudur. S. M. Abdul Ahad, S. Siddique, and T. Motahar, “Smart warehouse management system with RFID and cloud database,” In: 2019 Joint 8th International Conference on Informatics, Electronics & Vision (ICIEV) and 2019 3rd International Conference on Imaging, Vision & Pattern Recognition (icIVPR), pp. 218-222, 2019.

C. K. M. Lee, Y. Lv, K. H. Ng, and W. Ho, “Design and application of internet of things-based warehouse management system for smart logistics,” International Journal of Production Research, pp. 1-16, 2017.

H. K. H. Chow, K. L. Choy, W. B. Lee, and K. C. Lau, “Design of a RFID case-based resource management system for warehouse operations,” Expert Systems with Application, vol. 30, no. 4, pp. 561-576, 2006.

K. R. Lee, “Accurate, efficient, and robust 3D reconstruction of static and dynamic objects.” Doctor Thesis, University of California, American, 2014.

F. Isgro, F. Odone, and A. Verri, “An open system for 3D data acquisition from multiple sensor,” Seventh International Workshop on Computer Architecture for Machine Perception (CAMP'05), pp. 52-57, 2005.

L. X. Huan, X. W. Zheng, and J. Y. Gong, “GeoRec: Geometry-enhanced semantic 3D reconstruction of RGB-D indoor scenes,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 186, pp. 301-314, 2022.

C. L. Wen, J. B. Tan, F. S. Li, C. R. Wu, Y. T. Lin, Z. Y. Wang, and C. Wang, “Cooperative indoor 3D mapping and modeling using LiDAR data,” Information Sciences, vol. 574, pp. 192-209, 2021.

L. Yang, Y. H. Sheng, and B. Wang, “3D reconstruction of building facade with fused data of terrestrial LiDAR data and optical image,” Optik, vol. 127, no. 4, pp. 2165-2168. 2016.

D. Li, L. Xu, X. S. Tang, S. Sun, X. Cai, and P. Zhang, “3D imaging of greenhouse plants with an inexpensive binocular stereo vision system,” Remote Sensing, vol. 9, no. 5, 2017.

Y. S. Yin, and J. Antonio, “Application of 3D laser scanning technology for image data processing in the protection of ancient building sites through deep learning, ” Image and Vision Computing, vol. 102, 2020.

Y. Pan, Y. Han, L. Wang, J. Chen, H. Hao, G. Q. Wang, Z. C. Zhang, and S. B. Wang. “3D Reconstruction of Ground Crops Based on Airborne LiDAR Technology,” IFAC-PapersOnLine, vol: 52, pp, 35-40, 2019.

S. H. Venugopala. “Comparative study of 3D object detection frameworks based on LiDAR data and sensor fusion techniques,” Journal of Physics: Conference Series, 2022.

S. Vandenberghe, and J. Karp, “Rebinning and reconstruction techniques for 3D TOF-PET,” Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, vol. 569, no. 2, pp. 421-424.

T. X. Zheng, S. Huang, Y. F. Li, and M. C. Feng, “Key techniques for vision based 3D reconstruction-A review,” ACTA AUTOMATICA SINICA, vol. 46, no. 4, pp. 631-652, 2020.

S. Á. Guðmundsson, M. Pardàs, J. R. Casas, J. R. Sveinsson, H. Aanæs, and R. Larsen, “Improved 3D reconstruction in smart-room environments using ToF imaging,” Understanding, vol. 114, no. 12, pp. 1376-1384, 2010.

G. Alenyà, S. Foix, and C. Torras, “ToF cameras for active vision in robotics,” Sensors and Actuators A: Physical , vol. 218, pp. 10-22, 2014.

S. Á. Guðmundsson, M. Pardàs, J. R. Casas, J. R. Sveinsson, H. Aanæs, and R. Larsen, “Improved 3D reconstruction in smart-room environments using ToF imaging,” Computer Vision and Image Understanding, vol. 114, no. 12, pp. 1376-1384,2010.

J. Geng, “Structured-light 3D surface imaging: a tutorial,” pp. 131, 2008.

M. Y. Li, J. Liu, H. M. Yang, W. Q. Song, and Z. H. Yu, “Structured light 3D reconstruction system based on a stereo calibration plate,” Symmetry, vol. 12, no. 5, pp. 772, 2020.

Z. Z. Kang, J. T. Yang, Z. Yang, and S. Cheng, “A review of techniques for 3D reconstruction of indoor environments,” ISPRS International Journal of Geo-Information, vol. 9, no. 5, pp.330, 2020.

D. D. Lv, and G. E. Jiao “Experiment of stereo matching algorithm based on binocular vision,” Journal of Physics: Conference Series, vol. 1574, 2020.

S. F. Zhuang, X. Zhang, D. W. Tu, C. Zhang, and L. L. Xie, “A standard expression of underwater binocular vision for stereo matching,” Measurement Science and Technology, vol. 31, no. 11, pp. 5012, 2020.

N. Guo, L. Li, F. Yan, and T. Li, “Binocular stereo vision calibration based on constrained sparse beam adjustment algorithm,” Optik, vol. 208, pp. 1-11. 2020..

B. Q. Xu , and C. Liu “A 3D reconstruction method for buildings based on monocular vision,” Computer-Aided Civil and Infrastructure Engineering, 2021.

H. Y. Yang, “Binocular stereo vision based three dimensional reconstruction using domain-sized pooling local features,” 2nd International Conference on Artificial Intelligence: Techniques and Applications, pp. 364-368, doi: 978-1-60595-491-2, 2017.

G. Huang, “Binocular vision system for real-time badminton tracking,” Journal of electronic measurement and instrumentation, vol. 35, pp. 117-123, 2021.

L. E. Ortiz, E. V. Cabrera, and L. M. Goncalves, “Depth data error modeling of the ZED 3D vision sensor from stereolabs,” Electronic Letters on Computer Vision and Image Analysis, vol.17, no. 1, pp. 1-15, 2018.

Huo G, Wu Z, Li J, and Li S, “Underwater Target Detection and 3D Reconstruction System Based on Binocular Vision,” Sensors, vol, 18, no. 10, 2018.

Z. H. Zhang, W. Liu, G. D. Liu, L. M. Song, Y. F. Qu, X. D. Li, and Z. Z. Wei “Development of three-dimensional vision measurement technology and its application,” Journal of Image and Graphics, vol. 26, no. 6, pp. 1483-1502, 2021.

R. Hartly, and A. Zisserman, “Multiple view geometry in computer vision, ”2nd Edition. Cambridge University, England, 2004.

Y. N. Ma, Q. O. Li, J. Xing, G. Y. Huo, and Y. Liu, “An intelligent object detection and measurement system based on trinocular vision,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 30, no. 3, pp. 711-724, 2020.

X. Y. Wang, T. P. Lin, X. S. Jiang, K. Xiang, and F. Pan, “Reliable fusion of ToF and stereo data based on joint depth filter,” Journal of Visual Communication and Image Representation, vol. 74, 2021.

Q. Yang, K. H. Tan, B. Culbertson, and J. Apostolopoulos, “Fusion of active and passive sensors for fast 3D capture,” IEEE International Workshop on Multimedia Signal Processing, 2010.

L. Yang, Y. H. Sheng, and B. Wang, 3D reconstruction of building facade with fused data of terrestrial LiDAR data and optical image, Optik, vol. 127, no. 4, pp. 2165-2168, 2016.

P. Gurram, H. Rhody, J. Kerekes, S. Lach, and E. Saber, “3D scene reconstruction through a fusion of passive video and lidar imagery,” AIPR. 36th IEEE. pp. 133-138, 2007.

K. D. Kuhnert, and M. Stommel, “Fusion of stereo-camera and pmd-camera data for realtime suited precise 3D environment reconstruction,” Intelligent Robots and Systems. IEEE/RSJ International Conference, pp. 4780-4785, 2006.

K. M. Nickels, A. Castaño, and C. Cianci, “Fusion of Lidar and stereo range for mobile robots,” Proceedings of ICAR 2003, pp. 65-70, 2003.

Y. Deng, J. M. Xiao, and Z. Y. Zhou, “ToF and stereo data fusion using dynamic search range stereo matching,” IEEE Transactions on Multimedia, vol. 24, pp. 2739-2751, 2021.

M. Poggi, G. Agresti, F. Tosi, P. Zanuttigh and S. Mattoccia, “Confidence Estimation for ToF and Stereo Sensors and Its Application to Depth Data Fusion,” IEEE Sensors Journal, vol. 20, no. 3, pp. 1411-1421, 2020.

H. M. Huang, G. H. Liu, K. R. Duan, and J. Y. Yuan, “A convenient 3D reconstruction model based on parallel-axis structured light system,” Journal of Laser Applications, vol. 33, no. 4, 82021.

Y. B. He, T. L. Yang, G. F. Lu, and J. G. Liao, “Study of Target Pose Measurement Based on Binocular Vision of Linear Structure Light,” Journal of Mechanical Engineering, no. 5, pp. 1-3, 2017.

F. Bruno, G. Bianco, M. Muzzupappa, S. Barone, and A. V. Razionale, “Experimentation of structured light and stereo vision for underwater 3D reconstruction,” ISPRS Journal of Photogrammetry and Remote Sensing, no .66, pp. 508-518, 2011.

C. L. Wen, J. B. Tan, F. S. Li, C. R. Wu, Y. T. Lin, Z. Y. Wang, and C. Wang, “Cooperative indoor 3D mapping and modeling using LiDAR data,” Information Sciences, vol. 574, pp. 192-209, 2021.

Y. Deng, J. Xiao and S. Z. Zhou, “ToF and stereo data fusion using dynamic search range stereo matching,” IEEE Transactions on Multimedia, vol. 24, pp. 2739-2751, 2022.

F. Li, Q. L. Li, T. J. Zhang, Y. Niu, and G. M. Shi, “Depth acquisition with the combination of structured light and deep learning stereo matching,” Signal Processing: Image Communication, vol. 75, pp. 111-117, 2019.

X. X. Long, and X. J. Cheng, “三维视觉前沿进展 [Frontier Advances in 3D Vision],” Journal of image and Graphics, vol. 26, no. 6, pp.1389-1412, 2021.

Z. Kang, J. Yang, Z. Yang, and S. Cheng, “A review of techniques for 3D reconstruction of indoor environments,” ISPRS International Journal of Geo-Information, vol. 9, no. 5, pp. 339, 2020.

M. Vlaminck, H. Luong, W. Goeman, W. Philips, “3D scene reconstruction using omnidirectional vision and LiDAR: A hybrid approach,” Sensors, vol. 16, no. 11. pp. 1923, 2016.

G. Bianco, A. Gallo, F. Bruno, and M. Muzzupappa, “A comparative analysis between active and passive techniques for underwater 3D reconstruction of close-range objects,” Sensors. vol. 13. No. 8, 2013.

D. Ponsa, A. López, and F. Lumbreras, “3D vehicle sensor based on monocular vision,” In: Proceedings of the IEEE Intelligent Transportation Systems, NJ: IEEE Press, pp. 1096-1101, 2005.

Downloads

Published

2022-09-28

How to Cite

[1]
YaNan Hao, Y.C. Tan, V.C. Tai, X.D. Zhang, E.P. Wei, and S.C. Ng, “Review of key technologies for warehouse 3D reconstruction”, J. Mech. Eng. Sci., vol. 16, no. 3, pp. 9142–9156, Sep. 2022.

Issue

Vol. 16 No. 3 (2022): September 2022

Section

Review

License

Copyright (c) 2022 Universiti Malaysia Pahang Publishing

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.