Hand Signal Recognition of Workers on Construction Sites Using Deep Learning Networks

Hand signals, as one of the common ways to communicate, are widely used on construction sites due to their simple but effective nature. However, they may not always be captured timely or interpreted correctly in construction fields, which easily leads to worker injuries/fatalities, work interruption, and stoppage, etc. This paper investigated whether construction hand signals could be captured and interpreted automatically with deep learning networks. A new data set containing 11 classes of hand signals for instructing tower crane operations is created under different scenes. The created data set is employed to compare two state-of-the-art 3D convolutional neural networks (CNNs), namely ResNeXt-101 and Res3D+ConvLSTM+MobileNet, and measure their hand signal recognition performance. The comparison results indicate that a high classification accuracy (99.0%) and a short inference time (0.21 s/gesture) could be achieved, which illustrates the feasibility of using deep learning networks to achieve hand signal recognition in construction.