In order to solve the problems of low recognition accuracy and easily affected by illumination conditions in the gesture recognition, an improved YOLOv3 gesture recognition algorithm was proposed. Firstly, a smaller detection scale was added to the original three detection scales to improve the detection ability of small targets; secondly, DIoU was used instead of the original mean square error loss function as the coordinate error loss function, the improved focal loss function was used as the confidence loss function of the boundary frame, and the cross entropy was used as the loss function of the target classification loss function. The results show that when the improved YOLOv3 gesture recognition algorithm is applied to gesture detection, the map index reaches 90.38%, which is 6.62% higher than that before the improvement, and FPS is nearly twice as high as before. After the new model is trained by the improved YOLOv3 method, the gesture recognition accuracy is higher, the detection speed is faster, the overall recognition efficiency is greatly improved, the loss weights of simple samples and difficult samples are balanced, and the training quality and generalization ability of the model are effectively improved.