Aiming at the problem of the occlusion of the tracking target and the change of the target scale by the nuclear-related tracking filter algorithm in complex scenes, a correlation filter target tracking algorithm that combines depth features and scale adaptation is proposed. First, the deep residual network (ResNet) is used to extract the depth features of the tracked area in the image, and then the target area directional gradient histogram feature (FHOG) is extracted, and multiple response maps are obtained through the kernel correlation filter learning. Perform weighted fusion to obtain the tracking target position. Secondly, the scale filter is trained through the directional gradient histogram (FHOG) feature to realize the estimation of the target scale, so that the algorithm has a good adaptive ability to the change of the target scale. Finally, according to the peak fluctuation of the response graph, the model update strategy is improved and the re-detection mechanism is introduced to reduce the probability of model drift and improve the algorithm's ability to resist occlusion. Compare with other 6 target tracking algorithms in the standard data set OTB100. The experimental results show that the accuracy of the original KCF algorithm is improved by 29.3%, and the success rate is improved by 25.3%. The algorithm in this paper can have a higher tracking success rate and accuracy in the environment of illumination interference and occlusion.