Aiming to address the issue of increased production costs caused by the five-pass cold drawing forming for the sliding block in the rolling linear guide pair in a factory in Hebei Province, this study investigated the influence of parameters such as die cone angle and sizing belt length on billet deformation. The objective was to determine the optimal parameter combination to reduce the number of drawing passes and lower production costs. A finite element model for slider drawing and forming was established, and whether the stress at the minimum cross-section of the blank exceeded the tensile strength of the material during the drawing process was taken as the failure criterion for the orthogonal experiment. An orthogonal test analysis was conducted with four process parameters including die cone angle, sizing belt length, friction coefficient, and drawing speed, as influencing factors, to study the influence of process parameters on the stress at the minimum cross-sectional area of the billet during the forming process, as well as the primary and secondary relationships of interactions. The optimal parameter combination was obtained by combining range analysis (R-value method). Finally, the number of drawing passes was optimized through theoretical calculation, and the size structure of the die sizing belt was redesigned. The results indicate that the optimized drawing passes are reduced from five to three, and the influence degrees of various process parameters on the stress acting on the minimum cross-section of the billet are in the following order: die cone angle, sizing belt length, friction coefficient, and drawing speed. The optimal process parameters are as follows: the cone angle of the mold is 5 °, the sizing belt length is 10 mm, the friction coefficient is 0.06, and the drawing speed is 83 mm/s. The simulation results indicate that after optimization, the stress on the billet’s minimum cross-section remains below the tensile strength of the material (885 MPa) during all three-pass drawing passes, and the safety factor exceeds 1.25. Theoretical calculations show that the safety factors of the three-pass drawing all meet the requirements, which verifies the feasibility of the optimization scheme. The study reveals the primary and secondary relationships between key parameters of process and mold affecting drawing stress, providing theoretical reference for optimizing the multi-pass drawing process of rolling linear guide sliders.