Aiming at fault diagnosis problems caused by complex machinery parts, serious background noises and the application limitations of traditional blind signal processing algorithm to the mechanical acoustic signal processing, a failure acoustic diagnosis based on reference signal frequency domain semi-blind extraction is proposed. Key technologies are introduced: Based on frequency-domain blind deconvolution algorithm, the artificial fish swarm algorithm which is good for global optimization is used to construct improved multi-scale morphological filters which is applicable to mechanical failure in order to weaken the background noises; combining the structural parameters of parts to build a reference signal, complex components blind separation is carried out on the signals after noise reduction paragraph by paragraph by reference signal unit semi-blind extraction algorithm; then the improved KL-distance of complex independent components is employed as distance measure to resolve the permutation, and finally the mechanical fault characteristic signals are extracted and separated. The actual acoustic diagnosis of rolling bearing fault in sound field environment results proves the effectiveness of this algorithm.