The technology of adaptive equalization can effectively overcome the inter-symbol interference caused by dispersion of optical fiber channel and nonlinearity of optical fiber. But the traditional adaptive equalization algorithm has some disadvantages, such as slow convergence, poor stability and bad equalization result. These disadvantages restrict the application of adaptive equalizer in the high-speed optical fiber communication systems. An adaptive equalization algorithm based on QPSO algorithm is presented. Simulation experiments show that the quantum-behaved particle swarm optimization algorithm (QPSO) has many advantages, such as fast convergence, high calculation accuracy, etc. The equalization result of QPSO algorithm is better than the traditional control algorithm.