Traditionally, a simple model, such as Gaussian distribution, describes the optical heterodyne detection echo signal of the target, which often leads to judgment errors. In this paper, we present a more accurate method to describe the probability distribution of the echo signal by use of polynomial fitting. The echo signals from a surface coating specimen of an armored car are measured, with the statistical analysis showing that the probability of detection is higher and the probability of false alarm is lower, when using the method of polynomial fitting to determine the decision threshold, rather than using a simple Gaussian distribution to determine it. The results show that by the method of polynomial fitting on the statistical histogram of the measured intermediate frequency signal, the precision of setting the detection threshold will be improved, and that the probability of detection rises by 6.02％ and the probability of false alarm falls by 7.7％. The results of this paper provide a quantitative reference for designing the rough surface echo heterodyne detection system and also provide a useful tool for evaluating the performance of the heterodyne detection system.
