A Review of Neural Network Accuracy Improvement Using Averaged Outputs

Abstract

we studied the application of Richardson extrapolation to improving the accuracy of neural network computations. We analyzed the mathematical framework of the Richardson method, including the derivation of the relevant formulas, asymptotic error expansion, recursive extrapolation scheme, and convergence estimates. We developed a mathematical model of a neural network, formalized the main sources of computational error, and implemented an extrapolation-based procedure for reducing these errors. We then conducted a series of computational experiments on test problems involving approximation, numerical integration, and classification in the presence of noise. The results showed that Richardson extrapolation improved computational accuracy by a factor of 3.5 compared with conventional approaches at the same computational cost and increased the convergence order. We also compared Richardson extrapolation with alternative accuracy-enhancement methods. The comparison showed that Richardson extrapolation provided a more favorable trade-off between accuracy and computational cost.