Logical Foundation of Neural Networks as Categorical Splices, Axiomatics, Dualities, and Applications

Abstract

we developed a formal first-order axiomatic theory of neural networks with arbitrary topologies within classical logic. The theory is a special case of the author’s axiomatic framework for categorical systems, which generalizes and formalizes P. K. Anokhin’s theory of functional systems, developed in the 1930s and, as N. Wiener acknowledged after meeting P. K. Anokhin in Moscow in 1961, incorporating the fundamental principles of cybernetics, including feedback. We identified and described three types of dualities and proved the corresponding duality principles. We applied these results to justify and generalize S. Osowski’s well-known formulas for error backpropagation to neural networks with arbitrary topologies. A scientific theory reaches a mature stage of development when its fundamental principles can be expressed as a system of postulates and axioms. The proposed axiomatic theory provides such a formal foundation for neural networks and, through its systemic basis, contributes to the formalization of the foundations of cybernetics and artificial intelligence methods based on neural networks.