In a close analogy to matching data in Euclidean space, this monograph views parameter estimation as matching of the empirical distribution of data with a model-based distribution. Using an appealing Pythagorean-like geometry of the empirical and model distributions, the book brings a new solution to the problem of recursive estimation of non-Gaussian and nonlinear models which can be regarded as a specific approximation of Bayesian estimation. The cases of independent observations and controlled dynamic systems are considered in parallel; the former case giving initial insight into the latter case which is of primary interest to the control community. A number of examples illustrate the key concepts and tools used. This unique monograph follows some previous results on the Pythagorean theory of estimation in the literature (e.g., Chentsov, Csiszar and Amari) but extends the results to the case of controlled dynamic systems.