Positive definite kernels and their generalizations, as, e.g., the conditionally positive definite kernels, play an important role in various areas of mathematics. They are used in scattered data approximation as main ingredients of radial basis function methods, serve as models for variograms in statistical methods for spatial data and give rise to the famous kernel trick used in many methods of machine learning. A reason for the success of kernel-based methods lies in the intimate connection of positive definite kernels with reproducing kernel Hilbert spaces. While this is a well-studied connection, the corresponding relation of conditionally positive definite kernels and Pontryagin spaces attracted less attention. In this thesis, we want to shed some light on this connection. Furthermore, we extend the notion of conditionally positive definite functions to discrete hypergroups and give some applications to kernel design on graphs.