Multicarrier modulation is the key technology for the physicallayer of current and future broadband communication systems. Filter bank based multicarrier systems (FBMC) offer an increased bandwidth efficiency and an improved spectral confinement compared to conventional orthogonal frequency division multiplexing (OFDM) with cyclic prefix (CP). The spectral shaping of the subbands leads to a beneficial robustness against narrowband interference, simplifies the suppression of carrier frequency offsets and Doppler effects and facilitates the dynamic spectrum access.We derive efficient, DFT-based implementations of filter bank structures and illustrate general aspects like the power spectral density, the peak-to-average power ratio and the perfect reconstruction condition.In the main part we develop a framework encompassing analytical solutions for efficient linear and nonlinear equalization schemes which minimize the mean square error (MSE) for the symbol decisions in O-QAM FBMC systems.Adaptive equalization algorithms tailored to the needs of O-QAM FBMC systems extend this framework of analytical minimum MSE schemes afterwards. The adaptive equalization schemes are based on the (normalized) least mean square algorithm (LMS) and the set-membership normalized LMS.Finally, comparisons with the state-of-the-art wireless communication standards 3GPP Long Term Evolution (LTE) and WiMAX, which are based on CP-OFDM, highlight the practical relevance of the proposed solutions.