The demand for high performance image processing systems, driven by everevolving image processing algorithms, has begun to grow immensely in recent years. For the implementation of such systems, the designer faces a vast number of choices and has to confront a set of conflicting constraints comprising performance, power consumption, and flexibility, to name just a few. In order to identify optimal solutions there is a prevailing need to constantly evaluate and quantify existing and emerging architectures using relevant, state-of-the-art image processing algorithms. This issue is especially apparent in advanced driver assistance systems, where for many relevant algorithms efficient implementations for demanding throughput constraints are unavailable on any architecture. One of these algorithms is semi-global matching (SGM) which enables 3D perception using inexpensive, passive stereo cameras.This work presents the design, optimization and analysis of high efficiency implementations in the hardware and software domain of the semi-global matching algorithm. The target is a heterogeneous set of architectures comprising a wide spectrum of current architectural concepts for image processing: two ASPs (one RISC processor and one VLIW processor), a GPU, a multi-core GPP, and a dedicated architecture mapped onto FPGA and standard cell ASIC. A design space exploration (DSE) framework is introduced to assess architectures quantitatively in terms of silicon area, energy consumption, and throughput performance. It contains a specifically designed FPGA-based SoC framework for image processing tasks enabling emulation-based analysis and rapid prototyping. Using the DSE framework, detailed analyses of all respective implementation options for each architecture (intra-architecture analysis) and across architectures (interarchitecture analysis) are performed. The SoC framework and FPGA results are further applied to compose a fully functional real-time stereo vision system.