RELAY-ASSISTED data transmission will play a crucial role in future wireless communication systems for achieving high data rates and good coverage in a cost-efficient and flexible manner. In cellular networks, for example, relay stations are predestined to efficiently extend the range of a base station and particularly to solve common coverage problems without the need to deploy a wired backhaul connection to the actual core network. Aside from that, relaying also represents a key enabling technology for achieving connectivity in wireless ad-hoc and sensor networks. The analysis and optimization of relay-assisted communication systems is, however, generally significantly more involved than for conventional point-to-point transmissions due to a considerably larger number of degrees of freedom and since multiple channels as well as the mode of operation of the involved relay stations have to be taken into account.In this thesis, we investigate and optimize wireless communication systems with so-called regenerative relaying by quantitatively assessing the corresponding error rate performance, improving the resource allocation as well as designing and analyzing novel transmission schemes. First of all, we focus on classical multihop systems with detect-and-forward relaying, where the involved relay stations always perform hard decisions on all received bits before forwarding them on the corresponding next hop. For such systems, we derive exact analytical expressions for the end-to-end bit and symbol error rates for several different modulation schemes and we propose various different approaches for optimizing these error rates by means of proper power allocation. Furthermore, we address the problem of performing adaptive modulation over such multihop systems, where we particularly design and evaluate several different approaches for adapting the used modulation scheme to the current channel conditions.In the second part of the thesis, we then consider cooperative diversity systems with decodeand-forward relaying, where encoded messages are completely decoded by the relay stations, re-encoded again and then eventually forwarded on the next hop. Cooperative diversity systems are able to exploit spatial diversity even if all involved nodes are equipped with a single antenna element only. In this work, we aim at further improving the performance of these systems by using multiple antennas at the source node (and possibly other nodes as well), thus combining the advantages of conventional multiple-input multiple-output (MIMO) transmission with those of cooperative diversity systems. In this regard, we consider three different classes of transmission schemes, namely transmit antenna selection, transmit antenna switching as well as transmit beamforming. For each of these schemes, we propose and analyze various different approaches based on different kinds of channel state information and we thoroughly evaluate and compare the performance that can be achieved with them in terms of the end-to-end information outage probability.