The availability of laser radiation in the vacuum ultraviolet (VUV) wavelength range is important for the investigation of excited electronic states of atoms and molecules. The study of Rydberg states provides the possibility of unravelling the underlying processes of photoionization. In order to make studies of the fine and hyperfine structures in the VUV spectra of atoms and molecules feasible, high resolving powers are required. The short wavelength of VUV radiation and the difficulties that accompany its production render such studies an experimental challenge. This work summarizes the development, characterization and spectroscopic applications of a new narrow-bandwidth VUV laser system generating VUV radiation at a bandwidth of 55 MHz, five times less than that of previous tunable pulsed VUV laser systems. High-resolution measurements and analysis of the Rydberg spectra of xenon, krypton and the hydrogen molecule, revealing fine and hyperfine structures, isotope effects and details of the photoionization dynamics at an unprecedented level of precision demonstrate the versatility of this system.