Metal-halide perovskites feature very low deep-defect densities, thereby enabling high operating voltages at the solar cell level. Here, by precise extraction of their absorption spectra, we find that the low deep-defect density is unaffected when cations such as Cs+ and Rb+ are added during the perovskite synthesis. By comparing single crystals and polycrystalline thin films of methylammonium lead iodide/bromide, we find these defects to be predominantly localized at surfaces and grain boundaries. Furthermore, generally, for the most important photovoltaic materials, we demonstrate a strong correlation between their Urbach energy and open-circuit voltage deficiency at the solar cell level. Through external quantum yield photoluminescence efficiency measurements, we explain these results as a consequence of nonradiative open-circuit voltage losses in the solar cell. Finally, we define practical power conversion efficiency limits of solar cells by taking into account the Urbach energy.