In this paper, energy efficiency of relay-assisted millimeter wave (mmWave) cellular networks with Poisson Point Process (PPP) distributed base stations (BSs) and relay stations (RSs) is analyzed using tools from stochastic geometry. The distinguishing features of mmWave communications such as directional beamforming and having different path loss laws for line-of-sight (LOS) and non-line-of-sight (NLOS) links are incorporated into the energy efficiency analysis. Following the description of the system model for mmWave cellular networks, coverage probabilities are computed for each link. Subsequently, average power consumption of BSs and RSs are modeled and energy efficiency is determined in terms of system parameters. Energy efficiency in the presence of beamforming alignment errors is also investigated to get insight on the performance in practical scenarios. Finally, the impact of BS and RS densities, antenna gains, main lobe beam widths, LOS interference range, and alignment errors on the energy efficiency is analyzed via numerical results.