We present a new tractable quantum Rabi model for N-level atoms by extending the Z2 symmetry of the two-state Rabi model. The Hamiltonian is ZN symmetric and allows the parameters in the level separation terms to be complex while remaining hermitian. This latter property means that the new model is chiral, which makes it differ from any existing N-state Rabi models in the literature. The ZN symmetry provides partial diagonalization of the general Hamiltonian. The exact isolated (i.e. exceptional) energies of the model have the Rabi-like form but are N-fold degenerate. For the three-state case (N = 3), we obtain three transcendental functions whose zeros give the regular (i.e. non-exceptional) energies of the model.