We show that tri-layer structures combining epsilon-negative and magneto-optical material layers can exhibit unidirectional resonant photon tunneling phenomena that can discriminate between circularly polarized (CP) waves of given handedness impinging from opposite directions, or between CP waves with different handedness impinging from the same direction. This physical principle, which can also be interpreted in terms of a Fabry-Perot-type resonance, may be utilized to design compact optical isolators for CP waves. Within this framework, we derive simple analytical conditions and design formulae, and quantitatively assess the isolation performance, also taking into account the unavoidable imperfections and nonidealities.