An anti-holomorphic multiplication by the integers $\mathcal O_d$ of a quadratic imaginary number field, on a principally polarized complex abelian variety $A_{\mathbb C}$ is an action of $\mathcal O_d$ on $A_{\mathbb C}$ such that the purely imaginary elements act in an anti-holomorphic manner. The coarse moduli space $X_{\mathbb R}$ of such $A$ (with appropriate level structure) is shown to consist of finitely many isomorphic connected components, each of which is an arithmetic quotient of the quaternionic Siegel space, that is, the symmetric space for the complex symplectic group. The moduli space $X_{\mathbb R}$ is also identified as the fixed point set of a certain anti-holomorphic involution $\tau$ on the complex points $X_{\mathbb C}$ of the Siegel moduli space of all principally polarized abelian varieties (with appropriate level structure). The Siegel moduli space $X_{\mathbb C}$ admits a certain rational structure for which the involution $\tau$ is rationally defined. So the space $X_{\mathbb R}$ admits the structure of a rationally defined, real algebraic variety.