This paper, the second of a series, deals with the function space $\mathcal H$ of all smooth K\"ahler metrics in any given $n$-dimensional, closed complex manifold $V, $ these metrics being restricted to a given, fixed, real cohomology class, called a polarization of $V$. This function space is equipped with a pre-Hilbert metric structure introduced by T. Mabuchi [Ma87], who also showed that, formally, this metric has nonpositive curvature. In the first paper of this series [chen991], the second author showed that the same space is a path length space. He also proved that $\mathcal H$ is geodesically convex in the sense that, for any two points of $\mathcal H, $ there is a unique geodesic path joining them, which is always length minimizing and of class $C^{1,1}$. This partially verifies two conjectures of Donaldson [Dona96] on the subject. In the present paper, we show first of all, that the space is, as expected, a path length space of nonpositive curvature in the sense of A. D. Aleksandrov. A second result is related to the theory of extremal K\"ahler metrics, namely that the gradient flow in $\mathcal H$ of the ``K energy" of $V$ has the property that it strictly decreases the length of all paths in $\mathcal H$, except those induced by one parameter families of holomorphic automorphisms of $M$.