We provide a principle algorithm for maximum entropy constrained exploration on the distributional manifold.