We analyze perturbative aspects of gauged matrix models, including those where classically the gauge symmetry is partially broken. Ghost fields play a crucial role in the Feynman rules for these vacua. We use this formalism to elucidate the fact that nonperturbative aspects of N=1 gauge theories can be computed systematically using perturbative techniques of matrix models, even if we do not possess an exact solution for the matrix model. As examples we show how the Seiberg-Witten solution for N=2 gauge theory, the Montonen-Olive modular invariance for N=1(*), and the superpotential for the Leigh-Strassler deformation of N=4 can be systematically computed in perturbation theory of the matrix model or gauge theory (even though in some of these cases an exact answer can also be obtained by summing up planar diagrams of matrix models).Accepted Manuscript