Machine Learning problems often admit a solution space that is not unique. When multiple feasible solutions exist, picking from a diverse, representative set may lead to better generalization and task-specific performance. While the emphasis of much of the literature has been on directly finding the `best' solution, we show that often a diverse set of near optimal solutions can be found which may be useful to practitioners and experts using machine learning models in decision making. This thesis investigates methods for obtaining a useful collection of solutions in specific models. Non-negative Matrix Factorization (NMF) is a popular data exploration tool and its Bayesian formulation is a promising approach for understanding uncertainty within this structure. We demonstrate that current approaches are lacking in the proper characterization of uncertainties and present novel techniques to provide model flexibility and improve the quality and speed of the inference. These techniques are applied to standard benchmark datasets for NMF as well as a curated medical dataset for understanding comorbidities in the Autism Spectrum Disorder (ASD). We show how a distinct collection of NMFs of nearly equal quality give rise to variability in interpretation of features and subsequent predictions. Finally, we present extensions of our diverse collection-based approach to the on-policy and off-policy Reinforcement Learning setting. Here, a completely new set of technical tools is required. In both on-policy and off-policy variants, we use diversity as a regularization feature in order to obtain a set of high-quality diverse policies. In addition to finding diverse policies in simulate-able multi-goal domains, we find a diverse set of policies designed to aid clinical decision making using ICU data for sepsis and hypotension management.Engineering and Applied Sciences - Applied Math