We introduce a framework that enables an agent to autonomously learn its own symbolic representation of a low-level, continuous environment. Propositional symbols are formalized as names for probability distributions, providing a natural means of dealing with uncertain representations and probabilistic plans. We determine the symbols that are sufficient for computing the probability with which a plan will succeed, and demonstrate the acquisition of a symbolic representation in a computer game domain.National Science Foundation (U.S.) (grant 1420927)United States. Office of Naval Research (grant N00014-14-1-0486)United States. Air Force. Office of Scientific Research (grant FA23861014135)United States. Army Research Office (grant W911NF1410433)MIT Intelligence Initiative