Recent progress in mobile and cloud computing coupled with the increase in data has resulted in a data-driven ecosystem that is making an impact in several domains of science and engineering. However, this data-driven ecosystem lacks protective measures for privacy resulting in regulations and behaviors that restrict data sharing. Augmenting the existing data-driven ecosystem with privacy preserving solutions could unlock the access to data silos, increasing the impact manifold. In this thesis, I discuss and identify gaps in some of the existing works and develop privacy preserving mechanisms for data analysis and distributed computation. At an abstract level, existing work in this domain includes federated learning, differential privacy, and encrypted computations. I describe the practical scenarios where all these approaches do not suffice due to their intrinsic computation infeasibility or suboptimal privacy-utility trade-off. This work augments such existing approaches by improving certain trade-offs and utilizing priors specific to the problem.S.M.