Photometric redshift estimation is becoming an increasingly important technique, although the currently existing methods present several shortcomings which hinder their application. Most of those drawbacks are efficiently eliminated when Bayesian probability is consistently applied to this problem. The use of prior probabilities and Bayesian marginalization allows the inclusion of valuable information, e.g. the redshift distributions or the galaxy type mix, which is often ignored by other methods. In those cases when the a priori information is insufficient, it is shown how to `calibrate' the prior distributions, using even the data under consideration. There is an excellent agreement between the 108 HDF spectroscopic redshifts and the predictions of the method, with a rms error Delta z/(1+z_spec) = 0.08 up to z<6 and no systematic biases nor outliers. The results obtained are more reliable than those of standard techniques even when the latter include near-IR colors. The Bayesian formalism developed here can be generalized to deal with a wide range of problems which make use of photometric redshifts, e.g. the estimation of individual galaxy characteristics as the metallicity, dust content, etc., or the study of galaxy evolution and the cosmological parameters from large multicolor surveys. Finally, using Bayesian probability it is possible to develop an integrated statistical method for cluster mass reconstruction which simultaneously considers the information provided by gravitational lensing and photometric redshifts.