This treatise is written on an elementary level; in more difficult cases the final formulas are provided without proof. Nevertheless, it was impossible to leave out integrals and I also had to differentiate an integral with respect to a parameter. I include many examples taken from the history of probability and hope that my subject has thus become lively. I especially quote Karl Pearson's (1978, p. 1) repentant confession: I do feel how wrongful it was to work for so many years at statistics and neglect its history. In spite of a few mistakes, his book deserves serious attention. Thus, in § 4.1.1 I criticize his opinion about Jakob Bernoulli. I have devoted much attention to the notion of probability which fully conforms to Langevin's statement (1913/1914, p. 3): Dans toutes ces questions [in the kinetic theory] la difficulté principale est, comme nous le verrons, de donner une définition correcte et claire de la probabilité. Note however that correct definition sounds strangely.