lecture03: Revision: Probability Theory

Topics covered

• Probability
• Probability Axioms
• Immediate Consequences
• Random Variables
• Random Variables: formal approach
• Cumulative Distribution Function (CDF)
• Probability Mass Function (PMF)
• Joint Distributions
• Example Distributions
• Laplace's principle of insufficient reason
• Probability Axioms (revisited)
• Independence
• Bayes' Law
• Law of Total Probability (reprise) %
• Joint Probability
• Probabilistic Chain Rule
• Expectation
• Expectations of functions
• Convex functions
• Jensen's Inequality
• Gibbs' Inequality
• Properties of Expectation
• Conditional Expectation
• Conditional Expectation as a RV
• Example 1: binary communications system
• Example 2: English letter frequencies
• Assignment