lecture31: Information Theory and Estimation

Topics covered

• Estimation
• Data Processing Inequality
• Sufficient Statistics
• Sufficient Statistic Example
• Minimal Sufficient Statistics
• Laplace's principle of indifference
• Differential Entropy
• Maximum Entropy
• Maximum Entropy Distributions
• Solution
• Example 1
• Example 2
• Example 3
• Applications %
• Parameter estimation %
• Estimating Entropy