Machine learning/Statistics Fundamentals

Point estimation is the process of calculating a statistic from a sample dataset.

• A point estimator is any function for a sample.
• Therefore, any statistic is a point estimator

Methods of finding point estimations

• Method of Moments
• Maximum Likelihood Estimators
• Bayes Estimators
• Maximum a posteriori estimator

Bayes Estimators

Using the Bayesian rule$${\begin{aligned}P(\theta |D)&={\frac {P(D|\theta )P(\theta )}{P(D)}}\\&={\frac {P(D|\theta )P(\theta )}{\int P(D|\theta )P(\theta )d\theta }}\end{aligned}}$$Where $D=(X_{1},X_{2},\cdots ,X_{n})$ is the sample dataset.

$$P(D|\theta )=\prod _{i=1}^{n}P(X_{i}|\theta )$$