Generative Model vs. Discriminative Model

Generative Model

Given the observations $X$ and labels $Y$, the model specifies a joint probability $P(X,Y)$. It is a full probabilistic model for all variables $X$ and $Y$, for which the main goal is to explicitly model the actual distribution. In other words, generative methods model class-conditional PDFs and prior probabilities.

Predicting the label $Y$ from observations $X$ targets at solving the following:

$$f(x) = \argmax↙{Y} \ P(X,Y) = \argmax↙{Y} \ P(X|Y)·P(Y)$$

Discriminative Model

Given the observations $X$ and labels $Y$, the model specifies a conditional probability $P(Y | X)$. It is a partial probabilistic model only for target labels $Y$ conditional on observations $X$, for which the main goal is to model the decision boundary of the labels without needing to know the relationships between the variables. In other words, discriminative methods directly estimate posterior probabilities.

Predicting the label $Y$ from observations $X$ targets at solving the following:

$$f(x) = \argmax↙{Y} \ P(Y|X)$$

References

Open Comments