Why not logistic regression uses multiplication instead of addition for error matrix?

Diansheng

This is a very basic question but I cannot could not find enough reasons to convince myself. Why must logistic regression use multiplication instead of addition for the likelihood function l(w)?

ilanman

Your question is more general than just joint likelihood for logistic regression. You're asking why we multiply probabilities instead of add them to represent a joint probability distribution. Two notes:

  1. This applies when we assume random variables are independent. Otherwise we need to calculate conditional probabilities using the chain rule of probability. You can look at wikipedia for more information.

  2. We multiply because that's how the joint distribution is defined. Here is a simple example:

Say we have two probability distributions:

X = 1, 2, 3,  each with probability 1/3
Y = 0 or 1,  each with probability 1/2

We want to calculate the joint likelihood function, L(X=x,Y=y), which is that X takes on values x and Y takes on values y.

For example, L(X=1,Y=0) = P(X=1) * P(Y=0) = 1/6. It wouldn't make sense to write P(X=1) + P(Y=0) = 1/3 + 1/2 = 5/6.

Now it's true that in maximum likelihood estimation, we only care about those values of some parameter, theta, which maximizes the likelihood function. In this case, we know that if theta maximizes L(X=x,Y=y) then the same theta will also maximize log L(X=x,Y=y). This is where you may have seen addition of probabilities come into play.

Hence we can take the log P(X=x,Y=y) = log P(X=x) + log P(Y=y)

In short

This could be summarized as "joint probabilities represent an AND". When X and Y are independent, P(X AND Y) = P(X,Y) = P(X)P(Y). Not to be confused with P(X OR Y) = P(X) + P(Y) - P(X,Y).

Let me know if this helps.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

Confusion matrix on regression logistic

tensorflow logistic regression matrix

Logistic regression confusion matrix

Why Matrix Addition is slower than Matrix-Vector Multiplication in Eigen?

Matrix addition/multiplication in numpy

Matrix multiplication and addition in R

While performing logistic regression i get the error : LinAlgError: Singular matrix

Why is logistic regression called regression?

TensorFlow multivariate linear regression placeholder and matrix multiplication error

Oracle: two Selection in one query with multiplication and addition ,why it shows error?

logistic regression with gradient descent error

Method for error dataset in Logistic Regression

Simple Logistic Regression Error in Python

Matrix multiplication-style addition in numpy

Tensorflow: nan during matrix multiplication and addition

i am coding for matrix multiplication and addition and i got this error 'TypeError: 'map' object is not subscriptable'. all answers are appreciated :)

How to choose multiplication monoid instead of addition monoid?

Rotation matrix multiplication error

Matrix Multiplication Error

Why is my logistic regression classifier in Tensorflow not learning?

logistic regression using statsmodels error in python

Error message in Firth's logistic regression

Error while running Logistic Regression in pyspark mlib

Logistic regression from scratch: error keeps increasing

Getting an error while training a logistic regression model

Calculating Standard Error of Coefficients for Logistic Regression in Spark

Contrasts error when running logistic regression

Error with cross validation and lasso regularization for logistic regression

python scikit learn logistic regression error