You may initiate a PR on this for review?

There already is a log_loss function in the metrics module of scikit-learn. Naively I'd assume this would also work for your case. Could you explain in a bit more detail why you can't use that function instead of adding a new one?

Thank you for your comment @betatim . I want to measure the goodness of fit of a logistic regression model. However, I think the log_loss function measures the difference between the predicted probabilities and the actual outcomes.

Alternatively, can we have a metric to directly calculate the Pseudo R-square?

If I look at the code of your log_likelihood(model, X, y) I think you can perform the same computation using the log_loss that already exists.

Thanks @betatim for your guidance. I'm new to the field so that was immensely helpful.

I used the following code for my case

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X, y)

y_pred_proba = model.predict_proba(X)[:, 1]

from sklearn.metrics import log_loss

ll_null = log_loss(y, [y.mean()] * len(y))
ll_model = log_loss(y, y_pred_proba)

pseudo_r2 = 1 - ll_model / ll_null

However, I was wondering if we have an inbuilt function to compute pseudo R-squared, if not what might be the reason of not having it?

However, I was wondering if we have an inbuilt function to compute pseudo R-squared, if not what might be the reason of not having it?

I don't know the answer to that. I think there isn't a lot of "goodness of fit" tooling in scikit-learn because there are other libraries that do that.

I think the reason for this may be that there are different approaches to computing pseudo R-squared. Moreover, different pseudo-R-squared measures can have different interpretations and assumptions, which may not always apply in a particular context.

However, since sci-kit learn already has r2_score, IMO this library should also have pseudo R-squared for logistic regression (allowing the user to choose from indices) for consistency.
What do you think?

I'd be more than happy to work on it.

Metrics in the sklearn.metrics must be general enough to work with all models and not just linear models. In practice, I only seen pseudo R-squared applied to linear models. @AnkurSingh282000 Have you come across literature that applies pseudo R-squared to other models such as tree based models?

@lorentzenchr What do you think of providing a pseudo R-squared to the models in sklearn.linear_model?

Such a metric d2_log_loss is proposed in #20943. It‘s only a matter of someone implementing it.

I want to measure the goodness of fit of a logistic regression model. However, I think the log_loss function measures the difference between the predicted probabilities and the actual outcomes.

Note that a „good“ GOF does exactly that: it measures some kind of „distance“ between observations and predictions.

I think we can close then?

@lorentzenchr Okay that makes sense. I'm closing this issue as a duplicate of #20943.

@AnkurSingh282000 Have you come across literature that applies pseudo R-squared to other models such as tree based models?

Thanks @thomasjpfan for bringing this up and your time. I have also not seen this metric being applied to any tree based model. However I have a small concern.

As @lorentzenchr suggested-

Such a metric d2_log_loss is proposed in #20943.

I cannot locate any literature about it (I also noticed other people facing the same issue). If you could help me a little I could take shot at it.
Thanks!

@SinghAnkur28 There is literature and if you are still interested, I can point you to it.
I would very much like if you could give it a shot. I can assist in case you get stuck.